[2025-09-16T17:00:01.382871] [INFO ] [MAIN ] Using last completed trading day: 2025-09-16 [2025-09-16T17:00:01.383276] [INFO ] [MAIN ] Starting TDM symbol refresh process. [2025-09-16T17:00:01.383369] [INFO ] [MAIN ] Fetching latest NYSE symbols list. [2025-09-16T17:00:01.383408] [DEBUG] [SYMBOL ] Starting fetch of NYSE Symbols. [2025-09-16T17:00:03.029584] [DEBUG] [SYMBOL ] Found 2421 valid NYSE symbols. [2025-09-16T17:00:03.029791] [INFO ] [SYMBOL ] Completed fetch of NYSE Symbols. [2025-09-16T17:00:03.029990] [INFO ] [MAIN ] Filtered out 54 blacklisted symbols. [2025-09-16T17:00:03.030656] [INFO ] [LOCAL ] Symbols cache updated with 2369 symbols. [2025-09-16T17:00:03.030705] [INFO ] [MAIN ] Successfully refreshed symbols list. [2025-09-16T17:00:03.030737] [INFO ] [MAIN ] Finished TDM symbol refresh process. [2025-09-16T17:00:03.030797] [INFO ] [MAIN ] Starting TDM stale symbol pruning process. [2025-09-16T17:00:03.030893] [INFO ] [LOCAL ] Reading cached symbols list. [2025-09-16T17:00:03.031275] [DEBUG] [LOCAL ] Found 2369 symbols in cached list. [2025-09-16T17:00:03.031310] [INFO ] [LOCAL ] Completed reading cached symbols list. [2025-09-16T17:00:03.031475] [INFO ] [MAIN ] Including 54 blacklisted symbols for pruning. [2025-09-16T17:00:03.031511] [INFO ] [MAIN ] Pruning data for symbols not in current list of 2369 valid symbols. [2025-09-16T17:00:03.031543] [INFO ] [LOCAL ] Starting pruning of local symbol history data for inactive/invalid symbols. [2025-09-16T17:00:03.031573] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/primary_storage' [2025-09-16T17:00:03.060362] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/derivative_storage' [2025-09-16T17:00:03.085626] [INFO ] [LOCAL ] Deleted 0 obsolete data files. [2025-09-16T17:00:03.085822] [INFO ] [LOCAL ] Completed pruning of unlisted local symbol history data. [2025-09-16T17:00:03.085863] [INFO ] [MAIN ] Successfully pruned stale symbol data. [2025-09-16T17:00:03.085894] [INFO ] [MAIN ] Finished TDM stale symbol pruning process. [2025-09-16T17:00:03.086305] [WARN ] [STATE ] State set to: RUNNING [2025-09-16T17:00:03.086488] [INFO ] [LOCAL ] Reading cached symbols list. [2025-09-16T17:00:03.086926] [DEBUG] [LOCAL ] Found 2369 symbols in cached list. [2025-09-16T17:00:03.086975] [INFO ] [LOCAL ] Completed reading cached symbols list. [2025-09-16T17:00:03.087008] [INFO ] [MAIN ] Starting incremental update for 2369 symbols. [2025-09-16T17:00:03.087037] [INFO ] [MAIN ] Running in parallel mode [2025-09-16T17:00:03.088349] [INFO ] [MAIN ] Starting parallel primary update with 4 worker threads for 2369 symbols [2025-09-16T17:00:03.623978] [INFO ] [MAIN ] AA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:03.692446] [DEBUG] [LOCAL ] AA: No existing data file found in primary storage. [2025-09-16T17:00:03.830279] [WARN ] [MAIN ] AA: No existing data found, performing full refresh instead. [2025-09-16T17:00:03.849859] [INFO ] [MAIN ] AA: Wiping existing primary data before full refresh. [2025-09-16T17:00:03.849988] [INFO ] [LOCAL ] AA: Starting deletion of primary data. [2025-09-16T17:00:03.828055] [INFO ] [MAIN ] AAM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:03.834283] [INFO ] [MAIN ] AACT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:03.853543] [DEBUG] [LOCAL ] AA: No primary data file found to delete. [2025-09-16T17:00:03.853360] [INFO ] [MAIN ] A: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:03.853806] [DEBUG] [LOCAL ] AAM: No existing data file found in primary storage. [2025-09-16T17:00:03.853867] [INFO ] [MAIN ] AA: Starting full primary refresh. [2025-09-16T17:00:03.854798] [WARN ] [MAIN ] AAM: No existing data found, performing full refresh instead. [2025-09-16T17:00:03.854747] [DEBUG] [LOCAL ] A: No existing data file found in primary storage. [2025-09-16T17:00:03.855540] [WARN ] [MAIN ] A: No existing data found, performing full refresh instead. [2025-09-16T17:00:03.883160] [INFO ] [MAIN ] AAM: Wiping existing primary data before full refresh. [2025-09-16T17:00:03.883314] [INFO ] [LOCAL ] AAM: Starting deletion of primary data. [2025-09-16T17:00:03.889982] [DEBUG] [LOCAL ] AAM: No primary data file found to delete. [2025-09-16T17:00:03.900451] [INFO ] [MAIN ] AAM: Starting full primary refresh. [2025-09-16T17:00:03.889836] [INFO ] [MAIN ] A: Wiping existing primary data before full refresh. [2025-09-16T17:00:03.940916] [INFO ] [LOCAL ] A: Starting deletion of primary data. [2025-09-16T17:00:03.971745] [DEBUG] [LOCAL ] A: No primary data file found to delete. [2025-09-16T17:00:03.998680] [INFO ] [MAIN ] A: Starting full primary refresh. [2025-09-16T17:00:04.031871] [DEBUG] [LOCAL ] AACT: Loaded 7894 existing records from primary storage. [2025-09-16T17:00:04.102408] [INFO ] [MAIN ] AACT: Loaded 7894 existing records. [2025-09-16T17:00:05.010109] [WARN ] [MAIN ] AACT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:05.135169] [INFO ] [REMOTE ] AAM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:05.174767] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-02. [2025-09-16T17:00:05.154221] [INFO ] [MAIN ] AACT: Wiping existing primary data before full refresh. [2025-09-16T17:00:05.230167] [INFO ] [REMOTE ] A: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:05.243867] [INFO ] [REMOTE ] A: Fetching primary data for 2025-02. [2025-09-16T17:00:05.266469] [INFO ] [LOCAL ] AACT: Starting deletion of primary data. [2025-09-16T17:00:05.272132] [INFO ] [REMOTE ] AA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:05.273581] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-02. [2025-09-16T17:00:05.274252] [DEBUG] [LOCAL ] AACT: Deleted primary data file. [2025-09-16T17:00:05.274347] [INFO ] [MAIN ] AACT: Starting full primary refresh. [2025-09-16T17:00:05.658513] [INFO ] [REMOTE ] AACT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:05.658924] [INFO ] [REMOTE ] AACT: Fetching primary data for 2025-02. [2025-09-16T17:00:05.994792] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T17:00:05.995879] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-03. [2025-09-16T17:00:06.311637] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-09-16T17:00:06.312677] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-04. [2025-09-16T17:00:06.423234] [INFO ] [REMOTE ] AACT: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T17:00:06.424194] [INFO ] [REMOTE ] AACT: Fetching primary data for 2025-03. [2025-09-16T17:00:06.610501] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-09-16T17:00:06.612048] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-05. [2025-09-16T17:00:06.876113] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-05 (0.26 seconds). [2025-09-16T17:00:06.876995] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-06. [2025-09-16T17:00:07.071122] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-09-16T17:00:07.081781] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-03. [2025-09-16T17:00:07.168021] [INFO ] [REMOTE ] AACT: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-09-16T17:00:07.169542] [INFO ] [REMOTE ] AACT: Fetching primary data for 2025-04. [2025-09-16T17:00:07.200854] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-06 (0.32 seconds). [2025-09-16T17:00:07.201804] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-07. [2025-09-16T17:00:07.250545] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-09-16T17:00:07.261717] [INFO ] [REMOTE ] A: Fetching primary data for 2025-03. [2025-09-16T17:00:07.574550] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-09-16T17:00:07.575555] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-08. [2025-09-16T17:00:07.963459] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-09-16T17:00:07.964708] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-09. [2025-09-16T17:00:08.009408] [INFO ] [REMOTE ] AACT: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-09-16T17:00:08.011985] [INFO ] [REMOTE ] AACT: Fetching primary data for 2025-05. [2025-09-16T17:00:08.586453] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-09-16T17:00:08.587161] [INFO ] [REMOTE ] AAM: Full fetch complete (703 records). [2025-09-16T17:00:08.587262] [DEBUG] [VALIDATOR ] AAM: Validating primary data. [2025-09-16T17:00:08.587685] [FATAL] [VALIDATOR ] AAM: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:00:08.587776] [FATAL] [MAIN ] AAM: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:00:08.587824] [WARN ] [MAIN ] AAM: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:00:08.587882] [INFO ] [LOCAL ] AAM: Starting deletion of primary data. [2025-09-16T17:00:08.587943] [DEBUG] [LOCAL ] AAM: No primary data file found to delete. [2025-09-16T17:00:08.587979] [INFO ] [LOCAL ] AAM: Starting deletion of derivative data. [2025-09-16T17:00:08.588019] [DEBUG] [LOCAL ] AAM: No derivative data file found to delete. [2025-09-16T17:00:08.588076] [INFO ] [MAIN ] Update Progress: 1/2369 symbols completed. AAM took 5.4s. Estimated remaining: 3h 37m 6s. Rate: 10.9 symbols/min [2025-09-16T17:00:08.762893] [INFO ] [MAIN ] AAMI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:08.763088] [DEBUG] [LOCAL ] AAMI: No existing data file found in primary storage. [2025-09-16T17:00:08.763126] [WARN ] [MAIN ] AAMI: No existing data found, performing full refresh instead. [2025-09-16T17:00:08.763253] [INFO ] [MAIN ] AAMI: Wiping existing primary data before full refresh. [2025-09-16T17:00:08.763294] [INFO ] [LOCAL ] AAMI: Starting deletion of primary data. [2025-09-16T17:00:08.763336] [DEBUG] [LOCAL ] AAMI: No primary data file found to delete. [2025-09-16T17:00:08.763369] [INFO ] [MAIN ] AAMI: Starting full primary refresh. [2025-09-16T17:00:08.886749] [INFO ] [REMOTE ] AACT: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-09-16T17:00:08.977307] [INFO ] [REMOTE ] AACT: Fetching primary data for 2025-06. [2025-09-16T17:00:09.141235] [INFO ] [REMOTE ] AAMI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:09.141917] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-02. [2025-09-16T17:00:09.185870] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-09-16T17:00:09.197624] [INFO ] [REMOTE ] A: Fetching primary data for 2025-04. [2025-09-16T17:00:09.229321] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-03 (2.15 seconds). [2025-09-16T17:00:09.240779] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-04. [2025-09-16T17:00:09.930856] [INFO ] [REMOTE ] AACT: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-09-16T17:00:09.935277] [INFO ] [REMOTE ] AACT: Fetching primary data for 2025-07. [2025-09-16T17:00:10.162449] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T17:00:10.166666] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-03. [2025-09-16T17:00:10.606622] [INFO ] [REMOTE ] AACT: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-09-16T17:00:10.610906] [INFO ] [REMOTE ] AACT: Fetching primary data for 2025-08. [2025-09-16T17:00:10.976818] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-09-16T17:00:10.981795] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-04. [2025-09-16T17:00:11.090391] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-09-16T17:00:11.102426] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-05. [2025-09-16T17:00:11.227557] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-09-16T17:00:11.239348] [INFO ] [REMOTE ] A: Fetching primary data for 2025-05. [2025-09-16T17:00:12.290380] [INFO ] [REMOTE ] AACT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T17:00:12.294428] [INFO ] [REMOTE ] AACT: Fetching primary data for 2025-09. [2025-09-16T17:00:12.432272] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T17:00:12.443078] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-05. [2025-09-16T17:00:12.903348] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-09-16T17:00:12.921683] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-06. [2025-09-16T17:00:13.132980] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-09-16T17:00:13.144989] [INFO ] [REMOTE ] A: Fetching primary data for 2025-06. [2025-09-16T17:00:13.684420] [INFO ] [REMOTE ] AACT: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T17:00:13.686960] [INFO ] [REMOTE ] AACT: Full fetch complete (9645 records). [2025-09-16T17:00:13.687065] [DEBUG] [VALIDATOR ] AACT: Validating primary data. [2025-09-16T17:00:14.070743] [DEBUG] [VALIDATOR ] AACT: All expected trading days are present. [2025-09-16T17:00:14.070833] [DEBUG] [VALIDATOR ] AACT: Primary data validation passed. [2025-09-16T17:00:14.070993] [INFO ] [MAIN ] AACT: Fetched primary data is valid. [2025-09-16T17:00:14.071050] [DEBUG] [LOCAL ] AACT: Saving primary data. [2025-09-16T17:00:14.071552] [DEBUG] [LOCAL ] AACT: Trimmed 9645 records to 8064 (keeping 150 days of data) [2025-09-16T17:00:14.124476] [DEBUG] [LOCAL ] AACT: Primary data saved (8064 records). [2025-09-16T17:00:14.124606] [INFO ] [MAIN ] AACT: Full history saved. [2025-09-16T17:00:14.124723] [INFO ] [MAIN ] Update Progress: 2/2369 symbols completed. AACT took 11.0s. Estimated remaining: 3h 37m 43s. Rate: 10.9 symbols/min [2025-09-16T17:00:14.299348] [INFO ] [MAIN ] AAP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:14.299685] [DEBUG] [LOCAL ] AAP: No existing data file found in primary storage. [2025-09-16T17:00:14.299809] [WARN ] [MAIN ] AAP: No existing data found, performing full refresh instead. [2025-09-16T17:00:14.300071] [INFO ] [MAIN ] AAP: Wiping existing primary data before full refresh. [2025-09-16T17:00:14.300208] [INFO ] [LOCAL ] AAP: Starting deletion of primary data. [2025-09-16T17:00:14.300342] [DEBUG] [LOCAL ] AAP: No primary data file found to delete. [2025-09-16T17:00:14.300413] [INFO ] [MAIN ] AAP: Starting full primary refresh. [2025-09-16T17:00:14.398265] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-09-16T17:00:14.431512] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-06. [2025-09-16T17:00:14.685870] [INFO ] [REMOTE ] AAP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:14.686194] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-02. [2025-09-16T17:00:14.768608] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T17:00:14.779037] [INFO ] [REMOTE ] A: Fetching primary data for 2025-07. [2025-09-16T17:00:14.808890] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-09-16T17:00:14.820132] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-07. [2025-09-16T17:00:15.201215] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T17:00:15.206227] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-07. [2025-09-16T17:00:16.059045] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-09-16T17:00:16.065059] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-08. [2025-09-16T17:00:16.464126] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T17:00:16.475311] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-03. [2025-09-16T17:00:16.719666] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-09-16T17:00:16.731337] [INFO ] [REMOTE ] A: Fetching primary data for 2025-08. [2025-09-16T17:00:16.836841] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-09-16T17:00:16.848845] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-08. [2025-09-16T17:00:17.064634] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-09-16T17:00:17.070801] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-09. [2025-09-16T17:00:18.315110] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T17:00:18.326352] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-04. [2025-09-16T17:00:18.391193] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T17:00:18.403464] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-09. [2025-09-16T17:00:18.614823] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T17:00:18.618727] [INFO ] [REMOTE ] AAMI: Full fetch complete (25900 records). [2025-09-16T17:00:18.618859] [DEBUG] [VALIDATOR ] AAMI: Validating primary data. [2025-09-16T17:00:18.619447] [FATAL] [VALIDATOR ] AAMI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:00:18.619527] [FATAL] [MAIN ] AAMI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:00:18.619566] [WARN ] [MAIN ] AAMI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:00:18.619614] [INFO ] [LOCAL ] AAMI: Starting deletion of primary data. [2025-09-16T17:00:18.619698] [DEBUG] [LOCAL ] AAMI: No primary data file found to delete. [2025-09-16T17:00:18.619738] [INFO ] [LOCAL ] AAMI: Starting deletion of derivative data. [2025-09-16T17:00:18.619779] [DEBUG] [LOCAL ] AAMI: No derivative data file found to delete. [2025-09-16T17:00:18.619846] [INFO ] [MAIN ] Update Progress: 3/2369 symbols completed. AAMI took 10.0s. Estimated remaining: 3h 24m 10s. Rate: 11.6 symbols/min [2025-09-16T17:00:18.799498] [INFO ] [MAIN ] AAT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:18.799719] [DEBUG] [LOCAL ] AAT: No existing data file found in primary storage. [2025-09-16T17:00:18.799761] [WARN ] [MAIN ] AAT: No existing data found, performing full refresh instead. [2025-09-16T17:00:18.799885] [INFO ] [MAIN ] AAT: Wiping existing primary data before full refresh. [2025-09-16T17:00:18.799926] [INFO ] [LOCAL ] AAT: Starting deletion of primary data. [2025-09-16T17:00:18.799967] [DEBUG] [LOCAL ] AAT: No primary data file found to delete. [2025-09-16T17:00:18.800000] [INFO ] [MAIN ] AAT: Starting full primary refresh. [2025-09-16T17:00:19.178855] [INFO ] [REMOTE ] AAT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:19.179298] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-02. [2025-09-16T17:00:19.239989] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-08 (2.51 seconds). [2025-09-16T17:00:19.250591] [INFO ] [REMOTE ] A: Fetching primary data for 2025-09. [2025-09-16T17:00:20.124592] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-09-16T17:00:20.135687] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-05. [2025-09-16T17:00:20.167192] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T17:00:20.179034] [INFO ] [REMOTE ] AA: Full fetch complete (58254 records). [2025-09-16T17:00:20.179158] [DEBUG] [VALIDATOR ] AA: Validating primary data. [2025-09-16T17:00:20.578056] [DEBUG] [VALIDATOR ] AA: All expected trading days are present. [2025-09-16T17:00:20.578257] [DEBUG] [VALIDATOR ] AA: Primary data validation passed. [2025-09-16T17:00:20.578858] [INFO ] [MAIN ] AA: Fetched primary data is valid. [2025-09-16T17:00:20.578951] [DEBUG] [LOCAL ] AA: Saving primary data. [2025-09-16T17:00:20.596380] [DEBUG] [LOCAL ] AA: Trimmed 58254 records to 39972 (keeping 150 days of data) [2025-09-16T17:00:20.597454] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T17:00:20.604360] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-03. [2025-09-16T17:00:20.733853] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T17:00:20.750622] [INFO ] [REMOTE ] A: Full fetch complete (55297 records). [2025-09-16T17:00:20.757571] [DEBUG] [VALIDATOR ] A: Validating primary data. [2025-09-16T17:00:21.304699] [DEBUG] [VALIDATOR ] A: All expected trading days are present. [2025-09-16T17:00:21.305522] [DEBUG] [VALIDATOR ] A: Primary data validation passed. [2025-09-16T17:00:21.306257] [INFO ] [MAIN ] A: Fetched primary data is valid. [2025-09-16T17:00:21.306801] [DEBUG] [LOCAL ] A: Saving primary data. [2025-09-16T17:00:21.312218] [DEBUG] [LOCAL ] A: Trimmed 55297 records to 37747 (keeping 150 days of data) [2025-09-16T17:00:21.451512] [DEBUG] [LOCAL ] AA: Primary data saved (39972 records). [2025-09-16T17:00:21.452572] [INFO ] [MAIN ] AA: Full history saved. [2025-09-16T17:00:21.453181] [INFO ] [MAIN ] Update Progress: 4/2369 symbols completed. AA took 18.4s. Estimated remaining: 3h 0m 59s. Rate: 13.1 symbols/min [2025-09-16T17:00:21.654317] [INFO ] [MAIN ] AAUC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:21.655850] [DEBUG] [LOCAL ] AAUC: No existing data file found in primary storage. [2025-09-16T17:00:21.656227] [WARN ] [MAIN ] AAUC: No existing data found, performing full refresh instead. [2025-09-16T17:00:21.660989] [INFO ] [MAIN ] AAUC: Wiping existing primary data before full refresh. [2025-09-16T17:00:21.667374] [INFO ] [LOCAL ] AAUC: Starting deletion of primary data. [2025-09-16T17:00:21.667469] [DEBUG] [LOCAL ] AAUC: No primary data file found to delete. [2025-09-16T17:00:21.667509] [INFO ] [MAIN ] AAUC: Starting full primary refresh. [2025-09-16T17:00:22.148095] [INFO ] [REMOTE ] AAUC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:22.150117] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-02. [2025-09-16T17:00:22.172967] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-09-16T17:00:22.181983] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-04. [2025-09-16T17:00:22.221116] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-05 (2.09 seconds). [2025-09-16T17:00:22.234850] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-06. [2025-09-16T17:00:22.237664] [DEBUG] [LOCAL ] A: Primary data saved (37747 records). [2025-09-16T17:00:22.237783] [INFO ] [MAIN ] A: Full history saved. [2025-09-16T17:00:22.237878] [INFO ] [MAIN ] Update Progress: 5/2369 symbols completed. A took 19.1s. Estimated remaining: 2h 30m 54s. Rate: 15.7 symbols/min [2025-09-16T17:00:22.397240] [DEBUG] [REMOTE ] AAUC: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:00:22.416553] [INFO ] [MAIN ] AB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:22.416728] [WARN ] [REMOTE ] AAUC: No data returned for month 2025-02 [2025-09-16T17:00:22.417789] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-03. [2025-09-16T17:00:22.435240] [DEBUG] [LOCAL ] AB: Loaded 17717 existing records from primary storage. [2025-09-16T17:00:22.435383] [INFO ] [MAIN ] AB: Loaded 17717 existing records. [2025-09-16T17:00:22.612779] [WARN ] [MAIN ] AB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:22.613064] [INFO ] [MAIN ] AB: Wiping existing primary data before full refresh. [2025-09-16T17:00:22.613115] [INFO ] [LOCAL ] AB: Starting deletion of primary data. [2025-09-16T17:00:22.613762] [DEBUG] [LOCAL ] AB: Deleted primary data file. [2025-09-16T17:00:22.613813] [INFO ] [MAIN ] AB: Starting full primary refresh. [2025-09-16T17:00:22.694328] [DEBUG] [REMOTE ] AAUC: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:00:22.715830] [WARN ] [REMOTE ] AAUC: No data returned for month 2025-03 [2025-09-16T17:00:22.731933] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-04. [2025-09-16T17:00:22.974135] [INFO ] [REMOTE ] AB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:22.974512] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-02. [2025-09-16T17:00:23.050436] [DEBUG] [REMOTE ] AAUC: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:00:23.050756] [WARN ] [REMOTE ] AAUC: No data returned for month 2025-04 [2025-09-16T17:00:23.051250] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-05. [2025-09-16T17:00:23.259554] [DEBUG] [REMOTE ] AAUC: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:00:23.259908] [WARN ] [REMOTE ] AAUC: No data returned for month 2025-05 [2025-09-16T17:00:23.260362] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-06. [2025-09-16T17:00:23.574179] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T17:00:23.582597] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-05. [2025-09-16T17:00:24.017743] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-09-16T17:00:24.029503] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-07. [2025-09-16T17:00:24.190725] [INFO ] [REMOTE ] AAUC: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-09-16T17:00:24.192919] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-07. [2025-09-16T17:00:24.558959] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-09-16T17:00:24.566250] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-06. [2025-09-16T17:00:24.733851] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-09-16T17:00:24.742589] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-03. [2025-09-16T17:00:24.952821] [INFO ] [REMOTE ] AAUC: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-09-16T17:00:24.955700] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-08. [2025-09-16T17:00:25.572755] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T17:00:25.588425] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-07. [2025-09-16T17:00:25.590167] [INFO ] [REMOTE ] AAUC: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-09-16T17:00:25.594361] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-09. [2025-09-16T17:00:25.823405] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-09-16T17:00:25.835084] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-08. [2025-09-16T17:00:26.090101] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T17:00:26.099810] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-04. [2025-09-16T17:00:26.392939] [INFO ] [REMOTE ] AAUC: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T17:00:26.395539] [INFO ] [REMOTE ] AAUC: Full fetch complete (6523 records). [2025-09-16T17:00:26.395616] [DEBUG] [VALIDATOR ] AAUC: Validating primary data. [2025-09-16T17:00:26.579090] [WARN ] [VALIDATOR ] AAUC: Validation failed - insufficient historical data. Have 69 unique days, need 150 days [2025-09-16T17:00:26.579389] [FATAL] [MAIN ] AAUC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:00:26.579561] [WARN ] [MAIN ] AAUC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:00:26.579621] [INFO ] [LOCAL ] AAUC: Starting deletion of primary data. [2025-09-16T17:00:26.579863] [DEBUG] [LOCAL ] AAUC: No primary data file found to delete. [2025-09-16T17:00:26.579947] [INFO ] [LOCAL ] AAUC: Starting deletion of derivative data. [2025-09-16T17:00:26.580077] [DEBUG] [LOCAL ] AAUC: No derivative data file found to delete. [2025-09-16T17:00:26.580178] [INFO ] [MAIN ] Update Progress: 6/2369 symbols completed. AAUC took 5.1s. Estimated remaining: 2h 34m 12s. Rate: 15.3 symbols/min [2025-09-16T17:00:26.730850] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T17:00:26.782157] [INFO ] [MAIN ] ABBV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:26.782672] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-08. [2025-09-16T17:00:26.827976] [DEBUG] [LOCAL ] ABBV: Loaded 39654 existing records from primary storage. [2025-09-16T17:00:26.828133] [INFO ] [MAIN ] ABBV: Loaded 39654 existing records. [2025-09-16T17:00:27.010457] [WARN ] [MAIN ] ABBV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:27.010768] [INFO ] [MAIN ] ABBV: Wiping existing primary data before full refresh. [2025-09-16T17:00:27.010819] [INFO ] [LOCAL ] ABBV: Starting deletion of primary data. [2025-09-16T17:00:27.011846] [DEBUG] [LOCAL ] ABBV: Deleted primary data file. [2025-09-16T17:00:27.011899] [INFO ] [MAIN ] ABBV: Starting full primary refresh. [2025-09-16T17:00:27.390391] [INFO ] [REMOTE ] ABBV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:27.390786] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-02. [2025-09-16T17:00:27.502616] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T17:00:27.513852] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-09. [2025-09-16T17:00:27.621489] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T17:00:27.629870] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-05. [2025-09-16T17:00:27.665576] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-09-16T17:00:27.672471] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-09. [2025-09-16T17:00:28.944463] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T17:00:28.954588] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-03. [2025-09-16T17:00:28.992147] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T17:00:28.996409] [INFO ] [REMOTE ] AAT: Full fetch complete (32976 records). [2025-09-16T17:00:28.996554] [DEBUG] [VALIDATOR ] AAT: Validating primary data. [2025-09-16T17:00:29.157163] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T17:00:29.239163] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-06. [2025-09-16T17:00:29.400520] [DEBUG] [VALIDATOR ] AAT: All expected trading days are present. [2025-09-16T17:00:29.400721] [DEBUG] [VALIDATOR ] AAT: Primary data validation passed. [2025-09-16T17:00:29.401156] [INFO ] [MAIN ] AAT: Fetched primary data is valid. [2025-09-16T17:00:29.401303] [DEBUG] [LOCAL ] AAT: Saving primary data. [2025-09-16T17:00:29.402803] [DEBUG] [LOCAL ] AAT: Trimmed 32976 records to 22530 (keeping 150 days of data) [2025-09-16T17:00:29.522219] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-09-16T17:00:29.530375] [INFO ] [REMOTE ] AAP: Full fetch complete (56730 records). [2025-09-16T17:00:29.531430] [DEBUG] [VALIDATOR ] AAP: Validating primary data. [2025-09-16T17:00:29.974280] [DEBUG] [VALIDATOR ] AAP: All expected trading days are present. [2025-09-16T17:00:29.974061] [DEBUG] [LOCAL ] AAT: Primary data saved (22530 records). [2025-09-16T17:00:29.974575] [INFO ] [MAIN ] AAT: Full history saved. [2025-09-16T17:00:29.974487] [DEBUG] [VALIDATOR ] AAP: Primary data validation passed. [2025-09-16T17:00:29.975224] [INFO ] [MAIN ] AAP: Fetched primary data is valid. [2025-09-16T17:00:29.975304] [DEBUG] [LOCAL ] AAP: Saving primary data. [2025-09-16T17:00:29.975589] [INFO ] [MAIN ] Update Progress: 7/2369 symbols completed. AAT took 11.4s. Estimated remaining: 2h 31m 13s. Rate: 15.6 symbols/min [2025-09-16T17:00:30.085598] [DEBUG] [LOCAL ] AAP: Trimmed 56730 records to 38811 (keeping 150 days of data) [2025-09-16T17:00:30.261596] [INFO ] [MAIN ] ABCB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:30.297830] [DEBUG] [LOCAL ] ABCB: Loaded 21569 existing records from primary storage. [2025-09-16T17:00:30.298898] [INFO ] [MAIN ] ABCB: Loaded 21569 existing records. [2025-09-16T17:00:30.562605] [WARN ] [MAIN ] ABCB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:30.572655] [INFO ] [MAIN ] ABCB: Wiping existing primary data before full refresh. [2025-09-16T17:00:30.572891] [INFO ] [LOCAL ] ABCB: Starting deletion of primary data. [2025-09-16T17:00:30.599567] [DEBUG] [LOCAL ] ABCB: Deleted primary data file. [2025-09-16T17:00:30.600201] [INFO ] [MAIN ] ABCB: Starting full primary refresh. [2025-09-16T17:00:30.668558] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T17:00:30.733038] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-07. [2025-09-16T17:00:30.936168] [DEBUG] [LOCAL ] AAP: Primary data saved (38811 records). [2025-09-16T17:00:30.939892] [INFO ] [MAIN ] AAP: Full history saved. [2025-09-16T17:00:30.950287] [INFO ] [MAIN ] Update Progress: 8/2369 symbols completed. AAP took 16.8s. Estimated remaining: 2h 17m 3s. Rate: 17.2 symbols/min [2025-09-16T17:00:31.071751] [INFO ] [REMOTE ] ABCB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:31.124344] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-02. [2025-09-16T17:00:31.215540] [INFO ] [MAIN ] ABEV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:31.250149] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-03 (2.30 seconds). [2025-09-16T17:00:31.262277] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-04. [2025-09-16T17:00:31.296875] [DEBUG] [LOCAL ] ABEV: Loaded 39288 existing records from primary storage. [2025-09-16T17:00:31.297046] [INFO ] [MAIN ] ABEV: Loaded 39288 existing records. [2025-09-16T17:00:31.479421] [WARN ] [MAIN ] ABEV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:31.479759] [INFO ] [MAIN ] ABEV: Wiping existing primary data before full refresh. [2025-09-16T17:00:31.479817] [INFO ] [LOCAL ] ABEV: Starting deletion of primary data. [2025-09-16T17:00:31.480769] [DEBUG] [LOCAL ] ABEV: Deleted primary data file. [2025-09-16T17:00:31.480827] [INFO ] [MAIN ] ABEV: Starting full primary refresh. [2025-09-16T17:00:31.852842] [INFO ] [REMOTE ] ABEV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:31.853181] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-02. [2025-09-16T17:00:32.137943] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-09-16T17:00:32.144755] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-08. [2025-09-16T17:00:32.763991] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T17:00:32.770030] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-03. [2025-09-16T17:00:33.261775] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T17:00:33.267203] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-09. [2025-09-16T17:00:33.311475] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-09-16T17:00:33.322863] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-05. [2025-09-16T17:00:33.780183] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-09-16T17:00:33.787703] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-04. [2025-09-16T17:00:33.848913] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-09-16T17:00:33.858686] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-03. [2025-09-16T17:00:34.324365] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-09-16T17:00:34.327690] [INFO ] [REMOTE ] AB: Full fetch complete (33357 records). [2025-09-16T17:00:34.327770] [DEBUG] [VALIDATOR ] AB: Validating primary data. [2025-09-16T17:00:34.728217] [DEBUG] [VALIDATOR ] AB: All expected trading days are present. [2025-09-16T17:00:34.728299] [DEBUG] [VALIDATOR ] AB: Primary data validation passed. [2025-09-16T17:00:34.728670] [INFO ] [MAIN ] AB: Fetched primary data is valid. [2025-09-16T17:00:34.728717] [DEBUG] [LOCAL ] AB: Saving primary data. [2025-09-16T17:00:34.729448] [DEBUG] [LOCAL ] AB: Trimmed 33357 records to 17872 (keeping 150 days of data) [2025-09-16T17:00:34.838027] [DEBUG] [LOCAL ] AB: Primary data saved (17872 records). [2025-09-16T17:00:34.838159] [INFO ] [MAIN ] AB: Full history saved. [2025-09-16T17:00:34.838245] [INFO ] [MAIN ] Update Progress: 9/2369 symbols completed. AB took 12.6s. Estimated remaining: 2h 18m 46s. Rate: 17.0 symbols/min [2025-09-16T17:00:35.016883] [INFO ] [MAIN ] ABG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:35.030486] [DEBUG] [LOCAL ] ABG: Loaded 12853 existing records from primary storage. [2025-09-16T17:00:35.030596] [INFO ] [MAIN ] ABG: Loaded 12853 existing records. [2025-09-16T17:00:35.198341] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-09-16T17:00:35.240421] [WARN ] [MAIN ] ABG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:35.240312] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-05. [2025-09-16T17:00:35.241430] [INFO ] [MAIN ] ABG: Wiping existing primary data before full refresh. [2025-09-16T17:00:35.241488] [INFO ] [LOCAL ] ABG: Starting deletion of primary data. [2025-09-16T17:00:35.241881] [DEBUG] [LOCAL ] ABG: Deleted primary data file. [2025-09-16T17:00:35.241932] [INFO ] [MAIN ] ABG: Starting full primary refresh. [2025-09-16T17:00:35.652100] [INFO ] [REMOTE ] ABG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:35.652563] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-02. [2025-09-16T17:00:35.687323] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-05 (2.36 seconds). [2025-09-16T17:00:35.725751] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-06. [2025-09-16T17:00:35.728145] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-09-16T17:00:35.739233] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-04. [2025-09-16T17:00:36.544846] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T17:00:36.551070] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-06. [2025-09-16T17:00:36.819268] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-09-16T17:00:36.822974] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-03. [2025-09-16T17:00:37.183785] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-09-16T17:00:37.194900] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-05. [2025-09-16T17:00:37.645484] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-09-16T17:00:37.651580] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-07. [2025-09-16T17:00:37.926261] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T17:00:37.932067] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-04. [2025-09-16T17:00:37.974002] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-06 (2.25 seconds). [2025-09-16T17:00:37.985924] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-07. [2025-09-16T17:00:38.628546] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-09-16T17:00:38.640033] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-06. [2025-09-16T17:00:38.956965] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T17:00:38.964101] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-08. [2025-09-16T17:00:39.046063] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-09-16T17:00:39.052194] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-05. [2025-09-16T17:00:39.739411] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-09-16T17:00:39.742981] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-06. [2025-09-16T17:00:39.822859] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T17:00:39.860722] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-09. [2025-09-16T17:00:39.863498] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T17:00:39.875632] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-08. [2025-09-16T17:00:40.337337] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T17:00:40.348836] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-07. [2025-09-16T17:00:40.385714] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-09-16T17:00:40.391543] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-07. [2025-09-16T17:00:40.947150] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T17:00:40.951101] [INFO ] [REMOTE ] ABCB: Full fetch complete (33234 records). [2025-09-16T17:00:40.951202] [DEBUG] [VALIDATOR ] ABCB: Validating primary data. [2025-09-16T17:00:41.374282] [DEBUG] [VALIDATOR ] ABCB: All expected trading days are present. [2025-09-16T17:00:41.374369] [DEBUG] [VALIDATOR ] ABCB: Primary data validation passed. [2025-09-16T17:00:41.374818] [INFO ] [MAIN ] ABCB: Fetched primary data is valid. [2025-09-16T17:00:41.374904] [DEBUG] [LOCAL ] ABCB: Saving primary data. [2025-09-16T17:00:41.376024] [DEBUG] [LOCAL ] ABCB: Trimmed 33234 records to 21813 (keeping 150 days of data) [2025-09-16T17:00:41.499223] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-09-16T17:00:41.506165] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-08. [2025-09-16T17:00:41.536693] [DEBUG] [LOCAL ] ABCB: Primary data saved (21813 records). [2025-09-16T17:00:41.536819] [INFO ] [MAIN ] ABCB: Full history saved. [2025-09-16T17:00:41.536908] [INFO ] [MAIN ] Update Progress: 10/2369 symbols completed. ABCB took 11.6s. Estimated remaining: 2h 31m 10s. Rate: 15.6 symbols/min [2025-09-16T17:00:41.711931] [INFO ] [MAIN ] ABM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:41.735674] [DEBUG] [LOCAL ] ABM: Loaded 23119 existing records from primary storage. [2025-09-16T17:00:41.735832] [INFO ] [MAIN ] ABM: Loaded 23119 existing records. [2025-09-16T17:00:41.921936] [WARN ] [MAIN ] ABM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:41.922234] [INFO ] [MAIN ] ABM: Wiping existing primary data before full refresh. [2025-09-16T17:00:41.922372] [INFO ] [LOCAL ] ABM: Starting deletion of primary data. [2025-09-16T17:00:41.923010] [DEBUG] [LOCAL ] ABM: Deleted primary data file. [2025-09-16T17:00:41.923071] [INFO ] [MAIN ] ABM: Starting full primary refresh. [2025-09-16T17:00:42.072515] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-08 (2.20 seconds). [2025-09-16T17:00:42.151274] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-09. [2025-09-16T17:00:42.332151] [INFO ] [REMOTE ] ABM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:42.332570] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-02. [2025-09-16T17:00:42.368289] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-09-16T17:00:42.380057] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-08. [2025-09-16T17:00:42.464429] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-09-16T17:00:42.469006] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-09. [2025-09-16T17:00:43.228885] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-09-16T17:00:43.233912] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-03. [2025-09-16T17:00:43.262630] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T17:00:43.265701] [INFO ] [REMOTE ] ABG: Full fetch complete (20403 records). [2025-09-16T17:00:43.265838] [DEBUG] [VALIDATOR ] ABG: Validating primary data. [2025-09-16T17:00:43.660578] [DEBUG] [VALIDATOR ] ABG: All expected trading days are present. [2025-09-16T17:00:43.660687] [DEBUG] [VALIDATOR ] ABG: Primary data validation passed. [2025-09-16T17:00:43.660986] [INFO ] [MAIN ] ABG: Fetched primary data is valid. [2025-09-16T17:00:43.661031] [DEBUG] [LOCAL ] ABG: Saving primary data. [2025-09-16T17:00:43.661777] [DEBUG] [LOCAL ] ABG: Trimmed 20403 records to 12960 (keeping 150 days of data) [2025-09-16T17:00:43.742744] [DEBUG] [LOCAL ] ABG: Primary data saved (12960 records). [2025-09-16T17:00:43.742916] [INFO ] [MAIN ] ABG: Full history saved. [2025-09-16T17:00:43.743027] [INFO ] [MAIN ] Update Progress: 11/2369 symbols completed. ABG took 8.9s. Estimated remaining: 2h 25m 15s. Rate: 16.2 symbols/min [2025-09-16T17:00:43.920755] [INFO ] [MAIN ] ABR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:43.964823] [DEBUG] [LOCAL ] ABR: Loaded 38398 existing records from primary storage. [2025-09-16T17:00:43.964976] [INFO ] [MAIN ] ABR: Loaded 38398 existing records. [2025-09-16T17:00:44.153571] [WARN ] [MAIN ] ABR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:44.154125] [INFO ] [MAIN ] ABR: Wiping existing primary data before full refresh. [2025-09-16T17:00:44.154272] [INFO ] [LOCAL ] ABR: Starting deletion of primary data. [2025-09-16T17:00:44.155278] [DEBUG] [LOCAL ] ABR: Deleted primary data file. [2025-09-16T17:00:44.155356] [INFO ] [MAIN ] ABR: Starting full primary refresh. [2025-09-16T17:00:44.355312] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-09-16T17:00:44.433376] [INFO ] [REMOTE ] ABBV: Full fetch complete (58363 records). [2025-09-16T17:00:44.463104] [DEBUG] [VALIDATOR ] ABBV: Validating primary data. [2025-09-16T17:00:44.689830] [INFO ] [REMOTE ] ABR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:44.706022] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-02. [2025-09-16T17:00:44.911991] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-09-16T17:00:44.999281] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-04. [2025-09-16T17:00:45.041483] [DEBUG] [VALIDATOR ] ABBV: All expected trading days are present. [2025-09-16T17:00:45.045600] [DEBUG] [VALIDATOR ] ABBV: Primary data validation passed. [2025-09-16T17:00:45.046315] [INFO ] [MAIN ] ABBV: Fetched primary data is valid. [2025-09-16T17:00:45.046708] [DEBUG] [LOCAL ] ABBV: Saving primary data. [2025-09-16T17:00:45.049194] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-08 (2.67 seconds). [2025-09-16T17:00:45.061604] [DEBUG] [LOCAL ] ABBV: Trimmed 58363 records to 40044 (keeping 150 days of data) [2025-09-16T17:00:45.061489] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-09. [2025-09-16T17:00:45.313309] [DEBUG] [LOCAL ] ABBV: Primary data saved (40044 records). [2025-09-16T17:00:45.313444] [INFO ] [MAIN ] ABBV: Full history saved. [2025-09-16T17:00:45.315136] [INFO ] [MAIN ] Update Progress: 12/2369 symbols completed. ABBV took 18.7s. Estimated remaining: 2h 18m 14s. Rate: 17.1 symbols/min [2025-09-16T17:00:45.492109] [INFO ] [MAIN ] ABT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:45.536730] [DEBUG] [LOCAL ] ABT: Loaded 39593 existing records from primary storage. [2025-09-16T17:00:45.537111] [INFO ] [MAIN ] ABT: Loaded 39593 existing records. [2025-09-16T17:00:45.719178] [WARN ] [MAIN ] ABT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:45.719485] [INFO ] [MAIN ] ABT: Wiping existing primary data before full refresh. [2025-09-16T17:00:45.719534] [INFO ] [LOCAL ] ABT: Starting deletion of primary data. [2025-09-16T17:00:45.720552] [DEBUG] [LOCAL ] ABT: Deleted primary data file. [2025-09-16T17:00:45.720607] [INFO ] [MAIN ] ABT: Starting full primary refresh. [2025-09-16T17:00:46.076658] [INFO ] [REMOTE ] ABT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:46.077005] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-02. [2025-09-16T17:00:46.435311] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T17:00:46.445222] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-03. [2025-09-16T17:00:46.621913] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T17:00:46.630923] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-05. [2025-09-16T17:00:47.040032] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-09-16T17:00:47.046670] [INFO ] [REMOTE ] ABEV: Full fetch complete (57904 records). [2025-09-16T17:00:47.046813] [DEBUG] [VALIDATOR ] ABEV: Validating primary data. [2025-09-16T17:00:47.462086] [DEBUG] [VALIDATOR ] ABEV: All expected trading days are present. [2025-09-16T17:00:47.462299] [DEBUG] [VALIDATOR ] ABEV: Primary data validation passed. [2025-09-16T17:00:47.462992] [INFO ] [MAIN ] ABEV: Fetched primary data is valid. [2025-09-16T17:00:47.463085] [DEBUG] [LOCAL ] ABEV: Saving primary data. [2025-09-16T17:00:47.464776] [DEBUG] [LOCAL ] ABEV: Trimmed 57904 records to 39678 (keeping 150 days of data) [2025-09-16T17:00:47.592630] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T17:00:47.612318] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-03. [2025-09-16T17:00:47.748230] [DEBUG] [LOCAL ] ABEV: Primary data saved (39678 records). [2025-09-16T17:00:47.748455] [INFO ] [MAIN ] ABEV: Full history saved. [2025-09-16T17:00:47.748588] [INFO ] [MAIN ] Update Progress: 13/2369 symbols completed. ABEV took 16.8s. Estimated remaining: 2h 14m 54s. Rate: 17.5 symbols/min [2025-09-16T17:00:47.921848] [INFO ] [MAIN ] ACA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:47.940768] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T17:00:47.946835] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-06. [2025-09-16T17:00:47.962870] [DEBUG] [LOCAL ] ACA: Loaded 16627 existing records from primary storage. [2025-09-16T17:00:47.963068] [INFO ] [MAIN ] ACA: Loaded 16627 existing records. [2025-09-16T17:00:48.164108] [WARN ] [MAIN ] ACA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:48.189259] [INFO ] [MAIN ] ACA: Wiping existing primary data before full refresh. [2025-09-16T17:00:48.189368] [INFO ] [LOCAL ] ACA: Starting deletion of primary data. [2025-09-16T17:00:48.190121] [DEBUG] [LOCAL ] ACA: Deleted primary data file. [2025-09-16T17:00:48.190185] [INFO ] [MAIN ] ACA: Starting full primary refresh. [2025-09-16T17:00:48.191891] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T17:00:48.279366] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-04. [2025-09-16T17:00:48.563613] [INFO ] [REMOTE ] ACA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:48.563986] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-02. [2025-09-16T17:00:49.275777] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-09-16T17:00:49.289361] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-04. [2025-09-16T17:00:49.334310] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T17:00:49.341500] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-07. [2025-09-16T17:00:49.783617] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T17:00:49.789076] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-03. [2025-09-16T17:00:50.072436] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T17:00:50.083708] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-05. [2025-09-16T17:00:50.722820] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-09-16T17:00:50.741596] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-09-16T17:00:50.748511] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-08. [2025-09-16T17:00:50.750012] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-04. [2025-09-16T17:00:50.873239] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T17:00:50.884738] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-05. [2025-09-16T17:00:51.693234] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T17:00:51.705329] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-06. [2025-09-16T17:00:51.914626] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-09-16T17:00:51.920762] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-05. [2025-09-16T17:00:52.075432] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-09-16T17:00:52.082512] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-09. [2025-09-16T17:00:52.728030] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T17:00:52.739921] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-06. [2025-09-16T17:00:52.766029] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-09-16T17:00:52.772309] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-06. [2025-09-16T17:00:53.431804] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T17:00:53.436611] [INFO ] [REMOTE ] ABM: Full fetch complete (34314 records). [2025-09-16T17:00:53.436723] [DEBUG] [VALIDATOR ] ABM: Validating primary data. [2025-09-16T17:00:53.825115] [DEBUG] [VALIDATOR ] ABM: All expected trading days are present. [2025-09-16T17:00:53.825450] [DEBUG] [VALIDATOR ] ABM: Primary data validation passed. [2025-09-16T17:00:53.826330] [INFO ] [MAIN ] ABM: Fetched primary data is valid. [2025-09-16T17:00:53.826593] [DEBUG] [LOCAL ] ABM: Saving primary data. [2025-09-16T17:00:53.839922] [DEBUG] [LOCAL ] ABM: Trimmed 34314 records to 23397 (keeping 150 days of data) [2025-09-16T17:00:53.843530] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T17:00:53.850166] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-07. [2025-09-16T17:00:53.985326] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-06 (2.28 seconds). [2025-09-16T17:00:54.016104] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-07. [2025-09-16T17:00:54.041406] [DEBUG] [LOCAL ] ABM: Primary data saved (23397 records). [2025-09-16T17:00:54.041573] [INFO ] [MAIN ] ABM: Full history saved. [2025-09-16T17:00:54.041705] [INFO ] [MAIN ] Update Progress: 14/2369 symbols completed. ABM took 12.5s. Estimated remaining: 2h 22m 51s. Rate: 16.5 symbols/min [2025-09-16T17:00:54.215556] [INFO ] [MAIN ] ACCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:54.248681] [DEBUG] [LOCAL ] ACCO: Loaded 27389 existing records from primary storage. [2025-09-16T17:00:54.248840] [INFO ] [MAIN ] ACCO: Loaded 27389 existing records. [2025-09-16T17:00:54.434457] [WARN ] [MAIN ] ACCO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:00:54.434813] [INFO ] [MAIN ] ACCO: Wiping existing primary data before full refresh. [2025-09-16T17:00:54.434869] [INFO ] [LOCAL ] ACCO: Starting deletion of primary data. [2025-09-16T17:00:54.435714] [DEBUG] [LOCAL ] ACCO: Deleted primary data file. [2025-09-16T17:00:54.435781] [INFO ] [MAIN ] ACCO: Starting full primary refresh. [2025-09-16T17:00:54.813341] [INFO ] [REMOTE ] ACCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:54.813998] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-02. [2025-09-16T17:00:54.857335] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-09-16T17:00:54.867413] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-09-16T17:00:54.878232] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-08. [2025-09-16T17:00:54.880160] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-07. [2025-09-16T17:00:55.631368] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T17:00:55.637101] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-09. [2025-09-16T17:00:56.027739] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-09-16T17:00:56.039503] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-08. [2025-09-16T17:00:56.539457] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T17:00:56.547416] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-03. [2025-09-16T17:00:56.584248] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T17:00:56.597250] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-08. [2025-09-16T17:00:56.605000] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T17:00:56.608054] [INFO ] [REMOTE ] ACA: Full fetch complete (26463 records). [2025-09-16T17:00:56.608232] [DEBUG] [VALIDATOR ] ACA: Validating primary data. [2025-09-16T17:00:56.991436] [DEBUG] [VALIDATOR ] ACA: All expected trading days are present. [2025-09-16T17:00:56.991528] [DEBUG] [VALIDATOR ] ACA: Primary data validation passed. [2025-09-16T17:00:56.991871] [INFO ] [MAIN ] ACA: Fetched primary data is valid. [2025-09-16T17:00:56.991939] [DEBUG] [LOCAL ] ACA: Saving primary data. [2025-09-16T17:00:56.992982] [DEBUG] [LOCAL ] ACA: Trimmed 26463 records to 16738 (keeping 150 days of data) [2025-09-16T17:00:57.096067] [DEBUG] [LOCAL ] ACA: Primary data saved (16738 records). [2025-09-16T17:00:57.096230] [INFO ] [MAIN ] ACA: Full history saved. [2025-09-16T17:00:57.096336] [INFO ] [MAIN ] Update Progress: 15/2369 symbols completed. ACA took 9.3s. Estimated remaining: 2h 21m 16s. Rate: 16.7 symbols/min [2025-09-16T17:00:57.268777] [INFO ] [MAIN ] ACEL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:00:57.268949] [DEBUG] [LOCAL ] ACEL: No existing data file found in primary storage. [2025-09-16T17:00:57.268988] [WARN ] [MAIN ] ACEL: No existing data found, performing full refresh instead. [2025-09-16T17:00:57.269109] [INFO ] [MAIN ] ACEL: Wiping existing primary data before full refresh. [2025-09-16T17:00:57.269149] [INFO ] [LOCAL ] ACEL: Starting deletion of primary data. [2025-09-16T17:00:57.269190] [DEBUG] [LOCAL ] ACEL: No primary data file found to delete. [2025-09-16T17:00:57.269224] [INFO ] [MAIN ] ACEL: Starting full primary refresh. [2025-09-16T17:00:57.659326] [INFO ] [REMOTE ] ACEL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:00:57.659823] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-02. [2025-09-16T17:00:57.723274] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T17:00:57.735213] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-09. [2025-09-16T17:00:58.277466] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T17:00:58.289006] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-09. [2025-09-16T17:00:58.523978] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-09-16T17:00:58.533848] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-04. [2025-09-16T17:00:59.248790] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T17:00:59.254836] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-03. [2025-09-16T17:00:59.579208] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-09-16T17:00:59.585522] [INFO ] [REMOTE ] ABR: Full fetch complete (56739 records). [2025-09-16T17:00:59.585741] [DEBUG] [VALIDATOR ] ABR: Validating primary data. [2025-09-16T17:01:00.031766] [DEBUG] [VALIDATOR ] ABR: All expected trading days are present. [2025-09-16T17:01:00.054334] [DEBUG] [VALIDATOR ] ABR: Primary data validation passed. [2025-09-16T17:01:00.055452] [INFO ] [MAIN ] ABR: Fetched primary data is valid. [2025-09-16T17:01:00.055703] [DEBUG] [LOCAL ] ABR: Saving primary data. [2025-09-16T17:01:00.057569] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T17:01:00.067276] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-05. [2025-09-16T17:01:00.067792] [DEBUG] [LOCAL ] ABR: Trimmed 56739 records to 38749 (keeping 150 days of data) [2025-09-16T17:01:00.206092] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-09-16T17:01:00.223168] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-04. [2025-09-16T17:01:00.341431] [DEBUG] [LOCAL ] ABR: Primary data saved (38749 records). [2025-09-16T17:01:00.358256] [INFO ] [MAIN ] ABR: Full history saved. [2025-09-16T17:01:00.362388] [INFO ] [MAIN ] Update Progress: 16/2369 symbols completed. ABR took 16.6s. Estimated remaining: 2h 20m 23s. Rate: 16.8 symbols/min [2025-09-16T17:01:00.414131] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-09 (2.13 seconds). [2025-09-16T17:01:00.480261] [INFO ] [REMOTE ] ABT: Full fetch complete (58290 records). [2025-09-16T17:01:00.491079] [DEBUG] [VALIDATOR ] ABT: Validating primary data. [2025-09-16T17:01:00.603222] [INFO ] [MAIN ] ACHR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:00.974852] [DEBUG] [LOCAL ] ACHR: Loaded 39778 existing records from primary storage. [2025-09-16T17:01:00.993497] [INFO ] [MAIN ] ACHR: Loaded 39778 existing records. [2025-09-16T17:01:01.045434] [DEBUG] [VALIDATOR ] ABT: All expected trading days are present. [2025-09-16T17:01:01.088196] [DEBUG] [VALIDATOR ] ABT: Primary data validation passed. [2025-09-16T17:01:01.100697] [INFO ] [MAIN ] ABT: Fetched primary data is valid. [2025-09-16T17:01:01.131190] [DEBUG] [LOCAL ] ABT: Saving primary data. [2025-09-16T17:01:01.193111] [DEBUG] [LOCAL ] ABT: Trimmed 58290 records to 39983 (keeping 150 days of data) [2025-09-16T17:01:01.216607] [WARN ] [MAIN ] ACHR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:01:01.217284] [INFO ] [MAIN ] ACHR: Wiping existing primary data before full refresh. [2025-09-16T17:01:01.217339] [INFO ] [LOCAL ] ACHR: Starting deletion of primary data. [2025-09-16T17:01:01.218213] [DEBUG] [LOCAL ] ACHR: Deleted primary data file. [2025-09-16T17:01:01.218272] [INFO ] [MAIN ] ACHR: Starting full primary refresh. [2025-09-16T17:01:01.703271] [INFO ] [REMOTE ] ACHR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:01.703924] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-02. [2025-09-16T17:01:01.860169] [DEBUG] [LOCAL ] ABT: Primary data saved (39983 records). [2025-09-16T17:01:01.860355] [INFO ] [MAIN ] ABT: Full history saved. [2025-09-16T17:01:01.861744] [INFO ] [MAIN ] Update Progress: 17/2369 symbols completed. ABT took 16.5s. Estimated remaining: 2h 15m 32s. Rate: 17.4 symbols/min [2025-09-16T17:01:02.037295] [INFO ] [MAIN ] ACI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:02.037489] [DEBUG] [LOCAL ] ACI: No existing data file found in primary storage. [2025-09-16T17:01:02.037528] [WARN ] [MAIN ] ACI: No existing data found, performing full refresh instead. [2025-09-16T17:01:02.037711] [INFO ] [MAIN ] ACI: Wiping existing primary data before full refresh. [2025-09-16T17:01:02.037764] [INFO ] [LOCAL ] ACI: Starting deletion of primary data. [2025-09-16T17:01:02.037813] [DEBUG] [LOCAL ] ACI: No primary data file found to delete. [2025-09-16T17:01:02.037847] [INFO ] [MAIN ] ACI: Starting full primary refresh. [2025-09-16T17:01:02.414595] [INFO ] [REMOTE ] ACI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:02.415278] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-02. [2025-09-16T17:01:02.448404] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-05 (2.38 seconds). [2025-09-16T17:01:02.460560] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-06. [2025-09-16T17:01:02.825931] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-04 (2.60 seconds). [2025-09-16T17:01:02.833747] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-05. [2025-09-16T17:01:03.850893] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T17:01:03.859687] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-07. [2025-09-16T17:01:03.927345] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-09-16T17:01:04.125992] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-09-16T17:01:04.374985] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-03. [2025-09-16T17:01:04.921092] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-02 (3.22 seconds). [2025-09-16T17:01:05.136980] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-09-16T17:01:05.174935] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-03. [2025-09-16T17:01:05.974941] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-08. [2025-09-16T17:01:06.199699] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-09-16T17:01:06.774911] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-04. [2025-09-16T17:01:07.343126] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-09-16T17:01:07.416015] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-03 (2.24 seconds). [2025-09-16T17:01:07.574923] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-09. [2025-09-16T17:01:08.207935] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:01:08.374884] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-05. [2025-09-16T17:01:09.063687] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T17:01:09.069153] [INFO ] [REMOTE ] ACCO: Full fetch complete (41481 records). [2025-09-16T17:01:09.069413] [DEBUG] [VALIDATOR ] ACCO: Validating primary data. [2025-09-16T17:01:09.180599] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-06. [2025-09-16T17:01:09.478221] [DEBUG] [VALIDATOR ] ACCO: All expected trading days are present. [2025-09-16T17:01:09.478338] [DEBUG] [VALIDATOR ] ACCO: Primary data validation passed. [2025-09-16T17:01:09.478974] [INFO ] [MAIN ] ACCO: Fetched primary data is valid. [2025-09-16T17:01:09.479036] [DEBUG] [LOCAL ] ACCO: Saving primary data. [2025-09-16T17:01:09.480264] [DEBUG] [LOCAL ] ACCO: Trimmed 41481 records to 27598 (keeping 150 days of data) [2025-09-16T17:01:09.651009] [DEBUG] [LOCAL ] ACCO: Primary data saved (27598 records). [2025-09-16T17:01:09.651250] [INFO ] [MAIN ] ACCO: Full history saved. [2025-09-16T17:01:09.651379] [INFO ] [MAIN ] Update Progress: 18/2369 symbols completed. ACCO took 15.6s. Estimated remaining: 2h 24m 54s. Rate: 16.2 symbols/min [2025-09-16T17:01:09.841969] [INFO ] [MAIN ] ACM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:09.890527] [DEBUG] [LOCAL ] ACM: Loaded 31167 existing records from primary storage. [2025-09-16T17:01:09.908682] [INFO ] [MAIN ] ACM: Loaded 31167 existing records. [2025-09-16T17:01:09.983372] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T17:01:09.980726] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-04. [2025-09-16T17:01:10.119871] [WARN ] [MAIN ] ACM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:01:10.120224] [INFO ] [MAIN ] ACM: Wiping existing primary data before full refresh. [2025-09-16T17:01:10.120278] [INFO ] [LOCAL ] ACM: Starting deletion of primary data. [2025-09-16T17:01:10.121289] [DEBUG] [LOCAL ] ACM: Deleted primary data file. [2025-09-16T17:01:10.121356] [INFO ] [MAIN ] ACM: Starting full primary refresh. [2025-09-16T17:01:10.499705] [INFO ] [REMOTE ] ACM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:10.515119] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T17:01:10.774919] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-07. [2025-09-16T17:01:11.540332] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T17:01:11.574852] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-08. [2025-09-16T17:01:12.287824] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-04 (2.31 seconds). [2025-09-16T17:01:12.374855] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-05. [2025-09-16T17:01:12.767181] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-09-16T17:01:13.174921] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-06. [2025-09-16T17:01:13.974972] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-02. [2025-09-16T17:01:14.774886] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-09. [2025-09-16T17:01:15.046471] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-05 (2.67 seconds). [2025-09-16T17:01:15.151456] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-09-16T17:01:15.308757] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-09-16T17:01:15.574905] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-03. [2025-09-16T17:01:15.775481] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T17:01:15.778796] [INFO ] [REMOTE ] ACEL: Full fetch complete (30128 records). [2025-09-16T17:01:15.778894] [DEBUG] [VALIDATOR ] ACEL: Validating primary data. [2025-09-16T17:01:15.779531] [FATAL] [VALIDATOR ] ACEL: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:01:15.779616] [FATAL] [MAIN ] ACEL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:01:15.779675] [WARN ] [MAIN ] ACEL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:01:15.779716] [INFO ] [LOCAL ] ACEL: Starting deletion of primary data. [2025-09-16T17:01:15.779775] [DEBUG] [LOCAL ] ACEL: No primary data file found to delete. [2025-09-16T17:01:15.779818] [INFO ] [LOCAL ] ACEL: Starting deletion of derivative data. [2025-09-16T17:01:15.779867] [DEBUG] [LOCAL ] ACEL: No derivative data file found to delete. [2025-09-16T17:01:15.779929] [INFO ] [MAIN ] Update Progress: 19/2369 symbols completed. ACEL took 18.7s. Estimated remaining: 2h 29m 51s. Rate: 15.7 symbols/min [2025-09-16T17:01:15.954782] [INFO ] [MAIN ] ACN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:15.955002] [DEBUG] [LOCAL ] ACN: No existing data file found in primary storage. [2025-09-16T17:01:15.955043] [WARN ] [MAIN ] ACN: No existing data found, performing full refresh instead. [2025-09-16T17:01:15.955174] [INFO ] [MAIN ] ACN: Wiping existing primary data before full refresh. [2025-09-16T17:01:15.955221] [INFO ] [LOCAL ] ACN: Starting deletion of primary data. [2025-09-16T17:01:15.955266] [DEBUG] [LOCAL ] ACN: No primary data file found to delete. [2025-09-16T17:01:15.955301] [INFO ] [MAIN ] ACN: Starting full primary refresh. [2025-09-16T17:01:16.316720] [INFO ] [REMOTE ] ACN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:16.374894] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-02. [2025-09-16T17:01:16.938783] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T17:01:17.174931] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-07. [2025-09-16T17:01:17.974930] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-04. [2025-09-16T17:01:18.203214] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T17:01:18.774899] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-06. [2025-09-16T17:01:18.805824] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-09-16T17:01:19.574892] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-03. [2025-09-16T17:01:19.625220] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T17:01:20.374907] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-08. [2025-09-16T17:01:21.127544] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-06 (2.35 seconds). [2025-09-16T17:01:21.174867] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-07. [2025-09-16T17:01:21.472878] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-09-16T17:01:21.684512] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T17:01:21.974974] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-05. [2025-09-16T17:01:22.774937] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-09. [2025-09-16T17:01:23.541142] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T17:01:23.574972] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-04. [2025-09-16T17:01:23.711896] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-07 (2.54 seconds). [2025-09-16T17:01:24.374929] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-08. [2025-09-16T17:01:24.638005] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-09-16T17:01:24.645806] [INFO ] [REMOTE ] ACI: Full fetch complete (57874 records). [2025-09-16T17:01:24.646000] [DEBUG] [VALIDATOR ] ACI: Validating primary data. [2025-09-16T17:01:25.042948] [DEBUG] [VALIDATOR ] ACI: All expected trading days are present. [2025-09-16T17:01:25.043031] [DEBUG] [VALIDATOR ] ACI: Primary data validation passed. [2025-09-16T17:01:25.043679] [INFO ] [MAIN ] ACI: Fetched primary data is valid. [2025-09-16T17:01:25.043747] [DEBUG] [LOCAL ] ACI: Saving primary data. [2025-09-16T17:01:25.046085] [DEBUG] [LOCAL ] ACI: Trimmed 57874 records to 39673 (keeping 150 days of data) [2025-09-16T17:01:25.175257] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-06. [2025-09-16T17:01:25.303271] [DEBUG] [LOCAL ] ACI: Primary data saved (39673 records). [2025-09-16T17:01:25.303421] [INFO ] [MAIN ] ACI: Full history saved. [2025-09-16T17:01:25.303501] [INFO ] [MAIN ] Update Progress: 20/2369 symbols completed. ACI took 23.4s. Estimated remaining: 2h 40m 56s. Rate: 14.6 symbols/min [2025-09-16T17:01:25.520696] [INFO ] [MAIN ] ACP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:25.523424] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-09-16T17:01:25.555824] [DEBUG] [LOCAL ] ACP: Loaded 22894 existing records from primary storage. [2025-09-16T17:01:25.555949] [INFO ] [MAIN ] ACP: Loaded 22894 existing records. [2025-09-16T17:01:25.734665] [WARN ] [MAIN ] ACP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:01:25.734964] [INFO ] [MAIN ] ACP: Wiping existing primary data before full refresh. [2025-09-16T17:01:25.735045] [INFO ] [LOCAL ] ACP: Starting deletion of primary data. [2025-09-16T17:01:25.735582] [DEBUG] [LOCAL ] ACP: Deleted primary data file. [2025-09-16T17:01:25.735636] [INFO ] [MAIN ] ACP: Starting full primary refresh. [2025-09-16T17:01:25.979978] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-05. [2025-09-16T17:01:26.102025] [INFO ] [REMOTE ] ACP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:26.536102] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-09-16T17:01:26.774922] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-02. [2025-09-16T17:01:26.829534] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-08 (2.45 seconds). [2025-09-16T17:01:27.574902] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-09. [2025-09-16T17:01:27.846573] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-09-16T17:01:28.061164] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T17:01:28.374917] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-07. [2025-09-16T17:01:29.174921] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-06. [2025-09-16T17:01:29.974914] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-03. [2025-09-16T17:01:30.074354] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T17:01:30.100044] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-09 (2.53 seconds). [2025-09-16T17:01:30.106743] [INFO ] [REMOTE ] ACHR: Full fetch complete (58498 records). [2025-09-16T17:01:30.106857] [DEBUG] [VALIDATOR ] ACHR: Validating primary data. [2025-09-16T17:01:30.492970] [DEBUG] [VALIDATOR ] ACHR: All expected trading days are present. [2025-09-16T17:01:30.493065] [DEBUG] [VALIDATOR ] ACHR: Primary data validation passed. [2025-09-16T17:01:30.493685] [INFO ] [MAIN ] ACHR: Fetched primary data is valid. [2025-09-16T17:01:30.493746] [DEBUG] [LOCAL ] ACHR: Saving primary data. [2025-09-16T17:01:30.495244] [DEBUG] [LOCAL ] ACHR: Trimmed 58498 records to 40168 (keeping 150 days of data) [2025-09-16T17:01:30.742170] [DEBUG] [LOCAL ] ACHR: Primary data saved (40168 records). [2025-09-16T17:01:30.742312] [INFO ] [MAIN ] ACHR: Full history saved. [2025-09-16T17:01:30.743321] [INFO ] [MAIN ] Update Progress: 21/2369 symbols completed. ACHR took 30.4s. Estimated remaining: 2h 43m 21s. Rate: 14.4 symbols/min [2025-09-16T17:01:30.777038] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-08. [2025-09-16T17:01:30.924948] [INFO ] [MAIN ] ACR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:30.925168] [DEBUG] [LOCAL ] ACR: No existing data file found in primary storage. [2025-09-16T17:01:30.925241] [WARN ] [MAIN ] ACR: No existing data found, performing full refresh instead. [2025-09-16T17:01:30.925460] [INFO ] [MAIN ] ACR: Wiping existing primary data before full refresh. [2025-09-16T17:01:30.925533] [INFO ] [LOCAL ] ACR: Starting deletion of primary data. [2025-09-16T17:01:30.925694] [DEBUG] [LOCAL ] ACR: No primary data file found to delete. [2025-09-16T17:01:30.925737] [INFO ] [MAIN ] ACR: Starting full primary refresh. [2025-09-16T17:01:31.081451] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-09-16T17:01:31.340102] [INFO ] [REMOTE ] ACR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:31.359545] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-09-16T17:01:31.574907] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-07. [2025-09-16T17:01:32.374893] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-04. [2025-09-16T17:01:32.419817] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T17:01:33.174904] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-09. [2025-09-16T17:01:33.388533] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T17:01:33.810676] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-09-16T17:01:33.974947] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-05. [2025-09-16T17:01:34.546868] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T17:01:34.553207] [INFO ] [REMOTE ] ACM: Full fetch complete (47212 records). [2025-09-16T17:01:34.553390] [DEBUG] [VALIDATOR ] ACM: Validating primary data. [2025-09-16T17:01:34.780101] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-08. [2025-09-16T17:01:34.959950] [DEBUG] [VALIDATOR ] ACM: All expected trading days are present. [2025-09-16T17:01:34.960088] [DEBUG] [VALIDATOR ] ACM: Primary data validation passed. [2025-09-16T17:01:34.960608] [INFO ] [MAIN ] ACM: Fetched primary data is valid. [2025-09-16T17:01:34.960679] [DEBUG] [LOCAL ] ACM: Saving primary data. [2025-09-16T17:01:34.961917] [DEBUG] [LOCAL ] ACM: Trimmed 47212 records to 31522 (keeping 150 days of data) [2025-09-16T17:01:35.112263] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T17:01:35.205634] [DEBUG] [LOCAL ] ACM: Primary data saved (31522 records). [2025-09-16T17:01:35.205862] [INFO ] [MAIN ] ACM: Full history saved. [2025-09-16T17:01:35.207315] [INFO ] [MAIN ] Update Progress: 22/2369 symbols completed. ACM took 25.6s. Estimated remaining: 2h 43m 48s. Rate: 14.3 symbols/min [2025-09-16T17:01:35.383181] [INFO ] [MAIN ] ACRE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:35.383359] [DEBUG] [LOCAL ] ACRE: No existing data file found in primary storage. [2025-09-16T17:01:35.383408] [WARN ] [MAIN ] ACRE: No existing data found, performing full refresh instead. [2025-09-16T17:01:35.383548] [INFO ] [MAIN ] ACRE: Wiping existing primary data before full refresh. [2025-09-16T17:01:35.383593] [INFO ] [LOCAL ] ACRE: Starting deletion of primary data. [2025-09-16T17:01:35.383680] [DEBUG] [LOCAL ] ACRE: No primary data file found to delete. [2025-09-16T17:01:35.383733] [INFO ] [MAIN ] ACRE: Starting full primary refresh. [2025-09-16T17:01:35.580111] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-06. [2025-09-16T17:01:35.762537] [INFO ] [REMOTE ] ACRE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:36.374951] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-02. [2025-09-16T17:01:36.586715] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-09-16T17:01:36.944302] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-09-16T17:01:37.155292] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-09-16T17:01:37.174913] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-07. [2025-09-16T17:01:37.975014] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-03. [2025-09-16T17:01:38.500032] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-09-16T17:01:38.774935] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-08. [2025-09-16T17:01:38.858452] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-09-16T17:01:39.575177] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-04. [2025-09-16T17:01:39.995837] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-09-16T17:01:40.264730] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-09-16T17:01:40.374934] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-09. [2025-09-16T17:01:41.174916] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-09. [2025-09-16T17:01:41.397774] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T17:01:41.401696] [INFO ] [REMOTE ] ACP: Full fetch complete (34242 records). [2025-09-16T17:01:41.401883] [DEBUG] [VALIDATOR ] ACP: Validating primary data. [2025-09-16T17:01:41.402610] [FATAL] [VALIDATOR ] ACP: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:01:41.402719] [FATAL] [MAIN ] ACP: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:01:41.402807] [WARN ] [MAIN ] ACP: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:01:41.402884] [INFO ] [LOCAL ] ACP: Starting deletion of primary data. [2025-09-16T17:01:41.402957] [DEBUG] [LOCAL ] ACP: No primary data file found to delete. [2025-09-16T17:01:41.403014] [INFO ] [LOCAL ] ACP: Starting deletion of derivative data. [2025-09-16T17:01:41.403329] [DEBUG] [LOCAL ] ACP: Deleted derivative data file. [2025-09-16T17:01:41.403422] [INFO ] [MAIN ] Update Progress: 23/2369 symbols completed. ACP took 16.1s. Estimated remaining: 2h 47m 8s. Rate: 14.0 symbols/min [2025-09-16T17:01:41.586795] [INFO ] [MAIN ] ACV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:41.593191] [DEBUG] [LOCAL ] ACV: Loaded 4587 existing records from primary storage. [2025-09-16T17:01:41.593308] [INFO ] [MAIN ] ACV: Loaded 4587 existing records. [2025-09-16T17:01:41.785229] [WARN ] [MAIN ] ACV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:01:41.785505] [INFO ] [MAIN ] ACV: Wiping existing primary data before full refresh. [2025-09-16T17:01:41.785567] [INFO ] [LOCAL ] ACV: Starting deletion of primary data. [2025-09-16T17:01:41.785865] [DEBUG] [LOCAL ] ACV: Deleted primary data file. [2025-09-16T17:01:41.785916] [INFO ] [MAIN ] ACV: Starting full primary refresh. [2025-09-16T17:01:41.980029] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-02. [2025-09-16T17:01:42.166702] [INFO ] [REMOTE ] ACV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:42.774897] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-02. [2025-09-16T17:01:43.175859] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-09-16T17:01:43.184515] [INFO ] [REMOTE ] ACN: Full fetch complete (58032 records). [2025-09-16T17:01:43.184749] [DEBUG] [VALIDATOR ] ACN: Validating primary data. [2025-09-16T17:01:43.442745] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-09-16T17:01:43.577477] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-03. [2025-09-16T17:01:43.614369] [DEBUG] [VALIDATOR ] ACN: All expected trading days are present. [2025-09-16T17:01:43.614565] [DEBUG] [VALIDATOR ] ACN: Primary data validation passed. [2025-09-16T17:01:43.615249] [INFO ] [MAIN ] ACN: Fetched primary data is valid. [2025-09-16T17:01:43.615358] [DEBUG] [LOCAL ] ACN: Saving primary data. [2025-09-16T17:01:43.637188] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T17:01:43.635836] [DEBUG] [LOCAL ] ACN: Trimmed 58032 records to 39800 (keeping 150 days of data) [2025-09-16T17:01:43.923357] [DEBUG] [LOCAL ] ACN: Primary data saved (39800 records). [2025-09-16T17:01:43.923591] [INFO ] [MAIN ] ACN: Full history saved. [2025-09-16T17:01:43.923726] [INFO ] [MAIN ] Update Progress: 24/2369 symbols completed. ACN took 28.1s. Estimated remaining: 2h 44m 13s. Rate: 14.3 symbols/min [2025-09-16T17:01:44.103047] [INFO ] [MAIN ] ACVA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:44.142004] [DEBUG] [LOCAL ] ACVA: Loaded 35821 existing records from primary storage. [2025-09-16T17:01:44.142146] [INFO ] [MAIN ] ACVA: Loaded 35821 existing records. [2025-09-16T17:01:44.327986] [WARN ] [MAIN ] ACVA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:01:44.328450] [INFO ] [MAIN ] ACVA: Wiping existing primary data before full refresh. [2025-09-16T17:01:44.328498] [INFO ] [LOCAL ] ACVA: Starting deletion of primary data. [2025-09-16T17:01:44.329499] [DEBUG] [LOCAL ] ACVA: Deleted primary data file. [2025-09-16T17:01:44.329551] [INFO ] [MAIN ] ACVA: Starting full primary refresh. [2025-09-16T17:01:44.384386] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-05. [2025-09-16T17:01:44.471197] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-09-16T17:01:44.709326] [INFO ] [REMOTE ] ACVA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:45.113984] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T17:01:45.174889] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-03. [2025-09-16T17:01:45.974927] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-02. [2025-09-16T17:01:46.459511] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-09-16T17:01:46.774946] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-04. [2025-09-16T17:01:47.374753] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-09-16T17:01:47.574915] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-04. [2025-09-16T17:01:48.238501] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T17:01:48.374918] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-05. [2025-09-16T17:01:48.398518] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-09-16T17:01:49.174903] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-03. [2025-09-16T17:01:49.721882] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T17:01:49.974918] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-05. [2025-09-16T17:01:50.774956] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-06. [2025-09-16T17:01:51.011619] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T17:01:51.012404] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-09-16T17:01:51.574928] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-06. [2025-09-16T17:01:51.678437] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-09-16T17:01:52.374912] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-04. [2025-09-16T17:01:52.965947] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T17:01:53.174912] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-06. [2025-09-16T17:01:53.828144] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-09-16T17:01:53.974924] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-07. [2025-09-16T17:01:54.042361] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T17:01:54.574960] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-09-16T17:01:54.774907] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-07. [2025-09-16T17:01:55.249138] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-09-16T17:01:55.574913] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-08. [2025-09-16T17:01:56.146590] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-09-16T17:01:56.374911] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-07. [2025-09-16T17:01:57.174915] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-09. [2025-09-16T17:01:57.876847] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-09-16T17:01:57.876634] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T17:01:57.885878] [INFO ] [REMOTE ] ACV: Full fetch complete (7400 records). [2025-09-16T17:01:57.886080] [DEBUG] [VALIDATOR ] ACV: Validating primary data. [2025-09-16T17:01:57.979986] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-08. [2025-09-16T17:01:58.249919] [DEBUG] [VALIDATOR ] ACV: All expected trading days are present. [2025-09-16T17:01:58.250003] [DEBUG] [VALIDATOR ] ACV: Primary data validation passed. [2025-09-16T17:01:58.250168] [INFO ] [MAIN ] ACV: Fetched primary data is valid. [2025-09-16T17:01:58.250210] [DEBUG] [LOCAL ] ACV: Saving primary data. [2025-09-16T17:01:58.250713] [DEBUG] [LOCAL ] ACV: Trimmed 7400 records to 4619 (keeping 150 days of data) [2025-09-16T17:01:58.280544] [DEBUG] [LOCAL ] ACV: Primary data saved (4619 records). [2025-09-16T17:01:58.280717] [INFO ] [MAIN ] ACV: Full history saved. [2025-09-16T17:01:58.280817] [INFO ] [MAIN ] Update Progress: 25/2369 symbols completed. ACV took 16.9s. Estimated remaining: 3h 0m 1s. Rate: 13.0 symbols/min [2025-09-16T17:01:58.455582] [INFO ] [MAIN ] AD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:01:58.455808] [DEBUG] [LOCAL ] AD: No existing data file found in primary storage. [2025-09-16T17:01:58.455848] [WARN ] [MAIN ] AD: No existing data found, performing full refresh instead. [2025-09-16T17:01:58.455969] [INFO ] [MAIN ] AD: Wiping existing primary data before full refresh. [2025-09-16T17:01:58.456009] [INFO ] [LOCAL ] AD: Starting deletion of primary data. [2025-09-16T17:01:58.456050] [DEBUG] [LOCAL ] AD: No primary data file found to delete. [2025-09-16T17:01:58.456083] [INFO ] [MAIN ] AD: Starting full primary refresh. [2025-09-16T17:01:58.785099] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-08. [2025-09-16T17:01:58.825519] [INFO ] [REMOTE ] AD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:01:59.177778] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-09-16T17:01:59.447765] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-09-16T17:01:59.574875] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-09. [2025-09-16T17:02:00.374895] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-02. [2025-09-16T17:02:01.012629] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T17:02:01.016523] [INFO ] [REMOTE ] ACRE: Full fetch complete (37187 records). [2025-09-16T17:02:01.016634] [DEBUG] [VALIDATOR ] ACRE: Validating primary data. [2025-09-16T17:02:01.094601] [DEBUG] [REMOTE ] AD: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:02:01.111237] [WARN ] [REMOTE ] AD: No data returned for month 2025-02 [2025-09-16T17:02:01.181266] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-09. [2025-09-16T17:02:01.414026] [DEBUG] [VALIDATOR ] ACRE: All expected trading days are present. [2025-09-16T17:02:01.414111] [DEBUG] [VALIDATOR ] ACRE: Primary data validation passed. [2025-09-16T17:02:01.414506] [INFO ] [MAIN ] ACRE: Fetched primary data is valid. [2025-09-16T17:02:01.414549] [DEBUG] [LOCAL ] ACRE: Saving primary data. [2025-09-16T17:02:01.415404] [DEBUG] [LOCAL ] ACRE: Trimmed 37187 records to 24288 (keeping 150 days of data) [2025-09-16T17:02:01.565109] [DEBUG] [LOCAL ] ACRE: Primary data saved (24288 records). [2025-09-16T17:02:01.565262] [INFO ] [MAIN ] ACRE: Full history saved. [2025-09-16T17:02:01.565344] [INFO ] [MAIN ] Update Progress: 26/2369 symbols completed. ACRE took 26.4s. Estimated remaining: 2h 57m 57s. Rate: 13.2 symbols/min [2025-09-16T17:02:01.766509] [INFO ] [MAIN ] ADC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:02:01.807243] [DEBUG] [LOCAL ] ADC: Loaded 34268 existing records from primary storage. [2025-09-16T17:02:01.807392] [INFO ] [MAIN ] ADC: Loaded 34268 existing records. [2025-09-16T17:02:01.989722] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-03. [2025-09-16T17:02:01.994553] [WARN ] [MAIN ] ADC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:02:01.995541] [INFO ] [MAIN ] ADC: Wiping existing primary data before full refresh. [2025-09-16T17:02:01.995590] [INFO ] [LOCAL ] ADC: Starting deletion of primary data. [2025-09-16T17:02:01.996424] [DEBUG] [LOCAL ] ADC: Deleted primary data file. [2025-09-16T17:02:01.996477] [INFO ] [MAIN ] ADC: Starting full primary refresh. [2025-09-16T17:02:02.296827] [DEBUG] [REMOTE ] AD: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:02:02.308835] [WARN ] [REMOTE ] AD: No data returned for month 2025-03 [2025-09-16T17:02:02.384267] [INFO ] [REMOTE ] ADC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:02:02.774944] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-04. [2025-09-16T17:02:03.068618] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-09-16T17:02:03.069815] [INFO ] [REMOTE ] ACR: Full fetch complete (7063 records). [2025-09-16T17:02:03.069887] [DEBUG] [VALIDATOR ] ACR: Validating primary data. [2025-09-16T17:02:03.246491] [DEBUG] [REMOTE ] AD: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:02:03.266834] [WARN ] [REMOTE ] AD: No data returned for month 2025-04 [2025-09-16T17:02:03.463910] [DEBUG] [VALIDATOR ] ACR: All expected trading days are present. [2025-09-16T17:02:03.463994] [DEBUG] [VALIDATOR ] ACR: Primary data validation passed. [2025-09-16T17:02:03.464193] [INFO ] [MAIN ] ACR: Fetched primary data is valid. [2025-09-16T17:02:03.464235] [DEBUG] [LOCAL ] ACR: Saving primary data. [2025-09-16T17:02:03.464676] [DEBUG] [LOCAL ] ACR: Trimmed 7063 records to 4172 (keeping 150 days of data) [2025-09-16T17:02:03.490660] [DEBUG] [LOCAL ] ACR: Primary data saved (4172 records). [2025-09-16T17:02:03.490792] [INFO ] [MAIN ] ACR: Full history saved. [2025-09-16T17:02:03.490882] [INFO ] [MAIN ] Update Progress: 27/2369 symbols completed. ACR took 32.7s. Estimated remaining: 2h 54m 4s. Rate: 13.5 symbols/min [2025-09-16T17:02:03.580140] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-05. [2025-09-16T17:02:03.664629] [INFO ] [MAIN ] ADCT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:02:03.700561] [DEBUG] [LOCAL ] ADCT: Loaded 25686 existing records from primary storage. [2025-09-16T17:02:03.700720] [INFO ] [MAIN ] ADCT: Loaded 25686 existing records. [2025-09-16T17:02:03.837668] [DEBUG] [REMOTE ] AD: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:02:03.876369] [WARN ] [REMOTE ] AD: No data returned for month 2025-05 [2025-09-16T17:02:03.899698] [WARN ] [MAIN ] ADCT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:02:03.900515] [INFO ] [MAIN ] ADCT: Wiping existing primary data before full refresh. [2025-09-16T17:02:03.900566] [INFO ] [LOCAL ] ADCT: Starting deletion of primary data. [2025-09-16T17:02:03.901164] [DEBUG] [LOCAL ] ADCT: Deleted primary data file. [2025-09-16T17:02:03.901218] [INFO ] [MAIN ] ADCT: Starting full primary refresh. [2025-09-16T17:02:04.269457] [INFO ] [REMOTE ] ADCT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:02:04.374898] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-02. [2025-09-16T17:02:05.174951] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-06. [2025-09-16T17:02:05.397534] [DEBUG] [REMOTE ] AD: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:02:05.397924] [WARN ] [REMOTE ] AD: No data returned for month 2025-06 [2025-09-16T17:02:05.697206] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-09-16T17:02:05.974962] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-07. [2025-09-16T17:02:06.161343] [DEBUG] [REMOTE ] AD: Error fetching primary data for 2025-07: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:02:06.161685] [WARN ] [REMOTE ] AD: No data returned for month 2025-07 [2025-09-16T17:02:06.774906] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-08. [2025-09-16T17:02:07.574921] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-02. [2025-09-16T17:02:07.911385] [INFO ] [REMOTE ] AD: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T17:02:08.374973] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-09. [2025-09-16T17:02:08.751860] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-09-16T17:02:09.174905] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-03. [2025-09-16T17:02:09.498151] [INFO ] [REMOTE ] AD: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-09-16T17:02:09.502737] [INFO ] [REMOTE ] AD: Full fetch complete (4957 records). [2025-09-16T17:02:09.503017] [DEBUG] [VALIDATOR ] AD: Validating primary data. [2025-09-16T17:02:09.703823] [WARN ] [VALIDATOR ] AD: Validation failed - insufficient historical data. Have 25 unique days, need 150 days [2025-09-16T17:02:09.704072] [FATAL] [MAIN ] AD: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:02:09.704117] [WARN ] [MAIN ] AD: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:02:09.704161] [INFO ] [LOCAL ] AD: Starting deletion of primary data. [2025-09-16T17:02:09.704250] [DEBUG] [LOCAL ] AD: No primary data file found to delete. [2025-09-16T17:02:09.704290] [INFO ] [LOCAL ] AD: Starting deletion of derivative data. [2025-09-16T17:02:09.704331] [DEBUG] [LOCAL ] AD: No derivative data file found to delete. [2025-09-16T17:02:09.704391] [INFO ] [MAIN ] Update Progress: 28/2369 symbols completed. AD took 11.4s. Estimated remaining: 2h 56m 26s. Rate: 13.3 symbols/min [2025-09-16T17:02:09.904840] [INFO ] [MAIN ] ADM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:02:09.946049] [DEBUG] [LOCAL ] ADM: Loaded 39393 existing records from primary storage. [2025-09-16T17:02:09.946201] [INFO ] [MAIN ] ADM: Loaded 39393 existing records. [2025-09-16T17:02:09.980113] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-05. [2025-09-16T17:02:10.130242] [WARN ] [MAIN ] ADM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:02:10.130482] [INFO ] [MAIN ] ADM: Wiping existing primary data before full refresh. [2025-09-16T17:02:10.130537] [INFO ] [LOCAL ] ADM: Starting deletion of primary data. [2025-09-16T17:02:10.131558] [DEBUG] [LOCAL ] ADM: Deleted primary data file. [2025-09-16T17:02:10.131620] [INFO ] [MAIN ] ADM: Starting full primary refresh. [2025-09-16T17:02:10.489160] [INFO ] [REMOTE ] ADM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:02:10.580804] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-09-16T17:02:10.774932] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-02. [2025-09-16T17:02:11.574937] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-03. [2025-09-16T17:02:12.265438] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-05 (2.29 seconds). [2025-09-16T17:02:12.374982] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-06. [2025-09-16T17:02:12.634579] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-09-16T17:02:13.174949] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-03. [2025-09-16T17:02:13.260050] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-09-16T17:02:13.440586] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T17:02:13.974922] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-07. [2025-09-16T17:02:14.774912] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-04. [2025-09-16T17:02:14.907456] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T17:02:15.235587] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T17:02:15.575058] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-04. [2025-09-16T17:02:16.124044] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T17:02:16.374985] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-05. [2025-09-16T17:02:17.174964] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-08. [2025-09-16T17:02:17.365936] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T17:02:17.687691] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T17:02:17.974934] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-06. [2025-09-16T17:02:18.774921] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-04. [2025-09-16T17:02:18.965891] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T17:02:19.334316] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-09-16T17:02:19.574934] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-09. [2025-09-16T17:02:20.374917] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-05. [2025-09-16T17:02:20.538024] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T17:02:21.174908] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-05. [2025-09-16T17:02:21.248335] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-09-16T17:02:21.254499] [INFO ] [REMOTE ] ACVA: Full fetch complete (53622 records). [2025-09-16T17:02:21.254619] [DEBUG] [VALIDATOR ] ACVA: Validating primary data. [2025-09-16T17:02:21.665940] [DEBUG] [VALIDATOR ] ACVA: All expected trading days are present. [2025-09-16T17:02:21.666021] [DEBUG] [VALIDATOR ] ACVA: Primary data validation passed. [2025-09-16T17:02:21.666525] [INFO ] [MAIN ] ACVA: Fetched primary data is valid. [2025-09-16T17:02:21.666570] [DEBUG] [LOCAL ] ACVA: Saving primary data. [2025-09-16T17:02:21.667686] [DEBUG] [LOCAL ] ACVA: Trimmed 53622 records to 36184 (keeping 150 days of data) [2025-09-16T17:02:21.886715] [DEBUG] [LOCAL ] ACVA: Primary data saved (36184 records). [2025-09-16T17:02:21.886954] [INFO ] [MAIN ] ACVA: Full history saved. [2025-09-16T17:02:21.887057] [INFO ] [MAIN ] Update Progress: 29/2369 symbols completed. ACVA took 38.0s. Estimated remaining: 3h 6m 40s. Rate: 12.5 symbols/min [2025-09-16T17:02:21.980159] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-07. [2025-09-16T17:02:22.076913] [INFO ] [MAIN ] ADNT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:02:22.077251] [DEBUG] [LOCAL ] ADNT: No existing data file found in primary storage. [2025-09-16T17:02:22.077323] [WARN ] [MAIN ] ADNT: No existing data found, performing full refresh instead. [2025-09-16T17:02:22.077515] [INFO ] [MAIN ] ADNT: Wiping existing primary data before full refresh. [2025-09-16T17:02:22.077560] [INFO ] [LOCAL ] ADNT: Starting deletion of primary data. [2025-09-16T17:02:22.077601] [DEBUG] [LOCAL ] ADNT: No primary data file found to delete. [2025-09-16T17:02:22.077633] [INFO ] [MAIN ] ADNT: Starting full primary refresh. [2025-09-16T17:02:22.413662] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-09-16T17:02:22.501168] [INFO ] [REMOTE ] ADNT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:02:22.774940] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-02. [2025-09-16T17:02:22.822016] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T17:02:23.463537] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T17:02:23.574855] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-08. [2025-09-16T17:02:24.375764] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T17:02:24.375908] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-06. [2025-09-16T17:02:24.724109] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T17:02:25.174927] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-03. [2025-09-16T17:02:25.974952] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-06. [2025-09-16T17:02:26.068790] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T17:02:26.780035] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-07. [2025-09-16T17:02:26.802140] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T17:02:27.490785] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T17:02:27.574866] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-07. [2025-09-16T17:02:28.374878] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-09. [2025-09-16T17:02:28.459478] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-09-16T17:02:29.040331] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T17:02:29.174974] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-04. [2025-09-16T17:02:29.504172] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T17:02:29.508467] [INFO ] [REMOTE ] ADCT: Full fetch complete (36479 records). [2025-09-16T17:02:29.508574] [DEBUG] [VALIDATOR ] ADCT: Validating primary data. [2025-09-16T17:02:29.888591] [DEBUG] [VALIDATOR ] ADCT: All expected trading days are present. [2025-09-16T17:02:29.888695] [DEBUG] [VALIDATOR ] ADCT: Primary data validation passed. [2025-09-16T17:02:29.889081] [INFO ] [MAIN ] ADCT: Fetched primary data is valid. [2025-09-16T17:02:29.889125] [DEBUG] [LOCAL ] ADCT: Saving primary data. [2025-09-16T17:02:29.889984] [DEBUG] [LOCAL ] ADCT: Trimmed 36479 records to 25885 (keeping 150 days of data) [2025-09-16T17:02:29.975549] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-08. [2025-09-16T17:02:30.040676] [DEBUG] [LOCAL ] ADCT: Primary data saved (25885 records). [2025-09-16T17:02:30.040819] [INFO ] [MAIN ] ADCT: Full history saved. [2025-09-16T17:02:30.040928] [INFO ] [MAIN ] Update Progress: 30/2369 symbols completed. ADCT took 26.5s. Estimated remaining: 3h 10m 57s. Rate: 12.2 symbols/min [2025-09-16T17:02:30.218421] [INFO ] [MAIN ] ADT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:02:30.257272] [DEBUG] [LOCAL ] ADT: Loaded 39482 existing records from primary storage. [2025-09-16T17:02:30.257429] [INFO ] [MAIN ] ADT: Loaded 39482 existing records. [2025-09-16T17:02:30.437525] [WARN ] [MAIN ] ADT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:02:30.437776] [INFO ] [MAIN ] ADT: Wiping existing primary data before full refresh. [2025-09-16T17:02:30.437826] [INFO ] [LOCAL ] ADT: Starting deletion of primary data. [2025-09-16T17:02:30.438726] [DEBUG] [LOCAL ] ADT: Deleted primary data file. [2025-09-16T17:02:30.438778] [INFO ] [MAIN ] ADT: Starting full primary refresh. [2025-09-16T17:02:30.776725] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-08. [2025-09-16T17:02:30.802841] [INFO ] [REMOTE ] ADT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:02:30.866615] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T17:02:31.323432] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-09-16T17:02:31.574952] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-05. [2025-09-16T17:02:32.374941] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-09. [2025-09-16T17:02:32.555576] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-09-16T17:02:33.174915] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-09. [2025-09-16T17:02:33.216055] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T17:02:33.798389] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T17:02:33.804481] [INFO ] [REMOTE ] ADC: Full fetch complete (50715 records). [2025-09-16T17:02:33.804674] [DEBUG] [VALIDATOR ] ADC: Validating primary data. [2025-09-16T17:02:33.982873] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-06. [2025-09-16T17:02:34.193108] [DEBUG] [VALIDATOR ] ADC: All expected trading days are present. [2025-09-16T17:02:34.193194] [DEBUG] [VALIDATOR ] ADC: Primary data validation passed. [2025-09-16T17:02:34.193800] [INFO ] [MAIN ] ADC: Fetched primary data is valid. [2025-09-16T17:02:34.193850] [DEBUG] [LOCAL ] ADC: Saving primary data. [2025-09-16T17:02:34.194953] [DEBUG] [LOCAL ] ADC: Trimmed 50715 records to 34611 (keeping 150 days of data) [2025-09-16T17:02:34.414361] [DEBUG] [LOCAL ] ADC: Primary data saved (34611 records). [2025-09-16T17:02:34.414504] [INFO ] [MAIN ] ADC: Full history saved. [2025-09-16T17:02:34.415959] [INFO ] [MAIN ] Update Progress: 31/2369 symbols completed. ADC took 32.9s. Estimated remaining: 3h 10m 13s. Rate: 12.3 symbols/min [2025-09-16T17:02:34.589356] [INFO ] [MAIN ] ADX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:02:34.611323] [DEBUG] [LOCAL ] ADX: Loaded 20404 existing records from primary storage. [2025-09-16T17:02:34.611467] [INFO ] [MAIN ] ADX: Loaded 20404 existing records. [2025-09-16T17:02:34.776259] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-02. [2025-09-16T17:02:34.800385] [WARN ] [MAIN ] ADX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:02:34.800936] [INFO ] [MAIN ] ADX: Wiping existing primary data before full refresh. [2025-09-16T17:02:34.801053] [INFO ] [LOCAL ] ADX: Starting deletion of primary data. [2025-09-16T17:02:34.808275] [DEBUG] [LOCAL ] ADX: Deleted primary data file. [2025-09-16T17:02:34.820963] [INFO ] [MAIN ] ADX: Starting full primary refresh. [2025-09-16T17:02:34.828294] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-09-16T17:02:34.906626] [INFO ] [REMOTE ] ADM: Full fetch complete (58069 records). [2025-09-16T17:02:34.939014] [DEBUG] [VALIDATOR ] ADM: Validating primary data. [2025-09-16T17:02:35.509508] [INFO ] [REMOTE ] ADX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:02:35.582581] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-02. [2025-09-16T17:02:35.601515] [DEBUG] [VALIDATOR ] ADM: All expected trading days are present. [2025-09-16T17:02:35.601610] [DEBUG] [VALIDATOR ] ADM: Primary data validation passed. [2025-09-16T17:02:35.602277] [INFO ] [MAIN ] ADM: Fetched primary data is valid. [2025-09-16T17:02:35.602331] [DEBUG] [LOCAL ] ADM: Saving primary data. [2025-09-16T17:02:35.603975] [DEBUG] [LOCAL ] ADM: Trimmed 58069 records to 39782 (keeping 150 days of data) [2025-09-16T17:02:35.760046] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-09-16T17:02:35.886941] [DEBUG] [LOCAL ] ADM: Primary data saved (39782 records). [2025-09-16T17:02:35.887106] [INFO ] [MAIN ] ADM: Full history saved. [2025-09-16T17:02:35.887199] [INFO ] [MAIN ] Update Progress: 32/2369 symbols completed. ADM took 26.2s. Estimated remaining: 3h 5m 59s. Rate: 12.6 symbols/min [2025-09-16T17:02:36.058549] [INFO ] [MAIN ] AEE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:02:36.058732] [DEBUG] [LOCAL ] AEE: No existing data file found in primary storage. [2025-09-16T17:02:36.058772] [WARN ] [MAIN ] AEE: No existing data found, performing full refresh instead. [2025-09-16T17:02:36.058893] [INFO ] [MAIN ] AEE: Wiping existing primary data before full refresh. [2025-09-16T17:02:36.058933] [INFO ] [LOCAL ] AEE: Starting deletion of primary data. [2025-09-16T17:02:36.058975] [DEBUG] [LOCAL ] AEE: No primary data file found to delete. [2025-09-16T17:02:36.059008] [INFO ] [MAIN ] AEE: Starting full primary refresh. [2025-09-16T17:02:36.390753] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-07. [2025-09-16T17:02:36.427296] [INFO ] [REMOTE ] AEE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:02:36.457554] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T17:02:37.084958] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-09-16T17:02:37.174909] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-03. [2025-09-16T17:02:37.667283] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-09-16T17:02:37.974940] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-02. [2025-09-16T17:02:38.760750] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-09-16T17:02:38.774853] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-04. [2025-09-16T17:02:39.426673] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-09-16T17:02:39.574935] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-08. [2025-09-16T17:02:40.228462] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T17:02:40.374934] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-03. [2025-09-16T17:02:41.174972] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-03. [2025-09-16T17:02:41.248286] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T17:02:41.974929] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-09. [2025-09-16T17:02:42.189112] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T17:02:42.587226] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-09-16T17:02:42.774959] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-04. [2025-09-16T17:02:43.443229] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T17:02:43.449359] [INFO ] [REMOTE ] ADNT: Full fetch complete (51503 records). [2025-09-16T17:02:43.449546] [DEBUG] [VALIDATOR ] ADNT: Validating primary data. [2025-09-16T17:02:43.580093] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-04. [2025-09-16T17:02:43.883149] [DEBUG] [VALIDATOR ] ADNT: All expected trading days are present. [2025-09-16T17:02:43.883273] [DEBUG] [VALIDATOR ] ADNT: Primary data validation passed. [2025-09-16T17:02:43.883904] [INFO ] [MAIN ] ADNT: Fetched primary data is valid. [2025-09-16T17:02:43.883981] [DEBUG] [LOCAL ] ADNT: Saving primary data. [2025-09-16T17:02:43.885255] [DEBUG] [LOCAL ] ADNT: Trimmed 51503 records to 35081 (keeping 150 days of data) [2025-09-16T17:02:44.090467] [DEBUG] [LOCAL ] ADNT: Primary data saved (35081 records). [2025-09-16T17:02:44.090635] [INFO ] [MAIN ] ADNT: Full history saved. [2025-09-16T17:02:44.090742] [INFO ] [MAIN ] Update Progress: 33/2369 symbols completed. ADNT took 22.2s. Estimated remaining: 3h 9m 57s. Rate: 12.3 symbols/min [2025-09-16T17:02:44.268736] [INFO ] [MAIN ] AEFC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:02:44.275902] [DEBUG] [LOCAL ] AEFC: Loaded 5866 existing records from primary storage. [2025-09-16T17:02:44.276021] [INFO ] [MAIN ] AEFC: Loaded 5866 existing records. [2025-09-16T17:02:44.381132] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-05. [2025-09-16T17:02:44.465176] [WARN ] [MAIN ] AEFC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:02:44.465417] [INFO ] [MAIN ] AEFC: Wiping existing primary data before full refresh. [2025-09-16T17:02:44.465465] [INFO ] [LOCAL ] AEFC: Starting deletion of primary data. [2025-09-16T17:02:44.465839] [DEBUG] [LOCAL ] AEFC: Deleted primary data file. [2025-09-16T17:02:44.465887] [INFO ] [MAIN ] AEFC: Starting full primary refresh. [2025-09-16T17:02:44.852793] [INFO ] [REMOTE ] AEFC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:02:44.904664] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-04 (2.13 seconds). [2025-09-16T17:02:44.905935] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T17:02:45.174915] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-05. [2025-09-16T17:02:45.974880] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-02. [2025-09-16T17:02:46.008948] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-09-16T17:02:46.772230] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-09-16T17:02:46.774840] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-03. [2025-09-16T17:02:46.932920] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T17:02:47.365772] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-09-16T17:02:47.574937] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-04. [2025-09-16T17:02:48.374910] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-06. [2025-09-16T17:02:48.420312] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-09-16T17:02:49.174909] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-05. [2025-09-16T17:02:49.974945] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-05. [2025-09-16T17:02:50.039453] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T17:02:50.200509] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T17:02:50.774918] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-06. [2025-09-16T17:02:51.275744] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T17:02:51.491166] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-09-16T17:02:51.574917] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-06. [2025-09-16T17:02:52.374943] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-07. [2025-09-16T17:02:52.897819] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T17:02:53.174927] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-07. [2025-09-16T17:02:53.974912] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-07. [2025-09-16T17:02:54.091098] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-09-16T17:02:54.479715] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-09-16T17:02:54.523870] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-09-16T17:02:54.774913] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-08. [2025-09-16T17:02:55.574895] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-06. [2025-09-16T17:02:55.716673] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-09-16T17:02:56.374891] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-09. [2025-09-16T17:02:57.174869] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-08. [2025-09-16T17:02:57.269229] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T17:02:57.525870] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T17:02:57.530082] [INFO ] [REMOTE ] ADX: Full fetch complete (30720 records). [2025-09-16T17:02:57.530190] [DEBUG] [VALIDATOR ] ADX: Validating primary data. [2025-09-16T17:02:57.938134] [DEBUG] [VALIDATOR ] ADX: All expected trading days are present. [2025-09-16T17:02:57.938234] [DEBUG] [VALIDATOR ] ADX: Primary data validation passed. [2025-09-16T17:02:57.938622] [INFO ] [MAIN ] ADX: Fetched primary data is valid. [2025-09-16T17:02:57.938686] [DEBUG] [LOCAL ] ADX: Saving primary data. [2025-09-16T17:02:57.939616] [DEBUG] [LOCAL ] ADX: Trimmed 30720 records to 20576 (keeping 150 days of data) [2025-09-16T17:02:57.997739] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-08. [2025-09-16T17:02:58.075718] [DEBUG] [LOCAL ] ADX: Primary data saved (20576 records). [2025-09-16T17:02:58.075955] [INFO ] [MAIN ] ADX: Full history saved. [2025-09-16T17:02:58.076113] [INFO ] [MAIN ] Update Progress: 34/2369 symbols completed. ADX took 23.7s. Estimated remaining: 3h 20m 18s. Rate: 11.7 symbols/min [2025-09-16T17:02:58.088746] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-09-16T17:02:58.264241] [INFO ] [MAIN ] AEG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:02:58.309952] [DEBUG] [LOCAL ] AEG: Loaded 36681 existing records from primary storage. [2025-09-16T17:02:58.310112] [INFO ] [MAIN ] AEG: Loaded 36681 existing records. [2025-09-16T17:02:58.490837] [WARN ] [MAIN ] AEG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:02:58.491136] [INFO ] [MAIN ] AEG: Wiping existing primary data before full refresh. [2025-09-16T17:02:58.491188] [INFO ] [LOCAL ] AEG: Starting deletion of primary data. [2025-09-16T17:02:58.492076] [DEBUG] [LOCAL ] AEG: Deleted primary data file. [2025-09-16T17:02:58.492129] [INFO ] [MAIN ] AEG: Starting full primary refresh. [2025-09-16T17:02:58.780097] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-07. [2025-09-16T17:02:58.852413] [INFO ] [REMOTE ] AEG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:02:59.575126] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-09. [2025-09-16T17:02:59.824464] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-09-16T17:03:00.374922] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-09. [2025-09-16T17:03:00.385316] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-09-16T17:03:00.387291] [INFO ] [REMOTE ] AEFC: Full fetch complete (9465 records). [2025-09-16T17:03:00.387429] [DEBUG] [VALIDATOR ] AEFC: Validating primary data. [2025-09-16T17:03:00.776938] [DEBUG] [VALIDATOR ] AEFC: All expected trading days are present. [2025-09-16T17:03:00.777038] [DEBUG] [VALIDATOR ] AEFC: Primary data validation passed. [2025-09-16T17:03:00.777418] [INFO ] [MAIN ] AEFC: Fetched primary data is valid. [2025-09-16T17:03:00.777504] [DEBUG] [LOCAL ] AEFC: Saving primary data. [2025-09-16T17:03:00.778156] [DEBUG] [LOCAL ] AEFC: Trimmed 9465 records to 5914 (keeping 150 days of data) [2025-09-16T17:03:00.841172] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-09-16T17:03:00.860560] [DEBUG] [LOCAL ] AEFC: Primary data saved (5914 records). [2025-09-16T17:03:00.860718] [INFO ] [MAIN ] AEFC: Full history saved. [2025-09-16T17:03:00.860810] [INFO ] [MAIN ] Update Progress: 35/2369 symbols completed. AEFC took 16.8s. Estimated remaining: 3h 17m 35s. Rate: 11.8 symbols/min [2025-09-16T17:03:01.031433] [INFO ] [MAIN ] AEM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:01.072995] [DEBUG] [LOCAL ] AEM: Loaded 39467 existing records from primary storage. [2025-09-16T17:03:01.073164] [INFO ] [MAIN ] AEM: Loaded 39467 existing records. [2025-09-16T17:03:01.175148] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-02. [2025-09-16T17:03:01.255894] [WARN ] [MAIN ] AEM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:01.256113] [INFO ] [MAIN ] AEM: Wiping existing primary data before full refresh. [2025-09-16T17:03:01.256159] [INFO ] [LOCAL ] AEM: Starting deletion of primary data. [2025-09-16T17:03:01.257121] [DEBUG] [LOCAL ] AEM: Deleted primary data file. [2025-09-16T17:03:01.257173] [INFO ] [MAIN ] AEM: Starting full primary refresh. [2025-09-16T17:03:01.627084] [INFO ] [REMOTE ] AEM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:01.974974] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-02. [2025-09-16T17:03:02.774963] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-08. [2025-09-16T17:03:03.328877] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-02 (2.15 seconds). [2025-09-16T17:03:03.475676] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-09 (3.10 seconds). [2025-09-16T17:03:03.482314] [INFO ] [REMOTE ] ADT: Full fetch complete (58078 records). [2025-09-16T17:03:03.482425] [DEBUG] [VALIDATOR ] ADT: Validating primary data. [2025-09-16T17:03:03.579983] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-03. [2025-09-16T17:03:03.874602] [DEBUG] [VALIDATOR ] ADT: All expected trading days are present. [2025-09-16T17:03:03.874698] [DEBUG] [VALIDATOR ] ADT: Primary data validation passed. [2025-09-16T17:03:03.875305] [INFO ] [MAIN ] ADT: Fetched primary data is valid. [2025-09-16T17:03:03.875351] [DEBUG] [LOCAL ] ADT: Saving primary data. [2025-09-16T17:03:03.876554] [DEBUG] [LOCAL ] ADT: Trimmed 58078 records to 39870 (keeping 150 days of data) [2025-09-16T17:03:04.128020] [DEBUG] [LOCAL ] ADT: Primary data saved (39870 records). [2025-09-16T17:03:04.128190] [INFO ] [MAIN ] ADT: Full history saved. [2025-09-16T17:03:04.129128] [INFO ] [MAIN ] Update Progress: 36/2369 symbols completed. ADT took 34.1s. Estimated remaining: 3h 15m 33s. Rate: 11.9 symbols/min [2025-09-16T17:03:04.326773] [INFO ] [MAIN ] AEO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:04.371735] [DEBUG] [LOCAL ] AEO: Loaded 39531 existing records from primary storage. [2025-09-16T17:03:04.371911] [INFO ] [MAIN ] AEO: Loaded 39531 existing records. [2025-09-16T17:03:04.577701] [WARN ] [MAIN ] AEO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:04.578005] [INFO ] [MAIN ] AEO: Wiping existing primary data before full refresh. [2025-09-16T17:03:04.578058] [INFO ] [LOCAL ] AEO: Starting deletion of primary data. [2025-09-16T17:03:04.578983] [DEBUG] [LOCAL ] AEO: Deleted primary data file. [2025-09-16T17:03:04.579034] [INFO ] [MAIN ] AEO: Starting full primary refresh. [2025-09-16T17:03:04.892792] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-09-16T17:03:04.968754] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-09. [2025-09-16T17:03:05.046430] [INFO ] [REMOTE ] AEO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:05.174946] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-02. [2025-09-16T17:03:05.232983] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-09-16T17:03:05.757480] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-02 (3.78 seconds). [2025-09-16T17:03:05.974938] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-04. [2025-09-16T17:03:06.774944] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-03. [2025-09-16T17:03:06.955359] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-09-16T17:03:06.961685] [INFO ] [REMOTE ] AEE: Full fetch complete (54561 records). [2025-09-16T17:03:06.961787] [DEBUG] [VALIDATOR ] AEE: Validating primary data. [2025-09-16T17:03:06.962423] [FATAL] [VALIDATOR ] AEE: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:03:06.962519] [FATAL] [MAIN ] AEE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:03:06.962556] [WARN ] [MAIN ] AEE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:03:06.962594] [INFO ] [LOCAL ] AEE: Starting deletion of primary data. [2025-09-16T17:03:06.962670] [DEBUG] [LOCAL ] AEE: No primary data file found to delete. [2025-09-16T17:03:06.962710] [INFO ] [LOCAL ] AEE: Starting deletion of derivative data. [2025-09-16T17:03:06.962765] [DEBUG] [LOCAL ] AEE: No derivative data file found to delete. [2025-09-16T17:03:06.962825] [INFO ] [MAIN ] Update Progress: 37/2369 symbols completed. AEE took 31.1s. Estimated remaining: 3h 13m 9s. Rate: 12.1 symbols/min [2025-09-16T17:03:07.658932] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-02 (2.48 seconds). [2025-09-16T17:03:07.669429] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-03. [2025-09-16T17:03:07.701366] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T17:03:08.374956] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-05. [2025-09-16T17:03:08.914242] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-09-16T17:03:09.174984] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-04. [2025-09-16T17:03:09.287146] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T17:03:09.679809] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T17:03:09.974970] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-06. [2025-09-16T17:03:10.774902] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-04. [2025-09-16T17:03:11.597794] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-04 (2.42 seconds). [2025-09-16T17:03:11.609388] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-05. [2025-09-16T17:03:11.849700] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-09-16T17:03:12.379702] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-07. [2025-09-16T17:03:12.382064] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-09-16T17:03:13.174872] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-05. [2025-09-16T17:03:13.348063] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T17:03:13.883067] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T17:03:13.974886] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-06. [2025-09-16T17:03:14.774912] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-08. [2025-09-16T17:03:15.038049] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-09-16T17:03:15.574926] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-06. [2025-09-16T17:03:15.923022] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-09-16T17:03:16.374897] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-07. [2025-09-16T17:03:16.723103] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-09-16T17:03:17.021040] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T17:03:17.174880] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-07. [2025-09-16T17:03:17.974916] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-09. [2025-09-16T17:03:18.864463] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-07 (2.49 seconds). [2025-09-16T17:03:18.881332] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-08. [2025-09-16T17:03:19.420194] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-09-16T17:03:19.470512] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T17:03:19.476974] [INFO ] [REMOTE ] AEG: Full fetch complete (53716 records). [2025-09-16T17:03:19.477138] [DEBUG] [VALIDATOR ] AEG: Validating primary data. [2025-09-16T17:03:19.587221] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-08. [2025-09-16T17:03:19.902159] [DEBUG] [VALIDATOR ] AEG: All expected trading days are present. [2025-09-16T17:03:19.902239] [DEBUG] [VALIDATOR ] AEG: Primary data validation passed. [2025-09-16T17:03:19.902787] [INFO ] [MAIN ] AEG: Fetched primary data is valid. [2025-09-16T17:03:19.902836] [DEBUG] [LOCAL ] AEG: Saving primary data. [2025-09-16T17:03:19.903898] [DEBUG] [LOCAL ] AEG: Trimmed 53716 records to 37046 (keeping 150 days of data) [2025-09-16T17:03:20.138032] [DEBUG] [LOCAL ] AEG: Primary data saved (37046 records). [2025-09-16T17:03:20.138216] [INFO ] [MAIN ] AEG: Full history saved. [2025-09-16T17:03:20.139360] [INFO ] [MAIN ] Update Progress: 38/2369 symbols completed. AEG took 22.1s. Estimated remaining: 3h 21m 28s. Rate: 11.6 symbols/min [2025-09-16T17:03:20.848536] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-09-16T17:03:20.861140] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-09. [2025-09-16T17:03:21.792085] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-08 (2.20 seconds). [2025-09-16T17:03:21.806032] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-09. [2025-09-16T17:03:22.338589] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T17:03:22.344601] [INFO ] [REMOTE ] AEM: Full fetch complete (57744 records). [2025-09-16T17:03:22.344750] [DEBUG] [VALIDATOR ] AEM: Validating primary data. [2025-09-16T17:03:22.345524] [FATAL] [VALIDATOR ] AEM: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:03:22.345619] [FATAL] [MAIN ] AEM: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:03:22.345749] [WARN ] [MAIN ] AEM: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:03:22.345797] [INFO ] [LOCAL ] AEM: Starting deletion of primary data. [2025-09-16T17:03:22.345862] [DEBUG] [LOCAL ] AEM: No primary data file found to delete. [2025-09-16T17:03:22.345900] [INFO ] [LOCAL ] AEM: Starting deletion of derivative data. [2025-09-16T17:03:22.346176] [DEBUG] [LOCAL ] AEM: Deleted derivative data file. [2025-09-16T17:03:22.348337] [INFO ] [MAIN ] Update Progress: 39/2369 symbols completed. AEM took 21.5s. Estimated remaining: 3h 18m 25s. Rate: 11.7 symbols/min [2025-09-16T17:03:23.957782] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-09 (2.15 seconds). [2025-09-16T17:03:23.966259] [INFO ] [REMOTE ] AEO: Full fetch complete (58200 records). [2025-09-16T17:03:23.966551] [DEBUG] [VALIDATOR ] AEO: Validating primary data. [2025-09-16T17:03:24.357725] [DEBUG] [VALIDATOR ] AEO: All expected trading days are present. [2025-09-16T17:03:24.357879] [DEBUG] [VALIDATOR ] AEO: Primary data validation passed. [2025-09-16T17:03:24.358486] [INFO ] [MAIN ] AEO: Fetched primary data is valid. [2025-09-16T17:03:24.358556] [DEBUG] [LOCAL ] AEO: Saving primary data. [2025-09-16T17:03:24.359893] [DEBUG] [LOCAL ] AEO: Trimmed 58200 records to 39921 (keeping 150 days of data) [2025-09-16T17:03:24.606017] [DEBUG] [LOCAL ] AEO: Primary data saved (39921 records). [2025-09-16T17:03:24.606164] [INFO ] [MAIN ] AEO: Full history saved. [2025-09-16T17:03:24.606983] [INFO ] [MAIN ] Update Progress: 40/2369 symbols completed. AEO took 20.5s. Estimated remaining: 3h 15m 33s. Rate: 11.9 symbols/min [2025-09-16T17:03:25.319630] [INFO ] [MAIN ] AFB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:25.340191] [INFO ] [MAIN ] AESI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:25.316164] [INFO ] [MAIN ] AES: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:25.342871] [INFO ] [MAIN ] AER: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:25.361927] [DEBUG] [LOCAL ] AFB: Loaded 7849 existing records from primary storage. [2025-09-16T17:03:25.362076] [INFO ] [MAIN ] AFB: Loaded 7849 existing records. [2025-09-16T17:03:25.599739] [WARN ] [MAIN ] AFB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:25.612951] [DEBUG] [LOCAL ] AESI: Loaded 36439 existing records from primary storage. [2025-09-16T17:03:25.614688] [INFO ] [MAIN ] AFB: Wiping existing primary data before full refresh. [2025-09-16T17:03:25.614788] [INFO ] [LOCAL ] AFB: Starting deletion of primary data. [2025-09-16T17:03:25.614559] [INFO ] [MAIN ] AESI: Loaded 36439 existing records. [2025-09-16T17:03:25.615419] [DEBUG] [LOCAL ] AES: Loaded 39635 existing records from primary storage. [2025-09-16T17:03:25.650720] [DEBUG] [LOCAL ] AFB: Deleted primary data file. [2025-09-16T17:03:25.651473] [INFO ] [MAIN ] AFB: Starting full primary refresh. [2025-09-16T17:03:25.620713] [INFO ] [MAIN ] AES: Loaded 39635 existing records. [2025-09-16T17:03:25.662775] [DEBUG] [LOCAL ] AER: Loaded 35563 existing records from primary storage. [2025-09-16T17:03:25.697865] [INFO ] [MAIN ] AER: Loaded 35563 existing records. [2025-09-16T17:03:26.285167] [WARN ] [MAIN ] AESI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:26.460910] [WARN ] [MAIN ] AER: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:26.461127] [INFO ] [MAIN ] AESI: Wiping existing primary data before full refresh. [2025-09-16T17:03:26.467631] [WARN ] [MAIN ] AES: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:26.473166] [INFO ] [LOCAL ] AESI: Starting deletion of primary data. [2025-09-16T17:03:26.494900] [DEBUG] [LOCAL ] AESI: Deleted primary data file. [2025-09-16T17:03:26.478473] [INFO ] [MAIN ] AER: Wiping existing primary data before full refresh. [2025-09-16T17:03:26.500378] [INFO ] [MAIN ] AESI: Starting full primary refresh. [2025-09-16T17:03:26.508771] [INFO ] [LOCAL ] AER: Starting deletion of primary data. [2025-09-16T17:03:26.508935] [INFO ] [MAIN ] AES: Wiping existing primary data before full refresh. [2025-09-16T17:03:26.539719] [DEBUG] [LOCAL ] AER: Deleted primary data file. [2025-09-16T17:03:26.550800] [INFO ] [MAIN ] AER: Starting full primary refresh. [2025-09-16T17:03:26.539875] [INFO ] [LOCAL ] AES: Starting deletion of primary data. [2025-09-16T17:03:26.668272] [DEBUG] [LOCAL ] AES: Deleted primary data file. [2025-09-16T17:03:26.748123] [INFO ] [REMOTE ] AFB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:26.768151] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-02. [2025-09-16T17:03:26.748203] [INFO ] [MAIN ] AES: Starting full primary refresh. [2025-09-16T17:03:27.627880] [INFO ] [REMOTE ] AER: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:27.638560] [INFO ] [REMOTE ] AESI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:27.655498] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-02. [2025-09-16T17:03:27.656120] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-02. [2025-09-16T17:03:27.719946] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-09-16T17:03:27.760365] [INFO ] [REMOTE ] AES: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:27.760844] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-03. [2025-09-16T17:03:27.761123] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-02. [2025-09-16T17:03:28.739725] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T17:03:28.743372] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-04. [2025-09-16T17:03:29.411020] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T17:03:29.420316] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-03. [2025-09-16T17:03:29.596379] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-09-16T17:03:29.606674] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-03. [2025-09-16T17:03:29.619353] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-09-16T17:03:29.625207] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-05. [2025-09-16T17:03:29.835328] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-02 (2.18 seconds). [2025-09-16T17:03:29.844938] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-03. [2025-09-16T17:03:30.149839] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-09-16T17:03:30.152622] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-06. [2025-09-16T17:03:31.052671] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-09-16T17:03:31.055636] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-07. [2025-09-16T17:03:31.204498] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T17:03:31.303442] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T17:03:31.391118] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T17:03:31.574866] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-04. [2025-09-16T17:03:31.706481] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-09-16T17:03:32.374961] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-04. [2025-09-16T17:03:33.174912] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-08. [2025-09-16T17:03:33.249909] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-09-16T17:03:33.812710] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-09-16T17:03:33.870075] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-09-16T17:03:33.974873] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-09. [2025-09-16T17:03:34.579194] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-09-16T17:03:34.580984] [INFO ] [REMOTE ] AFB: Full fetch complete (12387 records). [2025-09-16T17:03:34.581063] [DEBUG] [VALIDATOR ] AFB: Validating primary data. [2025-09-16T17:03:34.781488] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-04. [2025-09-16T17:03:34.959283] [DEBUG] [VALIDATOR ] AFB: All expected trading days are present. [2025-09-16T17:03:34.959374] [DEBUG] [VALIDATOR ] AFB: Primary data validation passed. [2025-09-16T17:03:34.959599] [INFO ] [MAIN ] AFB: Fetched primary data is valid. [2025-09-16T17:03:34.959661] [DEBUG] [LOCAL ] AFB: Saving primary data. [2025-09-16T17:03:34.960331] [DEBUG] [LOCAL ] AFB: Trimmed 12387 records to 7923 (keeping 150 days of data) [2025-09-16T17:03:35.009325] [DEBUG] [LOCAL ] AFB: Primary data saved (7923 records). [2025-09-16T17:03:35.009472] [INFO ] [MAIN ] AFB: Full history saved. [2025-09-16T17:03:35.009570] [INFO ] [MAIN ] Update Progress: 41/2369 symbols completed. AFB took 10.4s. Estimated remaining: 3h 20m 33s. Rate: 11.6 symbols/min [2025-09-16T17:03:35.182844] [INFO ] [MAIN ] AFG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:35.208000] [DEBUG] [LOCAL ] AFG: Loaded 23462 existing records from primary storage. [2025-09-16T17:03:35.208147] [INFO ] [MAIN ] AFG: Loaded 23462 existing records. [2025-09-16T17:03:35.391953] [WARN ] [MAIN ] AFG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:35.392221] [INFO ] [MAIN ] AFG: Wiping existing primary data before full refresh. [2025-09-16T17:03:35.392284] [INFO ] [LOCAL ] AFG: Starting deletion of primary data. [2025-09-16T17:03:35.392900] [DEBUG] [LOCAL ] AFG: Deleted primary data file. [2025-09-16T17:03:35.392949] [INFO ] [MAIN ] AFG: Starting full primary refresh. [2025-09-16T17:03:35.580120] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-05. [2025-09-16T17:03:35.757859] [INFO ] [REMOTE ] AFG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:36.374905] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-05. [2025-09-16T17:03:36.979335] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-04 (2.20 seconds). [2025-09-16T17:03:37.174882] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-05. [2025-09-16T17:03:37.257847] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T17:03:37.928163] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-09-16T17:03:37.974858] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-06. [2025-09-16T17:03:38.774910] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-06. [2025-09-16T17:03:38.909115] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T17:03:39.452633] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T17:03:39.574923] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-07. [2025-09-16T17:03:40.374927] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-02. [2025-09-16T17:03:40.895832] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-06 (2.12 seconds). [2025-09-16T17:03:41.005150] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-09-16T17:03:41.174916] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-07. [2025-09-16T17:03:41.539779] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-09-16T17:03:41.974900] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-03. [2025-09-16T17:03:42.465720] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-09-16T17:03:42.774921] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-06. [2025-09-16T17:03:43.250368] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-09-16T17:03:43.574921] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-08. [2025-09-16T17:03:44.375067] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-04. [2025-09-16T17:03:44.716815] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-09-16T17:03:45.173802] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-09-16T17:03:45.183045] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-07. [2025-09-16T17:03:45.747938] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T17:03:45.974927] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-05. [2025-09-16T17:03:46.774900] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-08. [2025-09-16T17:03:47.038849] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-09-16T17:03:47.161014] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-09-16T17:03:47.574944] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-06. [2025-09-16T17:03:48.374946] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-08. [2025-09-16T17:03:48.483023] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-09-16T17:03:49.012136] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T17:03:49.174935] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-09. [2025-09-16T17:03:49.807077] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-09-16T17:03:49.974904] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-09. [2025-09-16T17:03:50.362096] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-09-16T17:03:50.367737] [INFO ] [REMOTE ] AESI: Full fetch complete (53514 records). [2025-09-16T17:03:50.367833] [DEBUG] [VALIDATOR ] AESI: Validating primary data. [2025-09-16T17:03:50.368516] [FATAL] [VALIDATOR ] AESI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:03:50.368596] [FATAL] [MAIN ] AESI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:03:50.368631] [WARN ] [MAIN ] AESI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:03:50.368696] [INFO ] [LOCAL ] AESI: Starting deletion of primary data. [2025-09-16T17:03:50.368755] [DEBUG] [LOCAL ] AESI: No primary data file found to delete. [2025-09-16T17:03:50.368790] [INFO ] [LOCAL ] AESI: Starting deletion of derivative data. [2025-09-16T17:03:50.369159] [DEBUG] [LOCAL ] AESI: Deleted derivative data file. [2025-09-16T17:03:50.369248] [INFO ] [MAIN ] Update Progress: 42/2369 symbols completed. AESI took 25.8s. Estimated remaining: 3h 29m 53s. Rate: 11.1 symbols/min [2025-09-16T17:03:50.542715] [INFO ] [MAIN ] AFGB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:50.545919] [DEBUG] [LOCAL ] AFGB: Loaded 1640 existing records from primary storage. [2025-09-16T17:03:50.545989] [INFO ] [MAIN ] AFGB: Loaded 1640 existing records. [2025-09-16T17:03:50.720390] [WARN ] [MAIN ] AFGB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:50.720682] [INFO ] [MAIN ] AFGB: Wiping existing primary data before full refresh. [2025-09-16T17:03:50.720730] [INFO ] [LOCAL ] AFGB: Starting deletion of primary data. [2025-09-16T17:03:50.720926] [DEBUG] [LOCAL ] AFGB: Deleted primary data file. [2025-09-16T17:03:50.720976] [INFO ] [MAIN ] AFGB: Starting full primary refresh. [2025-09-16T17:03:50.784505] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-07. [2025-09-16T17:03:51.089042] [INFO ] [REMOTE ] AFGB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:51.517033] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T17:03:51.523032] [INFO ] [REMOTE ] AER: Full fetch complete (53195 records). [2025-09-16T17:03:51.523160] [DEBUG] [VALIDATOR ] AER: Validating primary data. [2025-09-16T17:03:51.586379] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-02. [2025-09-16T17:03:51.902824] [DEBUG] [VALIDATOR ] AER: All expected trading days are present. [2025-09-16T17:03:51.902954] [DEBUG] [VALIDATOR ] AER: Primary data validation passed. [2025-09-16T17:03:51.903491] [INFO ] [MAIN ] AER: Fetched primary data is valid. [2025-09-16T17:03:51.903534] [DEBUG] [LOCAL ] AER: Saving primary data. [2025-09-16T17:03:51.904829] [DEBUG] [LOCAL ] AER: Trimmed 53195 records to 35913 (keeping 150 days of data) [2025-09-16T17:03:52.140110] [DEBUG] [LOCAL ] AER: Primary data saved (35913 records). [2025-09-16T17:03:52.140276] [INFO ] [MAIN ] AER: Full history saved. [2025-09-16T17:03:52.140374] [INFO ] [MAIN ] Update Progress: 43/2369 symbols completed. AER took 27.5s. Estimated remaining: 3h 26m 30s. Rate: 11.3 symbols/min [2025-09-16T17:03:52.271983] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T17:03:52.328193] [INFO ] [MAIN ] AFGC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:52.332221] [DEBUG] [LOCAL ] AFGC: Loaded 2025 existing records from primary storage. [2025-09-16T17:03:52.332299] [INFO ] [MAIN ] AFGC: Loaded 2025 existing records. [2025-09-16T17:03:52.385351] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-03. [2025-09-16T17:03:52.537871] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-09-16T17:03:52.541504] [WARN ] [MAIN ] AFGC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:52.549965] [INFO ] [MAIN ] AFGC: Wiping existing primary data before full refresh. [2025-09-16T17:03:52.550408] [INFO ] [LOCAL ] AFGC: Starting deletion of primary data. [2025-09-16T17:03:52.551277] [DEBUG] [LOCAL ] AFGC: Deleted primary data file. [2025-09-16T17:03:52.551343] [INFO ] [MAIN ] AFGC: Starting full primary refresh. [2025-09-16T17:03:52.904203] [INFO ] [REMOTE ] AFGC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:52.973757] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-09-16T17:03:53.174937] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-08. [2025-09-16T17:03:53.974891] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-09. [2025-09-16T17:03:54.538532] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-09-16T17:03:54.774905] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-04. [2025-09-16T17:03:55.209558] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-09-16T17:03:55.574897] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-09. [2025-09-16T17:03:56.093615] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-09 (2.12 seconds). [2025-09-16T17:03:56.100137] [INFO ] [REMOTE ] AES: Full fetch complete (58353 records). [2025-09-16T17:03:56.100244] [DEBUG] [VALIDATOR ] AES: Validating primary data. [2025-09-16T17:03:56.380117] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-02. [2025-09-16T17:03:56.529874] [DEBUG] [VALIDATOR ] AES: All expected trading days are present. [2025-09-16T17:03:56.529964] [DEBUG] [VALIDATOR ] AES: Primary data validation passed. [2025-09-16T17:03:56.530556] [INFO ] [MAIN ] AES: Fetched primary data is valid. [2025-09-16T17:03:56.530600] [DEBUG] [LOCAL ] AES: Saving primary data. [2025-09-16T17:03:56.532662] [DEBUG] [LOCAL ] AES: Trimmed 58353 records to 40025 (keeping 150 days of data) [2025-09-16T17:03:56.739438] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-09-16T17:03:56.772164] [INFO ] [REMOTE ] AFG: Full fetch complete (35810 records). [2025-09-16T17:03:56.773312] [DEBUG] [VALIDATOR ] AFG: Validating primary data. [2025-09-16T17:03:56.980574] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-09-16T17:03:57.186924] [DEBUG] [VALIDATOR ] AFG: All expected trading days are present. [2025-09-16T17:03:57.184074] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-03. [2025-09-16T17:03:57.187017] [DEBUG] [VALIDATOR ] AFG: Primary data validation passed. [2025-09-16T17:03:57.188414] [INFO ] [MAIN ] AFG: Fetched primary data is valid. [2025-09-16T17:03:57.189100] [DEBUG] [LOCAL ] AFG: Saving primary data. [2025-09-16T17:03:57.196756] [DEBUG] [LOCAL ] AFG: Trimmed 35810 records to 23723 (keeping 150 days of data) [2025-09-16T17:03:57.301721] [DEBUG] [LOCAL ] AES: Primary data saved (40025 records). [2025-09-16T17:03:57.302885] [INFO ] [MAIN ] AES: Full history saved. [2025-09-16T17:03:57.303242] [INFO ] [MAIN ] Update Progress: 44/2369 symbols completed. AES took 32.7s. Estimated remaining: 3h 26m 16s. Rate: 11.3 symbols/min [2025-09-16T17:03:57.494946] [INFO ] [MAIN ] AFGD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:57.496565] [DEBUG] [LOCAL ] AFGD: No existing data file found in primary storage. [2025-09-16T17:03:57.497106] [WARN ] [MAIN ] AFGD: No existing data found, performing full refresh instead. [2025-09-16T17:03:57.498841] [INFO ] [MAIN ] AFGD: Wiping existing primary data before full refresh. [2025-09-16T17:03:57.499766] [INFO ] [LOCAL ] AFGD: Starting deletion of primary data. [2025-09-16T17:03:57.500762] [DEBUG] [LOCAL ] AFGD: No primary data file found to delete. [2025-09-16T17:03:57.501419] [INFO ] [MAIN ] AFGD: Starting full primary refresh. [2025-09-16T17:03:57.663255] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-09-16T17:03:57.833512] [DEBUG] [LOCAL ] AFG: Primary data saved (23723 records). [2025-09-16T17:03:57.843911] [INFO ] [MAIN ] AFG: Full history saved. [2025-09-16T17:03:57.849137] [INFO ] [MAIN ] Update Progress: 45/2369 symbols completed. AFG took 22.8s. Estimated remaining: 3h 22m 4s. Rate: 11.5 symbols/min [2025-09-16T17:03:57.983545] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-04. [2025-09-16T17:03:57.972844] [INFO ] [REMOTE ] AFGD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:58.082380] [INFO ] [MAIN ] AFGE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:03:58.086044] [DEBUG] [LOCAL ] AFGE: Loaded 1879 existing records from primary storage. [2025-09-16T17:03:58.086130] [INFO ] [MAIN ] AFGE: Loaded 1879 existing records. [2025-09-16T17:03:58.259976] [WARN ] [MAIN ] AFGE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:03:58.260234] [INFO ] [MAIN ] AFGE: Wiping existing primary data before full refresh. [2025-09-16T17:03:58.260281] [INFO ] [LOCAL ] AFGE: Starting deletion of primary data. [2025-09-16T17:03:58.260480] [DEBUG] [LOCAL ] AFGE: Deleted primary data file. [2025-09-16T17:03:58.260523] [INFO ] [MAIN ] AFGE: Starting full primary refresh. [2025-09-16T17:03:58.540725] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-09-16T17:03:58.617797] [INFO ] [REMOTE ] AFGE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:03:58.774909] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-05. [2025-09-16T17:03:59.249006] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-09-16T17:03:59.574920] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-02. [2025-09-16T17:04:00.080861] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-09-16T17:04:00.374920] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-06. [2025-09-16T17:04:01.072492] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-09-16T17:04:01.174913] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-07. [2025-09-16T17:04:01.974982] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-03. [2025-09-16T17:04:02.059757] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-09-16T17:04:02.469246] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-09-16T17:04:02.774962] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-08. [2025-09-16T17:04:03.218386] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-09-16T17:04:03.574937] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-05. [2025-09-16T17:04:04.131542] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-09-16T17:04:04.374951] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-04. [2025-09-16T17:04:04.787671] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-09-16T17:04:05.174918] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-02. [2025-09-16T17:04:05.702381] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-09-16T17:04:05.974926] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-06. [2025-09-16T17:04:06.443606] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-09-16T17:04:06.774913] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-03. [2025-09-16T17:04:07.233934] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-09-16T17:04:07.574914] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-07. [2025-09-16T17:04:07.971026] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-09-16T17:04:08.374926] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-09. [2025-09-16T17:04:09.040376] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-09-16T17:04:09.041457] [INFO ] [REMOTE ] AFGC: Full fetch complete (2690 records). [2025-09-16T17:04:09.041530] [DEBUG] [VALIDATOR ] AFGC: Validating primary data. [2025-09-16T17:04:09.191466] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-08. [2025-09-16T17:04:09.419211] [DEBUG] [VALIDATOR ] AFGC: All expected trading days are present. [2025-09-16T17:04:09.419342] [DEBUG] [VALIDATOR ] AFGC: Primary data validation passed. [2025-09-16T17:04:09.419536] [INFO ] [MAIN ] AFGC: Fetched primary data is valid. [2025-09-16T17:04:09.419583] [DEBUG] [LOCAL ] AFGC: Saving primary data. [2025-09-16T17:04:09.420566] [DEBUG] [LOCAL ] AFGC: Trimmed 2690 records to 2047 (keeping 150 days of data) [2025-09-16T17:04:09.434019] [DEBUG] [LOCAL ] AFGC: Primary data saved (2047 records). [2025-09-16T17:04:09.434079] [INFO ] [MAIN ] AFGC: Full history saved. [2025-09-16T17:04:09.434174] [INFO ] [MAIN ] Update Progress: 46/2369 symbols completed. AFGC took 17.3s. Estimated remaining: 3h 27m 21s. Rate: 11.2 symbols/min [2025-09-16T17:04:09.602873] [INFO ] [MAIN ] AFL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:04:09.643803] [DEBUG] [LOCAL ] AFL: Loaded 39216 existing records from primary storage. [2025-09-16T17:04:09.643950] [INFO ] [MAIN ] AFL: Loaded 39216 existing records. [2025-09-16T17:04:09.825622] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-09-16T17:04:09.828823] [WARN ] [MAIN ] AFL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:04:09.830354] [INFO ] [MAIN ] AFL: Wiping existing primary data before full refresh. [2025-09-16T17:04:09.830430] [INFO ] [LOCAL ] AFL: Starting deletion of primary data. [2025-09-16T17:04:09.831414] [DEBUG] [LOCAL ] AFL: Deleted primary data file. [2025-09-16T17:04:09.831466] [INFO ] [MAIN ] AFL: Starting full primary refresh. [2025-09-16T17:04:09.980858] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-05. [2025-09-16T17:04:10.182388] [INFO ] [REMOTE ] AFL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:04:10.774889] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-04. [2025-09-16T17:04:10.825176] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T17:04:11.231576] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-09-16T17:04:11.574912] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-05. [2025-09-16T17:04:12.035320] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-09-16T17:04:12.374939] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-06. [2025-09-16T17:04:12.806978] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-09-16T17:04:13.174955] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-06. [2025-09-16T17:04:13.627753] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-09-16T17:04:13.974900] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-07. [2025-09-16T17:04:14.417078] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-09-16T17:04:14.774928] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-08. [2025-09-16T17:04:15.319205] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-09-16T17:04:15.574909] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-09. [2025-09-16T17:04:16.167949] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-09-16T17:04:16.168826] [INFO ] [REMOTE ] AFGB: Full fetch complete (2640 records). [2025-09-16T17:04:16.168898] [DEBUG] [VALIDATOR ] AFGB: Validating primary data. [2025-09-16T17:04:16.169281] [FATAL] [VALIDATOR ] AFGB: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:04:16.169375] [FATAL] [MAIN ] AFGB: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:04:16.169439] [WARN ] [MAIN ] AFGB: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:04:16.169524] [INFO ] [LOCAL ] AFGB: Starting deletion of primary data. [2025-09-16T17:04:16.169600] [DEBUG] [LOCAL ] AFGB: No primary data file found to delete. [2025-09-16T17:04:16.169687] [INFO ] [LOCAL ] AFGB: Starting deletion of derivative data. [2025-09-16T17:04:16.170007] [DEBUG] [LOCAL ] AFGB: Deleted derivative data file. [2025-09-16T17:04:16.170119] [INFO ] [MAIN ] Update Progress: 47/2369 symbols completed. AFGB took 25.8s. Estimated remaining: 3h 28m 23s. Rate: 11.1 symbols/min [2025-09-16T17:04:16.339174] [INFO ] [MAIN ] AG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:04:16.379999] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-02. [2025-09-16T17:04:16.382759] [DEBUG] [LOCAL ] AG: Loaded 39606 existing records from primary storage. [2025-09-16T17:04:16.382925] [INFO ] [MAIN ] AG: Loaded 39606 existing records. [2025-09-16T17:04:16.569627] [WARN ] [MAIN ] AG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:04:16.570059] [INFO ] [MAIN ] AG: Wiping existing primary data before full refresh. [2025-09-16T17:04:16.570108] [INFO ] [LOCAL ] AG: Starting deletion of primary data. [2025-09-16T17:04:16.571047] [DEBUG] [LOCAL ] AG: Deleted primary data file. [2025-09-16T17:04:16.571107] [INFO ] [MAIN ] AG: Starting full primary refresh. [2025-09-16T17:04:16.937249] [INFO ] [REMOTE ] AG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:04:17.174903] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-02. [2025-09-16T17:04:17.974909] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-09. [2025-09-16T17:04:18.095662] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-09-16T17:04:18.514699] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-09-16T17:04:18.515480] [INFO ] [REMOTE ] AFGE: Full fetch complete (2644 records). [2025-09-16T17:04:18.515547] [DEBUG] [VALIDATOR ] AFGE: Validating primary data. [2025-09-16T17:04:18.515976] [FATAL] [VALIDATOR ] AFGE: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:04:18.516072] [FATAL] [MAIN ] AFGE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:04:18.516123] [WARN ] [MAIN ] AFGE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:04:18.516172] [INFO ] [LOCAL ] AFGE: Starting deletion of primary data. [2025-09-16T17:04:18.516230] [DEBUG] [LOCAL ] AFGE: No primary data file found to delete. [2025-09-16T17:04:18.516273] [INFO ] [LOCAL ] AFGE: Starting deletion of derivative data. [2025-09-16T17:04:18.516534] [DEBUG] [LOCAL ] AFGE: Deleted derivative data file. [2025-09-16T17:04:18.516612] [INFO ] [MAIN ] Update Progress: 48/2369 symbols completed. AFGE took 20.7s. Estimated remaining: 3h 25m 51s. Rate: 11.3 symbols/min [2025-09-16T17:04:18.692335] [INFO ] [MAIN ] AGCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:04:18.727595] [DEBUG] [LOCAL ] AGCO: Loaded 31094 existing records from primary storage. [2025-09-16T17:04:18.727757] [INFO ] [MAIN ] AGCO: Loaded 31094 existing records. [2025-09-16T17:04:18.782562] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-03. [2025-09-16T17:04:18.909123] [WARN ] [MAIN ] AGCO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:04:18.909361] [INFO ] [MAIN ] AGCO: Wiping existing primary data before full refresh. [2025-09-16T17:04:18.909408] [INFO ] [LOCAL ] AGCO: Starting deletion of primary data. [2025-09-16T17:04:18.910167] [DEBUG] [LOCAL ] AGCO: Deleted primary data file. [2025-09-16T17:04:18.910229] [INFO ] [MAIN ] AGCO: Starting full primary refresh. [2025-09-16T17:04:19.284135] [INFO ] [REMOTE ] AGCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:04:19.312114] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-02 (2.14 seconds). [2025-09-16T17:04:19.574913] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-03. [2025-09-16T17:04:20.246989] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T17:04:20.374897] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-04. [2025-09-16T17:04:21.174963] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-02. [2025-09-16T17:04:21.430852] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-09-16T17:04:21.745995] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T17:04:21.974899] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-04. [2025-09-16T17:04:22.281913] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-09-16T17:04:22.774913] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-03. [2025-09-16T17:04:23.574880] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-07. [2025-09-16T17:04:23.835802] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-09-16T17:04:23.974567] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-09-16T17:04:24.268574] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-09-16T17:04:24.374896] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-05. [2025-09-16T17:04:25.174937] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-04. [2025-09-16T17:04:25.974919] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-08. [2025-09-16T17:04:26.327374] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-09-16T17:04:26.340696] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-09-16T17:04:26.774864] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-06. [2025-09-16T17:04:26.809968] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T17:04:27.574935] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-09. [2025-09-16T17:04:28.126300] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-09-16T17:04:28.127247] [INFO ] [REMOTE ] AFGD: Full fetch complete (2065 records). [2025-09-16T17:04:28.127315] [DEBUG] [VALIDATOR ] AFGD: Validating primary data. [2025-09-16T17:04:28.385076] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-05. [2025-09-16T17:04:28.505524] [DEBUG] [VALIDATOR ] AFGD: All expected trading days are present. [2025-09-16T17:04:28.505760] [DEBUG] [VALIDATOR ] AFGD: Primary data validation passed. [2025-09-16T17:04:28.505935] [INFO ] [MAIN ] AFGD: Fetched primary data is valid. [2025-09-16T17:04:28.506010] [DEBUG] [LOCAL ] AFGD: Saving primary data. [2025-09-16T17:04:28.506542] [DEBUG] [LOCAL ] AFGD: Trimmed 2065 records to 1344 (keeping 150 days of data) [2025-09-16T17:04:28.515797] [DEBUG] [LOCAL ] AFGD: Primary data saved (1344 records). [2025-09-16T17:04:28.516053] [INFO ] [MAIN ] AFGD: Full history saved. [2025-09-16T17:04:28.516156] [INFO ] [MAIN ] Update Progress: 49/2369 symbols completed. AFGD took 31.2s. Estimated remaining: 3h 29m 27s. Rate: 11.1 symbols/min [2025-09-16T17:04:28.636421] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-09-16T17:04:28.730229] [INFO ] [MAIN ] AGD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:04:28.745003] [DEBUG] [LOCAL ] AGD: Loaded 13602 existing records from primary storage. [2025-09-16T17:04:28.745115] [INFO ] [MAIN ] AGD: Loaded 13602 existing records. [2025-09-16T17:04:28.920962] [WARN ] [MAIN ] AGD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:04:28.921237] [INFO ] [MAIN ] AGD: Wiping existing primary data before full refresh. [2025-09-16T17:04:28.921284] [INFO ] [LOCAL ] AGD: Starting deletion of primary data. [2025-09-16T17:04:28.921699] [DEBUG] [LOCAL ] AGD: Deleted primary data file. [2025-09-16T17:04:28.921748] [INFO ] [MAIN ] AGD: Starting full primary refresh. [2025-09-16T17:04:29.180088] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-07. [2025-09-16T17:04:29.279437] [INFO ] [REMOTE ] AGD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:04:29.966442] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T17:04:29.976936] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-05. [2025-09-16T17:04:30.774936] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-06. [2025-09-16T17:04:31.072540] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-09-16T17:04:31.473175] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-07 (2.29 seconds). [2025-09-16T17:04:31.574931] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-08. [2025-09-16T17:04:32.279910] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T17:04:32.374894] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-07. [2025-09-16T17:04:33.174956] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-02. [2025-09-16T17:04:33.633569] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-09-16T17:04:33.974956] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-09. [2025-09-16T17:04:34.073962] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T17:04:34.165149] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-09-16T17:04:34.774902] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-08. [2025-09-16T17:04:35.574892] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-06. [2025-09-16T17:04:35.943931] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-09-16T17:04:35.950319] [INFO ] [REMOTE ] AG: Full fetch complete (58316 records). [2025-09-16T17:04:35.950433] [DEBUG] [VALIDATOR ] AG: Validating primary data. [2025-09-16T17:04:36.337752] [DEBUG] [VALIDATOR ] AG: All expected trading days are present. [2025-09-16T17:04:36.337882] [DEBUG] [VALIDATOR ] AG: Primary data validation passed. [2025-09-16T17:04:36.338534] [INFO ] [MAIN ] AG: Fetched primary data is valid. [2025-09-16T17:04:36.338590] [DEBUG] [LOCAL ] AG: Saving primary data. [2025-09-16T17:04:36.339895] [DEBUG] [LOCAL ] AG: Trimmed 58316 records to 39996 (keeping 150 days of data) [2025-09-16T17:04:36.403516] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-03. [2025-09-16T17:04:36.459855] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T17:04:36.627766] [DEBUG] [LOCAL ] AG: Primary data saved (39996 records). [2025-09-16T17:04:36.627954] [INFO ] [MAIN ] AG: Full history saved. [2025-09-16T17:04:36.628070] [INFO ] [MAIN ] Update Progress: 50/2369 symbols completed. AG took 20.5s. Estimated remaining: 3h 31m 27s. Rate: 11.0 symbols/min [2025-09-16T17:04:36.799988] [INFO ] [MAIN ] AGI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:04:36.836696] [DEBUG] [LOCAL ] AGI: Loaded 39404 existing records from primary storage. [2025-09-16T17:04:36.836864] [INFO ] [MAIN ] AGI: Loaded 39404 existing records. [2025-09-16T17:04:37.019798] [WARN ] [MAIN ] AGI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:04:37.020120] [INFO ] [MAIN ] AGI: Wiping existing primary data before full refresh. [2025-09-16T17:04:37.020175] [INFO ] [LOCAL ] AGI: Starting deletion of primary data. [2025-09-16T17:04:37.021258] [DEBUG] [LOCAL ] AGI: Deleted primary data file. [2025-09-16T17:04:37.021314] [INFO ] [MAIN ] AGI: Starting full primary refresh. [2025-09-16T17:04:37.196777] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-09. [2025-09-16T17:04:37.271560] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T17:04:37.434792] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-09-16T17:04:37.434376] [INFO ] [REMOTE ] AGI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:04:37.974956] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-07. [2025-09-16T17:04:38.569538] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T17:04:38.576240] [INFO ] [REMOTE ] AFL: Full fetch complete (57753 records). [2025-09-16T17:04:38.576369] [DEBUG] [VALIDATOR ] AFL: Validating primary data. [2025-09-16T17:04:38.780557] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-04. [2025-09-16T17:04:38.962731] [DEBUG] [VALIDATOR ] AFL: All expected trading days are present. [2025-09-16T17:04:38.962821] [DEBUG] [VALIDATOR ] AFL: Primary data validation passed. [2025-09-16T17:04:38.963381] [INFO ] [MAIN ] AFL: Fetched primary data is valid. [2025-09-16T17:04:38.963442] [DEBUG] [LOCAL ] AFL: Saving primary data. [2025-09-16T17:04:38.964750] [DEBUG] [LOCAL ] AFL: Trimmed 57753 records to 39605 (keeping 150 days of data) [2025-09-16T17:04:39.222017] [DEBUG] [LOCAL ] AFL: Primary data saved (39605 records). [2025-09-16T17:04:39.222179] [INFO ] [MAIN ] AFL: Full history saved. [2025-09-16T17:04:39.224105] [INFO ] [MAIN ] Update Progress: 51/2369 symbols completed. AFL took 29.8s. Estimated remaining: 3h 29m 11s. Rate: 11.1 symbols/min [2025-09-16T17:04:39.398188] [INFO ] [MAIN ] AGL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:04:39.427973] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T17:04:39.473446] [DEBUG] [LOCAL ] AGL: Loaded 38685 existing records from primary storage. [2025-09-16T17:04:39.473589] [INFO ] [MAIN ] AGL: Loaded 38685 existing records. [2025-09-16T17:04:39.575237] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-02. [2025-09-16T17:04:39.652136] [WARN ] [MAIN ] AGL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:04:39.652402] [INFO ] [MAIN ] AGL: Wiping existing primary data before full refresh. [2025-09-16T17:04:39.652453] [INFO ] [LOCAL ] AGL: Starting deletion of primary data. [2025-09-16T17:04:39.653278] [DEBUG] [LOCAL ] AGL: Deleted primary data file. [2025-09-16T17:04:39.653341] [INFO ] [MAIN ] AGL: Starting full primary refresh. [2025-09-16T17:04:40.013723] [INFO ] [REMOTE ] AGL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:04:40.023668] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-09-16T17:04:40.374892] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-05. [2025-09-16T17:04:41.174879] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-08. [2025-09-16T17:04:41.377867] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T17:04:41.539068] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-09-16T17:04:41.974914] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-03. [2025-09-16T17:04:42.216526] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-09-16T17:04:42.774938] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-02. [2025-09-16T17:04:43.574878] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-06. [2025-09-16T17:04:43.757059] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T17:04:44.011621] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-09-16T17:04:44.374915] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-09. [2025-09-16T17:04:44.567265] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-09-16T17:04:45.174930] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-04. [2025-09-16T17:04:45.792994] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T17:04:45.798070] [INFO ] [REMOTE ] AGCO: Full fetch complete (46548 records). [2025-09-16T17:04:45.798189] [DEBUG] [VALIDATOR ] AGCO: Validating primary data. [2025-09-16T17:04:45.979067] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-03. [2025-09-16T17:04:46.183277] [DEBUG] [VALIDATOR ] AGCO: All expected trading days are present. [2025-09-16T17:04:46.183364] [DEBUG] [VALIDATOR ] AGCO: Primary data validation passed. [2025-09-16T17:04:46.183935] [INFO ] [MAIN ] AGCO: Fetched primary data is valid. [2025-09-16T17:04:46.183989] [DEBUG] [LOCAL ] AGCO: Saving primary data. [2025-09-16T17:04:46.185031] [DEBUG] [LOCAL ] AGCO: Trimmed 46548 records to 31416 (keeping 150 days of data) [2025-09-16T17:04:46.395563] [DEBUG] [LOCAL ] AGCO: Primary data saved (31416 records). [2025-09-16T17:04:46.395785] [INFO ] [MAIN ] AGCO: Full history saved. [2025-09-16T17:04:46.397810] [INFO ] [MAIN ] Update Progress: 52/2369 symbols completed. AGCO took 27.9s. Estimated remaining: 3h 30m 24s. Rate: 11.0 symbols/min [2025-09-16T17:04:46.575387] [INFO ] [MAIN ] AGM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:04:46.587423] [DEBUG] [LOCAL ] AGM: Loaded 7961 existing records from primary storage. [2025-09-16T17:04:46.587604] [INFO ] [MAIN ] AGM: Loaded 7961 existing records. [2025-09-16T17:04:46.768385] [WARN ] [MAIN ] AGM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:04:46.768777] [INFO ] [MAIN ] AGM: Wiping existing primary data before full refresh. [2025-09-16T17:04:46.768938] [INFO ] [LOCAL ] AGM: Starting deletion of primary data. [2025-09-16T17:04:46.769407] [DEBUG] [LOCAL ] AGM: Deleted primary data file. [2025-09-16T17:04:46.769506] [INFO ] [MAIN ] AGM: Starting full primary refresh. [2025-09-16T17:04:46.790119] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-07. [2025-09-16T17:04:47.091836] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-09-16T17:04:47.192610] [INFO ] [REMOTE ] AGM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:04:47.574913] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-02. [2025-09-16T17:04:47.747191] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T17:04:47.926816] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T17:04:48.374901] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-08. [2025-09-16T17:04:48.382228] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-09-16T17:04:49.174909] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-05. [2025-09-16T17:04:49.395633] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-09-16T17:04:49.974918] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-03. [2025-09-16T17:04:50.633028] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-09-16T17:04:50.774899] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-09. [2025-09-16T17:04:51.163126] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-09-16T17:04:51.505377] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T17:04:51.508347] [INFO ] [REMOTE ] AGD: Full fetch complete (18772 records). [2025-09-16T17:04:51.508426] [DEBUG] [VALIDATOR ] AGD: Validating primary data. [2025-09-16T17:04:51.580002] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-04. [2025-09-16T17:04:51.906536] [DEBUG] [VALIDATOR ] AGD: All expected trading days are present. [2025-09-16T17:04:51.906614] [DEBUG] [VALIDATOR ] AGD: Primary data validation passed. [2025-09-16T17:04:51.906889] [INFO ] [MAIN ] AGD: Fetched primary data is valid. [2025-09-16T17:04:51.906932] [DEBUG] [LOCAL ] AGD: Saving primary data. [2025-09-16T17:04:51.907546] [DEBUG] [LOCAL ] AGD: Trimmed 18772 records to 13784 (keeping 150 days of data) [2025-09-16T17:04:51.991552] [DEBUG] [LOCAL ] AGD: Primary data saved (13784 records). [2025-09-16T17:04:51.991724] [INFO ] [MAIN ] AGD: Full history saved. [2025-09-16T17:04:51.991818] [INFO ] [MAIN ] Update Progress: 53/2369 symbols completed. AGD took 23.4s. Estimated remaining: 3h 30m 25s. Rate: 11.0 symbols/min [2025-09-16T17:04:52.172717] [INFO ] [MAIN ] AGO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:04:52.194576] [DEBUG] [LOCAL ] AGO: Loaded 20242 existing records from primary storage. [2025-09-16T17:04:52.194799] [INFO ] [MAIN ] AGO: Loaded 20242 existing records. [2025-09-16T17:04:52.375121] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-04. [2025-09-16T17:04:52.375280] [WARN ] [MAIN ] AGO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:04:52.376323] [INFO ] [MAIN ] AGO: Wiping existing primary data before full refresh. [2025-09-16T17:04:52.376380] [INFO ] [LOCAL ] AGO: Starting deletion of primary data. [2025-09-16T17:04:52.376919] [DEBUG] [LOCAL ] AGO: Deleted primary data file. [2025-09-16T17:04:52.376970] [INFO ] [MAIN ] AGO: Starting full primary refresh. [2025-09-16T17:04:52.508276] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T17:04:52.748023] [INFO ] [REMOTE ] AGO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:04:53.174905] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-05. [2025-09-16T17:04:53.824588] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-09-16T17:04:53.974902] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-06. [2025-09-16T17:04:54.167988] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T17:04:54.774970] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-06. [2025-09-16T17:04:54.823785] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-09-16T17:04:55.574905] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-07. [2025-09-16T17:04:56.357298] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-09-16T17:04:56.374839] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-08. [2025-09-16T17:04:56.880186] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-09-16T17:04:56.952566] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-06 (2.18 seconds). [2025-09-16T17:04:57.174913] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-02. [2025-09-16T17:04:57.974918] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-07. [2025-09-16T17:04:58.068845] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-09-16T17:04:58.774911] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-03. [2025-09-16T17:04:59.574938] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-05. [2025-09-16T17:04:59.632552] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-09-16T17:04:59.837630] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-09-16T17:05:00.375055] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-08. [2025-09-16T17:05:01.174903] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-09. [2025-09-16T17:05:01.419216] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T17:05:01.974932] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-04. [2025-09-16T17:05:02.557333] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-09-16T17:05:02.574490] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:05:02.576184] [INFO ] [REMOTE ] AGM: Full fetch complete (10310 records). [2025-09-16T17:05:02.576251] [DEBUG] [VALIDATOR ] AGM: Validating primary data. [2025-09-16T17:05:02.780023] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-09. [2025-09-16T17:05:02.943019] [DEBUG] [VALIDATOR ] AGM: All expected trading days are present. [2025-09-16T17:05:02.943100] [DEBUG] [VALIDATOR ] AGM: Primary data validation passed. [2025-09-16T17:05:02.943298] [INFO ] [MAIN ] AGM: Fetched primary data is valid. [2025-09-16T17:05:02.943337] [DEBUG] [LOCAL ] AGM: Saving primary data. [2025-09-16T17:05:02.943822] [DEBUG] [LOCAL ] AGM: Trimmed 10310 records to 8023 (keeping 150 days of data) [2025-09-16T17:05:02.995675] [DEBUG] [LOCAL ] AGM: Primary data saved (8023 records). [2025-09-16T17:05:02.995808] [INFO ] [MAIN ] AGM: Full history saved. [2025-09-16T17:05:02.995890] [INFO ] [MAIN ] Update Progress: 54/2369 symbols completed. AGM took 16.6s. Estimated remaining: 3h 34m 17s. Rate: 10.8 symbols/min [2025-09-16T17:05:03.164246] [INFO ] [MAIN ] AGRO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:05:03.187136] [DEBUG] [LOCAL ] AGRO: Loaded 23968 existing records from primary storage. [2025-09-16T17:05:03.187272] [INFO ] [MAIN ] AGRO: Loaded 23968 existing records. [2025-09-16T17:05:03.365069] [WARN ] [MAIN ] AGRO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:05:03.365307] [INFO ] [MAIN ] AGRO: Wiping existing primary data before full refresh. [2025-09-16T17:05:03.365373] [INFO ] [LOCAL ] AGRO: Starting deletion of primary data. [2025-09-16T17:05:03.366008] [DEBUG] [LOCAL ] AGRO: Deleted primary data file. [2025-09-16T17:05:03.366060] [INFO ] [MAIN ] AGRO: Starting full primary refresh. [2025-09-16T17:05:03.598160] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-06. [2025-09-16T17:05:03.759876] [INFO ] [REMOTE ] AGRO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:05:03.780845] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T17:05:04.374875] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-05. [2025-09-16T17:05:04.818301] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-09-16T17:05:04.826089] [INFO ] [REMOTE ] AGI: Full fetch complete (58064 records). [2025-09-16T17:05:04.826231] [DEBUG] [VALIDATOR ] AGI: Validating primary data. [2025-09-16T17:05:05.183929] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-02. [2025-09-16T17:05:05.225367] [DEBUG] [VALIDATOR ] AGI: All expected trading days are present. [2025-09-16T17:05:05.225449] [DEBUG] [VALIDATOR ] AGI: Primary data validation passed. [2025-09-16T17:05:05.226168] [INFO ] [MAIN ] AGI: Fetched primary data is valid. [2025-09-16T17:05:05.241923] [DEBUG] [LOCAL ] AGI: Saving primary data. [2025-09-16T17:05:05.254384] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T17:05:05.265441] [DEBUG] [LOCAL ] AGI: Trimmed 58064 records to 39794 (keeping 150 days of data) [2025-09-16T17:05:05.520026] [DEBUG] [LOCAL ] AGI: Primary data saved (39794 records). [2025-09-16T17:05:05.520221] [INFO ] [MAIN ] AGI: Full history saved. [2025-09-16T17:05:05.520337] [INFO ] [MAIN ] Update Progress: 55/2369 symbols completed. AGI took 28.9s. Estimated remaining: 3h 32m 4s. Rate: 10.9 symbols/min [2025-09-16T17:05:05.692183] [INFO ] [MAIN ] AGX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:05:05.716340] [DEBUG] [LOCAL ] AGX: Loaded 24492 existing records from primary storage. [2025-09-16T17:05:05.716465] [INFO ] [MAIN ] AGX: Loaded 24492 existing records. [2025-09-16T17:05:05.841836] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T17:05:05.918478] [WARN ] [MAIN ] AGX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:05:05.918765] [INFO ] [MAIN ] AGX: Wiping existing primary data before full refresh. [2025-09-16T17:05:05.918822] [INFO ] [LOCAL ] AGX: Starting deletion of primary data. [2025-09-16T17:05:05.919463] [DEBUG] [LOCAL ] AGX: Deleted primary data file. [2025-09-16T17:05:05.919511] [INFO ] [MAIN ] AGX: Starting full primary refresh. [2025-09-16T17:05:05.982050] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-06. [2025-09-16T17:05:06.277045] [INFO ] [REMOTE ] AGX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:05:06.456196] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T17:05:06.774894] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-02. [2025-09-16T17:05:07.204559] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-09-16T17:05:07.574936] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-07. [2025-09-16T17:05:08.206587] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-09-16T17:05:08.374923] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-03. [2025-09-16T17:05:09.175020] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-03. [2025-09-16T17:05:09.425001] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T17:05:09.927112] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T17:05:09.974898] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-04. [2025-09-16T17:05:10.679001] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-09-16T17:05:10.774912] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-04. [2025-09-16T17:05:11.065217] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T17:05:11.574926] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-05. [2025-09-16T17:05:12.247254] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T17:05:12.374903] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-05. [2025-09-16T17:05:12.920566] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T17:05:13.174955] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-07. [2025-09-16T17:05:13.452630] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-09-16T17:05:13.974913] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-06. [2025-09-16T17:05:14.778595] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-06. [2025-09-16T17:05:14.780483] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-09-16T17:05:15.216490] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T17:05:15.574902] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-08. [2025-09-16T17:05:16.257918] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T17:05:16.374915] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-07. [2025-09-16T17:05:16.560873] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T17:05:17.174953] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-07. [2025-09-16T17:05:17.745998] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T17:05:17.974905] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-08. [2025-09-16T17:05:18.634182] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T17:05:18.774911] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-09. [2025-09-16T17:05:19.066476] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-09-16T17:05:19.574891] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-08. [2025-09-16T17:05:19.864903] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T17:05:19.868520] [INFO ] [REMOTE ] AGO: Full fetch complete (30365 records). [2025-09-16T17:05:19.868617] [DEBUG] [VALIDATOR ] AGO: Validating primary data. [2025-09-16T17:05:20.247325] [DEBUG] [VALIDATOR ] AGO: All expected trading days are present. [2025-09-16T17:05:20.247442] [DEBUG] [VALIDATOR ] AGO: Primary data validation passed. [2025-09-16T17:05:20.248019] [INFO ] [MAIN ] AGO: Fetched primary data is valid. [2025-09-16T17:05:20.248109] [DEBUG] [LOCAL ] AGO: Saving primary data. [2025-09-16T17:05:20.249374] [DEBUG] [LOCAL ] AGO: Trimmed 30365 records to 20430 (keeping 150 days of data) [2025-09-16T17:05:20.374875] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-09. [2025-09-16T17:05:20.380734] [DEBUG] [LOCAL ] AGO: Primary data saved (20430 records). [2025-09-16T17:05:20.380821] [INFO ] [MAIN ] AGO: Full history saved. [2025-09-16T17:05:20.380911] [INFO ] [MAIN ] Update Progress: 56/2369 symbols completed. AGO took 28.4s. Estimated remaining: 3h 38m 25s. Rate: 10.6 symbols/min [2025-09-16T17:05:20.559570] [INFO ] [MAIN ] AHH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:05:20.589816] [DEBUG] [LOCAL ] AHH: Loaded 29529 existing records from primary storage. [2025-09-16T17:05:20.589963] [INFO ] [MAIN ] AHH: Loaded 29529 existing records. [2025-09-16T17:05:20.773271] [WARN ] [MAIN ] AHH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:05:20.773557] [INFO ] [MAIN ] AHH: Wiping existing primary data before full refresh. [2025-09-16T17:05:20.773606] [INFO ] [LOCAL ] AHH: Starting deletion of primary data. [2025-09-16T17:05:20.774263] [DEBUG] [LOCAL ] AHH: Deleted primary data file. [2025-09-16T17:05:20.774325] [INFO ] [MAIN ] AHH: Starting full primary refresh. [2025-09-16T17:05:21.128696] [INFO ] [REMOTE ] AHH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:05:21.174928] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-02. [2025-09-16T17:05:21.485290] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-09-16T17:05:21.755714] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T17:05:21.760073] [INFO ] [REMOTE ] AGRO: Full fetch complete (37461 records). [2025-09-16T17:05:21.760228] [DEBUG] [VALIDATOR ] AGRO: Validating primary data. [2025-09-16T17:05:21.983977] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-09. [2025-09-16T17:05:22.161407] [DEBUG] [VALIDATOR ] AGRO: All expected trading days are present. [2025-09-16T17:05:22.161493] [DEBUG] [VALIDATOR ] AGRO: Primary data validation passed. [2025-09-16T17:05:22.162027] [INFO ] [MAIN ] AGRO: Fetched primary data is valid. [2025-09-16T17:05:22.162080] [DEBUG] [LOCAL ] AGRO: Saving primary data. [2025-09-16T17:05:22.163009] [DEBUG] [LOCAL ] AGRO: Trimmed 37461 records to 24149 (keeping 150 days of data) [2025-09-16T17:05:22.310748] [DEBUG] [LOCAL ] AGRO: Primary data saved (24149 records). [2025-09-16T17:05:22.310915] [INFO ] [MAIN ] AGRO: Full history saved. [2025-09-16T17:05:22.311046] [INFO ] [MAIN ] Update Progress: 57/2369 symbols completed. AGRO took 19.3s. Estimated remaining: 3h 35m 48s. Rate: 10.7 symbols/min [2025-09-16T17:05:22.486259] [INFO ] [MAIN ] AHL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:05:22.486440] [DEBUG] [LOCAL ] AHL: No existing data file found in primary storage. [2025-09-16T17:05:22.486480] [WARN ] [MAIN ] AHL: No existing data found, performing full refresh instead. [2025-09-16T17:05:22.486732] [INFO ] [MAIN ] AHL: Wiping existing primary data before full refresh. [2025-09-16T17:05:22.505325] [INFO ] [LOCAL ] AHL: Starting deletion of primary data. [2025-09-16T17:05:22.505674] [DEBUG] [LOCAL ] AHL: No primary data file found to delete. [2025-09-16T17:05:22.505931] [INFO ] [MAIN ] AHL: Starting full primary refresh. [2025-09-16T17:05:22.506585] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-09-16T17:05:22.782019] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-03. [2025-09-16T17:05:22.879053] [INFO ] [REMOTE ] AHL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:05:23.574953] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-02. [2025-09-16T17:05:23.722621] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-09-16T17:05:23.731309] [INFO ] [REMOTE ] AGL: Full fetch complete (56964 records). [2025-09-16T17:05:23.731456] [DEBUG] [VALIDATOR ] AGL: Validating primary data. [2025-09-16T17:05:23.814495] [DEBUG] [REMOTE ] AHL: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:05:23.836706] [WARN ] [REMOTE ] AHL: No data returned for month 2025-02 [2025-09-16T17:05:24.123923] [DEBUG] [VALIDATOR ] AGL: All expected trading days are present. [2025-09-16T17:05:24.124019] [DEBUG] [VALIDATOR ] AGL: Primary data validation passed. [2025-09-16T17:05:24.124590] [INFO ] [MAIN ] AGL: Fetched primary data is valid. [2025-09-16T17:05:24.124677] [DEBUG] [LOCAL ] AGL: Saving primary data. [2025-09-16T17:05:24.126154] [DEBUG] [LOCAL ] AGL: Trimmed 56964 records to 39074 (keeping 150 days of data) [2025-09-16T17:05:24.356725] [DEBUG] [LOCAL ] AGL: Primary data saved (39074 records). [2025-09-16T17:05:24.356985] [INFO ] [MAIN ] AGL: Full history saved. [2025-09-16T17:05:24.358211] [INFO ] [MAIN ] Update Progress: 58/2369 symbols completed. AGL took 45.1s. Estimated remaining: 3h 33m 21s. Rate: 10.8 symbols/min [2025-09-16T17:05:24.379942] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-08. [2025-09-16T17:05:24.555790] [INFO ] [MAIN ] AHR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:05:24.557343] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T17:05:24.614429] [DEBUG] [LOCAL ] AHR: Loaded 36372 existing records from primary storage. [2025-09-16T17:05:24.614590] [INFO ] [MAIN ] AHR: Loaded 36372 existing records. [2025-09-16T17:05:24.792864] [WARN ] [MAIN ] AHR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:05:24.793110] [INFO ] [MAIN ] AHR: Wiping existing primary data before full refresh. [2025-09-16T17:05:24.793159] [INFO ] [LOCAL ] AHR: Starting deletion of primary data. [2025-09-16T17:05:24.793991] [DEBUG] [LOCAL ] AHR: Deleted primary data file. [2025-09-16T17:05:24.794068] [INFO ] [MAIN ] AHR: Starting full primary refresh. [2025-09-16T17:05:25.177080] [INFO ] [REMOTE ] AHR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:05:25.177155] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-03. [2025-09-16T17:05:25.409697] [DEBUG] [REMOTE ] AHL: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:05:25.410619] [WARN ] [REMOTE ] AHL: No data returned for month 2025-03 [2025-09-16T17:05:25.475806] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-09-16T17:05:25.974920] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-02. [2025-09-16T17:05:26.774924] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-04. [2025-09-16T17:05:27.574929] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-09. [2025-09-16T17:05:27.647708] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T17:05:28.208843] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:05:28.374911] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-05. [2025-09-16T17:05:28.653716] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T17:05:28.658300] [INFO ] [REMOTE ] AGX: Full fetch complete (36379 records). [2025-09-16T17:05:28.658395] [DEBUG] [VALIDATOR ] AGX: Validating primary data. [2025-09-16T17:05:29.035305] [DEBUG] [VALIDATOR ] AGX: All expected trading days are present. [2025-09-16T17:05:29.035407] [DEBUG] [VALIDATOR ] AGX: Primary data validation passed. [2025-09-16T17:05:29.035853] [INFO ] [MAIN ] AGX: Fetched primary data is valid. [2025-09-16T17:05:29.035940] [DEBUG] [LOCAL ] AGX: Saving primary data. [2025-09-16T17:05:29.036929] [DEBUG] [LOCAL ] AGX: Trimmed 36379 records to 24652 (keeping 150 days of data) [2025-09-16T17:05:29.184547] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-03. [2025-09-16T17:05:29.202929] [DEBUG] [LOCAL ] AGX: Primary data saved (24652 records). [2025-09-16T17:05:29.203038] [INFO ] [MAIN ] AGX: Full history saved. [2025-09-16T17:05:29.203138] [INFO ] [MAIN ] Update Progress: 59/2369 symbols completed. AGX took 23.7s. Estimated remaining: 3h 32m 48s. Rate: 10.9 symbols/min [2025-09-16T17:05:29.374795] [INFO ] [MAIN ] AHT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:05:29.380869] [DEBUG] [LOCAL ] AHT: Loaded 4581 existing records from primary storage. [2025-09-16T17:05:29.380967] [INFO ] [MAIN ] AHT: Loaded 4581 existing records. [2025-09-16T17:05:29.555437] [WARN ] [MAIN ] AHT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:05:29.555677] [INFO ] [MAIN ] AHT: Wiping existing primary data before full refresh. [2025-09-16T17:05:29.555745] [INFO ] [LOCAL ] AHT: Starting deletion of primary data. [2025-09-16T17:05:29.556044] [DEBUG] [LOCAL ] AHT: Deleted primary data file. [2025-09-16T17:05:29.556091] [INFO ] [MAIN ] AHT: Starting full primary refresh. [2025-09-16T17:05:29.909868] [INFO ] [REMOTE ] AHT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:05:29.934543] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T17:05:29.974846] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-02. [2025-09-16T17:05:30.531412] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-09-16T17:05:30.774898] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-04. [2025-09-16T17:05:30.898273] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T17:05:30.963546] [DEBUG] [REMOTE ] AHL: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:05:30.963871] [WARN ] [REMOTE ] AHL: No data returned for month 2025-04 [2025-09-16T17:05:31.574903] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-03. [2025-09-16T17:05:32.374968] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-04. [2025-09-16T17:05:33.174906] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-06. [2025-09-16T17:05:33.974874] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-05. [2025-09-16T17:05:34.080790] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-09-16T17:05:34.698272] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T17:05:34.774853] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-07. [2025-09-16T17:05:35.050460] [INFO ] [REMOTE ] AHL: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-09-16T17:05:35.574926] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-05. [2025-09-16T17:05:35.971667] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T17:05:36.374904] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-06. [2025-09-16T17:05:36.846227] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T17:05:37.050534] [INFO ] [REMOTE ] AHL: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-09-16T17:05:37.174914] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-06. [2025-09-16T17:05:37.974952] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-08. [2025-09-16T17:05:38.774896] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-07. [2025-09-16T17:05:38.867502] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T17:05:39.292302] [INFO ] [REMOTE ] AHL: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-09-16T17:05:39.330261] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-09-16T17:05:39.574948] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-07. [2025-09-16T17:05:40.374947] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-08. [2025-09-16T17:05:41.174935] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-09. [2025-09-16T17:05:41.281094] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T17:05:41.307801] [INFO ] [REMOTE ] AHL: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T17:05:41.974912] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-09. [2025-09-16T17:05:42.085822] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-09-16T17:05:42.090064] [INFO ] [REMOTE ] AHH: Full fetch complete (45139 records). [2025-09-16T17:05:42.090149] [DEBUG] [VALIDATOR ] AHH: Validating primary data. [2025-09-16T17:05:42.471948] [DEBUG] [VALIDATOR ] AHH: All expected trading days are present. [2025-09-16T17:05:42.472051] [DEBUG] [VALIDATOR ] AHH: Primary data validation passed. [2025-09-16T17:05:42.472578] [INFO ] [MAIN ] AHH: Fetched primary data is valid. [2025-09-16T17:05:42.472625] [DEBUG] [LOCAL ] AHH: Saving primary data. [2025-09-16T17:05:42.473881] [DEBUG] [LOCAL ] AHH: Trimmed 45139 records to 29751 (keeping 150 days of data) [2025-09-16T17:05:42.648047] [DEBUG] [LOCAL ] AHH: Primary data saved (29751 records). [2025-09-16T17:05:42.648192] [INFO ] [MAIN ] AHH: Full history saved. [2025-09-16T17:05:42.648287] [INFO ] [MAIN ] Update Progress: 60/2369 symbols completed. AHH took 22.3s. Estimated remaining: 3h 37m 47s. Rate: 10.6 symbols/min [2025-09-16T17:05:42.789175] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-08. [2025-09-16T17:05:42.818786] [INFO ] [MAIN ] AI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:05:42.863517] [DEBUG] [LOCAL ] AI: Loaded 39727 existing records from primary storage. [2025-09-16T17:05:42.863701] [INFO ] [MAIN ] AI: Loaded 39727 existing records. [2025-09-16T17:05:43.042479] [WARN ] [MAIN ] AI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:05:43.042753] [INFO ] [MAIN ] AI: Wiping existing primary data before full refresh. [2025-09-16T17:05:43.042802] [INFO ] [LOCAL ] AI: Starting deletion of primary data. [2025-09-16T17:05:43.043797] [DEBUG] [LOCAL ] AI: Deleted primary data file. [2025-09-16T17:05:43.043859] [INFO ] [MAIN ] AI: Starting full primary refresh. [2025-09-16T17:05:43.308185] [INFO ] [REMOTE ] AHL: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-09-16T17:05:43.344793] [INFO ] [REMOTE ] AHL: Full fetch complete (15974 records). [2025-09-16T17:05:43.360377] [DEBUG] [VALIDATOR ] AHL: Validating primary data. [2025-09-16T17:05:43.489731] [INFO ] [REMOTE ] AI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:05:43.577685] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-02. [2025-09-16T17:05:43.601524] [WARN ] [VALIDATOR ] AHL: Validation failed - insufficient historical data. Have 90 unique days, need 150 days [2025-09-16T17:05:43.602442] [FATAL] [MAIN ] AHL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:05:43.602509] [WARN ] [MAIN ] AHL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:05:43.602559] [INFO ] [LOCAL ] AHL: Starting deletion of primary data. [2025-09-16T17:05:43.602631] [DEBUG] [LOCAL ] AHL: No primary data file found to delete. [2025-09-16T17:05:43.602697] [INFO ] [LOCAL ] AHL: Starting deletion of derivative data. [2025-09-16T17:05:43.602751] [DEBUG] [LOCAL ] AHL: No derivative data file found to delete. [2025-09-16T17:05:43.602813] [INFO ] [MAIN ] Update Progress: 61/2369 symbols completed. AHL took 21.3s. Estimated remaining: 3h 34m 44s. Rate: 10.7 symbols/min [2025-09-16T17:05:43.775901] [INFO ] [MAIN ] AIG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:05:43.819251] [DEBUG] [LOCAL ] AIG: Loaded 39454 existing records from primary storage. [2025-09-16T17:05:43.819412] [INFO ] [MAIN ] AIG: Loaded 39454 existing records. [2025-09-16T17:05:43.995393] [WARN ] [MAIN ] AIG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:05:43.995692] [INFO ] [MAIN ] AIG: Wiping existing primary data before full refresh. [2025-09-16T17:05:43.995742] [INFO ] [LOCAL ] AIG: Starting deletion of primary data. [2025-09-16T17:05:43.996690] [DEBUG] [LOCAL ] AIG: Deleted primary data file. [2025-09-16T17:05:43.996747] [INFO ] [MAIN ] AIG: Starting full primary refresh. [2025-09-16T17:05:44.351161] [INFO ] [REMOTE ] AIG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:05:44.374852] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-02. [2025-09-16T17:05:44.471458] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T17:05:45.174915] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-09. [2025-09-16T17:05:46.245057] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-02 (2.67 seconds). [2025-09-16T17:05:46.255182] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-03. [2025-09-16T17:05:46.389812] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-09-16T17:05:46.395755] [INFO ] [REMOTE ] AHR: Full fetch complete (53661 records). [2025-09-16T17:05:46.395884] [DEBUG] [VALIDATOR ] AHR: Validating primary data. [2025-09-16T17:05:46.772806] [DEBUG] [VALIDATOR ] AHR: All expected trading days are present. [2025-09-16T17:05:46.772894] [DEBUG] [VALIDATOR ] AHR: Primary data validation passed. [2025-09-16T17:05:46.773444] [INFO ] [MAIN ] AHR: Fetched primary data is valid. [2025-09-16T17:05:46.773489] [DEBUG] [LOCAL ] AHR: Saving primary data. [2025-09-16T17:05:46.775411] [DEBUG] [LOCAL ] AHR: Trimmed 53661 records to 36730 (keeping 150 days of data) [2025-09-16T17:05:47.007132] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-02 (2.63 seconds). [2025-09-16T17:05:47.018307] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-03. [2025-09-16T17:05:47.037376] [DEBUG] [LOCAL ] AHR: Primary data saved (36730 records). [2025-09-16T17:05:47.037545] [INFO ] [MAIN ] AHR: Full history saved. [2025-09-16T17:05:47.039251] [INFO ] [MAIN ] Update Progress: 62/2369 symbols completed. AHR took 22.7s. Estimated remaining: 3h 33m 18s. Rate: 10.8 symbols/min [2025-09-16T17:05:47.212689] [INFO ] [MAIN ] AII: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:05:47.212855] [DEBUG] [LOCAL ] AII: No existing data file found in primary storage. [2025-09-16T17:05:47.212893] [WARN ] [MAIN ] AII: No existing data found, performing full refresh instead. [2025-09-16T17:05:47.213019] [INFO ] [MAIN ] AII: Wiping existing primary data before full refresh. [2025-09-16T17:05:47.213062] [INFO ] [LOCAL ] AII: Starting deletion of primary data. [2025-09-16T17:05:47.213107] [DEBUG] [LOCAL ] AII: No primary data file found to delete. [2025-09-16T17:05:47.213141] [INFO ] [MAIN ] AII: Starting full primary refresh. [2025-09-16T17:05:47.331675] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-03 (15.76 seconds). [2025-09-16T17:05:47.577300] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-04. [2025-09-16T17:05:47.580547] [INFO ] [REMOTE ] AII: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:05:48.358946] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-09-16T17:05:48.374847] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-04. [2025-09-16T17:05:48.633252] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-03 (2.38 seconds). [2025-09-16T17:05:48.634032] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-09-16T17:05:49.174915] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-04. [2025-09-16T17:05:49.777068] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T17:05:49.974907] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-05. [2025-09-16T17:05:50.774890] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-05. [2025-09-16T17:05:51.146016] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-09-16T17:05:51.271444] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-09-16T17:05:51.574931] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-06. [2025-09-16T17:05:51.752376] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-09-16T17:05:52.122515] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-09-16T17:05:52.374946] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-02. [2025-09-16T17:05:52.569774] [DEBUG] [REMOTE ] AII: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:05:52.570067] [WARN ] [REMOTE ] AII: No data returned for month 2025-02 [2025-09-16T17:05:53.174906] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-05. [2025-09-16T17:05:53.974915] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-07. [2025-09-16T17:05:54.623048] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-09-16T17:05:54.774899] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-03. [2025-09-16T17:05:54.943947] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-09-16T17:05:55.014036] [DEBUG] [REMOTE ] AII: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:05:55.014303] [WARN ] [REMOTE ] AII: No data returned for month 2025-03 [2025-09-16T17:05:55.574897] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-04. [2025-09-16T17:05:55.795045] [DEBUG] [REMOTE ] AII: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:05:55.795353] [WARN ] [REMOTE ] AII: No data returned for month 2025-04 [2025-09-16T17:05:56.374906] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-08. [2025-09-16T17:05:57.030551] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-09-16T17:05:57.174894] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-06. [2025-09-16T17:05:57.974899] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-05. [2025-09-16T17:05:58.774889] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-09. [2025-09-16T17:05:58.838300] [INFO ] [REMOTE ] AII: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T17:05:58.875966] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T17:05:59.459696] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-09-16T17:05:59.461124] [INFO ] [REMOTE ] AHT: Full fetch complete (6591 records). [2025-09-16T17:05:59.461193] [DEBUG] [VALIDATOR ] AHT: Validating primary data. [2025-09-16T17:05:59.580001] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-07. [2025-09-16T17:05:59.827831] [DEBUG] [VALIDATOR ] AHT: All expected trading days are present. [2025-09-16T17:05:59.827908] [DEBUG] [VALIDATOR ] AHT: Primary data validation passed. [2025-09-16T17:05:59.828096] [INFO ] [MAIN ] AHT: Fetched primary data is valid. [2025-09-16T17:05:59.828170] [DEBUG] [LOCAL ] AHT: Saving primary data. [2025-09-16T17:05:59.828573] [DEBUG] [LOCAL ] AHT: Trimmed 6591 records to 4639 (keeping 150 days of data) [2025-09-16T17:05:59.856120] [DEBUG] [LOCAL ] AHT: Primary data saved (4639 records). [2025-09-16T17:05:59.856231] [INFO ] [MAIN ] AHT: Full history saved. [2025-09-16T17:05:59.856331] [INFO ] [MAIN ] Update Progress: 63/2369 symbols completed. AHT took 30.7s. Estimated remaining: 3h 37m 39s. Rate: 10.6 symbols/min [2025-09-16T17:06:00.032883] [INFO ] [MAIN ] AIN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:00.050905] [DEBUG] [LOCAL ] AIN: Loaded 17889 existing records from primary storage. [2025-09-16T17:06:00.051034] [INFO ] [MAIN ] AIN: Loaded 17889 existing records. [2025-09-16T17:06:00.228248] [WARN ] [MAIN ] AIN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:06:00.228492] [INFO ] [MAIN ] AIN: Wiping existing primary data before full refresh. [2025-09-16T17:06:00.228545] [INFO ] [LOCAL ] AIN: Starting deletion of primary data. [2025-09-16T17:06:00.229036] [DEBUG] [LOCAL ] AIN: Deleted primary data file. [2025-09-16T17:06:00.229082] [INFO ] [MAIN ] AIN: Starting full primary refresh. [2025-09-16T17:06:00.398015] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-06. [2025-09-16T17:06:00.579573] [INFO ] [REMOTE ] AIN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:01.174913] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-02. [2025-09-16T17:06:01.749138] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-09-16T17:06:01.974912] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-08. [2025-09-16T17:06:02.366914] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T17:06:02.774930] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-03. [2025-09-16T17:06:03.266895] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T17:06:03.574877] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-06. [2025-09-16T17:06:03.764623] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-06 (3.37 seconds). [2025-09-16T17:06:03.892361] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-09-16T17:06:04.374965] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-04. [2025-09-16T17:06:04.480090] [INFO ] [REMOTE ] AII: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-09-16T17:06:05.174908] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-09. [2025-09-16T17:06:05.638499] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-09-16T17:06:05.974912] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-05. [2025-09-16T17:06:06.639457] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-09-16T17:06:06.646952] [INFO ] [REMOTE ] AIG: Full fetch complete (58153 records). [2025-09-16T17:06:06.647206] [DEBUG] [VALIDATOR ] AIG: Validating primary data. [2025-09-16T17:06:06.780253] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-07. [2025-09-16T17:06:07.164497] [DEBUG] [VALIDATOR ] AIG: All expected trading days are present. [2025-09-16T17:06:07.164620] [DEBUG] [VALIDATOR ] AIG: Primary data validation passed. [2025-09-16T17:06:07.165583] [INFO ] [MAIN ] AIG: Fetched primary data is valid. [2025-09-16T17:06:07.165674] [DEBUG] [LOCAL ] AIG: Saving primary data. [2025-09-16T17:06:07.167410] [DEBUG] [LOCAL ] AIG: Trimmed 58153 records to 39844 (keeping 150 days of data) [2025-09-16T17:06:07.508223] [INFO ] [REMOTE ] AII: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-09-16T17:06:07.545038] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T17:06:07.566829] [DEBUG] [LOCAL ] AIG: Primary data saved (39844 records). [2025-09-16T17:06:07.567046] [INFO ] [MAIN ] AIG: Full history saved. [2025-09-16T17:06:07.569894] [INFO ] [MAIN ] Update Progress: 64/2369 symbols completed. AIG took 24.0s. Estimated remaining: 3h 38m 47s. Rate: 10.5 symbols/min [2025-09-16T17:06:07.580117] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-06. [2025-09-16T17:06:07.800485] [INFO ] [MAIN ] AIO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:07.813824] [DEBUG] [LOCAL ] AIO: Loaded 9165 existing records from primary storage. [2025-09-16T17:06:07.814007] [INFO ] [MAIN ] AIO: Loaded 9165 existing records. [2025-09-16T17:06:08.042467] [WARN ] [MAIN ] AIO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:06:08.042732] [INFO ] [MAIN ] AIO: Wiping existing primary data before full refresh. [2025-09-16T17:06:08.042781] [INFO ] [LOCAL ] AIO: Starting deletion of primary data. [2025-09-16T17:06:08.043126] [DEBUG] [LOCAL ] AIO: Deleted primary data file. [2025-09-16T17:06:08.043172] [INFO ] [MAIN ] AIO: Starting full primary refresh. [2025-09-16T17:06:08.384531] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-08. [2025-09-16T17:06:08.412391] [INFO ] [REMOTE ] AIO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:08.816139] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T17:06:09.158318] [INFO ] [REMOTE ] AII: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-09-16T17:06:09.174846] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-09. [2025-09-16T17:06:09.974945] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-07. [2025-09-16T17:06:10.774921] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-07. [2025-09-16T17:06:11.574975] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-02. [2025-09-16T17:06:12.262307] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T17:06:12.380058] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-03. [2025-09-16T17:06:12.404239] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-07 (2.43 seconds). [2025-09-16T17:06:13.174963] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-08. [2025-09-16T17:06:15.439348] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-08 (2.26 seconds). [2025-09-16T17:06:15.450992] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-09. [2025-09-16T17:06:15.956734] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-07 (5.18 seconds). [2025-09-16T17:06:15.962405] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-08. [2025-09-16T17:06:17.162986] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-03 (4.78 seconds). [2025-09-16T17:06:17.166674] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-04. [2025-09-16T17:06:17.199294] [INFO ] [REMOTE ] AII: Incremental fetch complete for 2025-09 (8.02 seconds). [2025-09-16T17:06:17.202076] [INFO ] [REMOTE ] AII: Full fetch complete (9600 records). [2025-09-16T17:06:17.202199] [DEBUG] [VALIDATOR ] AII: Validating primary data. [2025-09-16T17:06:17.395439] [WARN ] [VALIDATOR ] AII: Validation failed - insufficient historical data. Have 90 unique days, need 150 days [2025-09-16T17:06:17.395777] [FATAL] [MAIN ] AII: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:06:17.395833] [WARN ] [MAIN ] AII: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:06:17.395884] [INFO ] [LOCAL ] AII: Starting deletion of primary data. [2025-09-16T17:06:17.395954] [DEBUG] [LOCAL ] AII: No primary data file found to delete. [2025-09-16T17:06:17.395994] [INFO ] [LOCAL ] AII: Starting deletion of derivative data. [2025-09-16T17:06:17.396037] [DEBUG] [LOCAL ] AII: No derivative data file found to delete. [2025-09-16T17:06:17.396102] [INFO ] [MAIN ] Update Progress: 65/2369 symbols completed. AII took 30.4s. Estimated remaining: 3h 41m 8s. Rate: 10.4 symbols/min [2025-09-16T17:06:17.568368] [INFO ] [MAIN ] AIR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:17.589916] [DEBUG] [LOCAL ] AIR: Loaded 19182 existing records from primary storage. [2025-09-16T17:06:17.590062] [INFO ] [MAIN ] AIR: Loaded 19182 existing records. [2025-09-16T17:06:17.776290] [WARN ] [MAIN ] AIR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:06:17.776737] [INFO ] [MAIN ] AIR: Wiping existing primary data before full refresh. [2025-09-16T17:06:17.776854] [INFO ] [LOCAL ] AIR: Starting deletion of primary data. [2025-09-16T17:06:17.784123] [DEBUG] [LOCAL ] AIR: Deleted primary data file. [2025-09-16T17:06:17.796359] [INFO ] [MAIN ] AIR: Starting full primary refresh. [2025-09-16T17:06:17.830605] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-09 (2.38 seconds). [2025-09-16T17:06:17.903935] [INFO ] [REMOTE ] AI: Full fetch complete (58428 records). [2025-09-16T17:06:17.917630] [DEBUG] [VALIDATOR ] AI: Validating primary data. [2025-09-16T17:06:18.436273] [INFO ] [REMOTE ] AIR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:18.475030] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-02. [2025-09-16T17:06:18.529661] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-08 (2.57 seconds). [2025-09-16T17:06:18.593695] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:06:18.641093] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-09. [2025-09-16T17:06:18.645338] [DEBUG] [VALIDATOR ] AI: All expected trading days are present. [2025-09-16T17:06:18.645580] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-05. [2025-09-16T17:06:18.646472] [DEBUG] [VALIDATOR ] AI: Primary data validation passed. [2025-09-16T17:06:18.647667] [INFO ] [MAIN ] AI: Fetched primary data is valid. [2025-09-16T17:06:18.647876] [DEBUG] [LOCAL ] AI: Saving primary data. [2025-09-16T17:06:18.649042] [DEBUG] [LOCAL ] AI: Trimmed 58428 records to 40117 (keeping 150 days of data) [2025-09-16T17:06:18.888424] [DEBUG] [LOCAL ] AI: Primary data saved (40117 records). [2025-09-16T17:06:18.888559] [INFO ] [MAIN ] AI: Full history saved. [2025-09-16T17:06:18.890932] [INFO ] [MAIN ] Update Progress: 66/2369 symbols completed. AI took 36.2s. Estimated remaining: 3h 38m 33s. Rate: 10.5 symbols/min [2025-09-16T17:06:19.060930] [INFO ] [MAIN ] AIT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:19.081818] [DEBUG] [LOCAL ] AIT: Loaded 20259 existing records from primary storage. [2025-09-16T17:06:19.081978] [INFO ] [MAIN ] AIT: Loaded 20259 existing records. [2025-09-16T17:06:19.263238] [WARN ] [MAIN ] AIT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:06:19.263480] [INFO ] [MAIN ] AIT: Wiping existing primary data before full refresh. [2025-09-16T17:06:19.263527] [INFO ] [LOCAL ] AIT: Starting deletion of primary data. [2025-09-16T17:06:19.264124] [DEBUG] [LOCAL ] AIT: Deleted primary data file. [2025-09-16T17:06:19.264186] [INFO ] [MAIN ] AIT: Starting full primary refresh. [2025-09-16T17:06:19.637347] [INFO ] [REMOTE ] AIT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:19.638192] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-02. [2025-09-16T17:06:19.638590] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T17:06:19.642818] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-06. [2025-09-16T17:06:19.650912] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T17:06:19.653701] [INFO ] [REMOTE ] AIN: Full fetch complete (26057 records). [2025-09-16T17:06:19.653774] [DEBUG] [VALIDATOR ] AIN: Validating primary data. [2025-09-16T17:06:19.951087] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T17:06:20.034799] [DEBUG] [VALIDATOR ] AIN: All expected trading days are present. [2025-09-16T17:06:20.034871] [DEBUG] [VALIDATOR ] AIN: Primary data validation passed. [2025-09-16T17:06:20.035253] [INFO ] [MAIN ] AIN: Fetched primary data is valid. [2025-09-16T17:06:20.035296] [DEBUG] [LOCAL ] AIN: Saving primary data. [2025-09-16T17:06:20.035946] [DEBUG] [LOCAL ] AIN: Trimmed 26057 records to 18026 (keeping 150 days of data) [2025-09-16T17:06:20.143183] [DEBUG] [LOCAL ] AIN: Primary data saved (18026 records). [2025-09-16T17:06:20.143327] [INFO ] [MAIN ] AIN: Full history saved. [2025-09-16T17:06:20.143412] [INFO ] [MAIN ] Update Progress: 67/2369 symbols completed. AIN took 20.3s. Estimated remaining: 3h 35m 55s. Rate: 10.7 symbols/min [2025-09-16T17:06:20.314776] [INFO ] [MAIN ] AIV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:20.324488] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-09-16T17:06:20.352826] [DEBUG] [LOCAL ] AIV: Loaded 29479 existing records from primary storage. [2025-09-16T17:06:20.352956] [INFO ] [MAIN ] AIV: Loaded 29479 existing records. [2025-09-16T17:06:20.383736] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-03. [2025-09-16T17:06:20.530892] [WARN ] [MAIN ] AIV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:06:20.531189] [INFO ] [MAIN ] AIV: Wiping existing primary data before full refresh. [2025-09-16T17:06:20.531236] [INFO ] [LOCAL ] AIV: Starting deletion of primary data. [2025-09-16T17:06:20.532016] [DEBUG] [LOCAL ] AIV: Deleted primary data file. [2025-09-16T17:06:20.532068] [INFO ] [MAIN ] AIV: Starting full primary refresh. [2025-09-16T17:06:20.885677] [INFO ] [REMOTE ] AIV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:20.899745] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-09-16T17:06:21.174917] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-07. [2025-09-16T17:06:21.722992] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-09-16T17:06:21.974880] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-04. [2025-09-16T17:06:22.078605] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-09-16T17:06:22.774887] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-03. [2025-09-16T17:06:23.264283] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-09-16T17:06:23.574914] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-05. [2025-09-16T17:06:23.669773] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-09-16T17:06:24.374887] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-08. [2025-09-16T17:06:24.672135] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-09-16T17:06:25.107270] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-09-16T17:06:25.174871] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-09. [2025-09-16T17:06:25.775983] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-09-16T17:06:25.777815] [INFO ] [REMOTE ] AIO: Full fetch complete (14686 records). [2025-09-16T17:06:25.777888] [DEBUG] [VALIDATOR ] AIO: Validating primary data. [2025-09-16T17:06:25.980024] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-02. [2025-09-16T17:06:26.162609] [DEBUG] [VALIDATOR ] AIO: All expected trading days are present. [2025-09-16T17:06:26.162781] [DEBUG] [VALIDATOR ] AIO: Primary data validation passed. [2025-09-16T17:06:26.163049] [INFO ] [MAIN ] AIO: Fetched primary data is valid. [2025-09-16T17:06:26.163092] [DEBUG] [LOCAL ] AIO: Saving primary data. [2025-09-16T17:06:26.163772] [DEBUG] [LOCAL ] AIO: Trimmed 14686 records to 9228 (keeping 150 days of data) [2025-09-16T17:06:26.224827] [DEBUG] [LOCAL ] AIO: Primary data saved (9228 records). [2025-09-16T17:06:26.224997] [INFO ] [MAIN ] AIO: Full history saved. [2025-09-16T17:06:26.225109] [INFO ] [MAIN ] Update Progress: 68/2369 symbols completed. AIO took 18.7s. Estimated remaining: 3h 36m 5s. Rate: 10.6 symbols/min [2025-09-16T17:06:26.393956] [INFO ] [MAIN ] AIZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:26.417386] [DEBUG] [LOCAL ] AIZ: Loaded 22213 existing records from primary storage. [2025-09-16T17:06:26.417528] [INFO ] [MAIN ] AIZ: Loaded 22213 existing records. [2025-09-16T17:06:26.591925] [WARN ] [MAIN ] AIZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:06:26.592183] [INFO ] [MAIN ] AIZ: Wiping existing primary data before full refresh. [2025-09-16T17:06:26.592230] [INFO ] [LOCAL ] AIZ: Starting deletion of primary data. [2025-09-16T17:06:26.592802] [DEBUG] [LOCAL ] AIZ: Deleted primary data file. [2025-09-16T17:06:26.592851] [INFO ] [MAIN ] AIZ: Starting full primary refresh. [2025-09-16T17:06:26.787703] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-04. [2025-09-16T17:06:26.973491] [INFO ] [REMOTE ] AIZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:27.574985] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-06. [2025-09-16T17:06:27.607396] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T17:06:27.867999] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-09-16T17:06:28.374915] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-05. [2025-09-16T17:06:28.455233] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-09-16T17:06:29.174908] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-07. [2025-09-16T17:06:29.641974] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T17:06:29.974907] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-03. [2025-09-16T17:06:30.325882] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T17:06:30.774908] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-06. [2025-09-16T17:06:30.983823] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-09-16T17:06:31.574904] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-08. [2025-09-16T17:06:32.374877] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-04. [2025-09-16T17:06:32.410442] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T17:06:33.037888] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-09-16T17:06:33.174911] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-09. [2025-09-16T17:06:33.615809] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-09-16T17:06:33.974950] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-05. [2025-09-16T17:06:34.312790] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T17:06:34.316327] [INFO ] [REMOTE ] AIR: Full fetch complete (28776 records). [2025-09-16T17:06:34.316418] [DEBUG] [VALIDATOR ] AIR: Validating primary data. [2025-09-16T17:06:34.711078] [DEBUG] [VALIDATOR ] AIR: All expected trading days are present. [2025-09-16T17:06:34.711164] [DEBUG] [VALIDATOR ] AIR: Primary data validation passed. [2025-09-16T17:06:34.711557] [INFO ] [MAIN ] AIR: Fetched primary data is valid. [2025-09-16T17:06:34.711607] [DEBUG] [LOCAL ] AIR: Saving primary data. [2025-09-16T17:06:34.712537] [DEBUG] [LOCAL ] AIR: Trimmed 28776 records to 19453 (keeping 150 days of data) [2025-09-16T17:06:34.795723] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-07. [2025-09-16T17:06:34.828617] [DEBUG] [LOCAL ] AIR: Primary data saved (19453 records). [2025-09-16T17:06:34.828804] [INFO ] [MAIN ] AIR: Full history saved. [2025-09-16T17:06:34.828896] [INFO ] [MAIN ] Update Progress: 69/2369 symbols completed. AIR took 17.4s. Estimated remaining: 3h 37m 38s. Rate: 10.6 symbols/min [2025-09-16T17:06:35.013268] [INFO ] [MAIN ] AIZN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:35.018070] [DEBUG] [LOCAL ] AIZN: Loaded 2612 existing records from primary storage. [2025-09-16T17:06:35.018169] [INFO ] [MAIN ] AIZN: Loaded 2612 existing records. [2025-09-16T17:06:35.206959] [WARN ] [MAIN ] AIZN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:06:35.207401] [INFO ] [MAIN ] AIZN: Wiping existing primary data before full refresh. [2025-09-16T17:06:35.207464] [INFO ] [LOCAL ] AIZN: Starting deletion of primary data. [2025-09-16T17:06:35.207889] [DEBUG] [LOCAL ] AIZN: Deleted primary data file. [2025-09-16T17:06:35.207946] [INFO ] [MAIN ] AIZN: Starting full primary refresh. [2025-09-16T17:06:35.276345] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T17:06:35.586172] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-06. [2025-09-16T17:06:35.637102] [INFO ] [REMOTE ] AIZN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:36.186235] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-09-16T17:06:36.374928] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-08. [2025-09-16T17:06:36.994722] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T17:06:37.118870] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-09-16T17:06:37.174945] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-02. [2025-09-16T17:06:37.748920] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-09-16T17:06:37.974961] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-07. [2025-09-16T17:06:38.774936] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-03. [2025-09-16T17:06:39.166913] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-09-16T17:06:39.555429] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-09-16T17:06:39.574868] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-08. [2025-09-16T17:06:40.374934] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-02. [2025-09-16T17:06:40.945451] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-09-16T17:06:41.174917] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-04. [2025-09-16T17:06:41.664701] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-09-16T17:06:41.937855] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T17:06:41.974874] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-03. [2025-09-16T17:06:42.774920] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-09. [2025-09-16T17:06:43.438473] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T17:06:43.574889] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-04. [2025-09-16T17:06:43.889098] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T17:06:43.894476] [INFO ] [REMOTE ] AIV: Full fetch complete (43626 records). [2025-09-16T17:06:43.894572] [DEBUG] [VALIDATOR ] AIV: Validating primary data. [2025-09-16T17:06:44.276204] [DEBUG] [VALIDATOR ] AIV: All expected trading days are present. [2025-09-16T17:06:44.276298] [DEBUG] [VALIDATOR ] AIV: Primary data validation passed. [2025-09-16T17:06:44.276808] [INFO ] [MAIN ] AIV: Fetched primary data is valid. [2025-09-16T17:06:44.276855] [DEBUG] [LOCAL ] AIV: Saving primary data. [2025-09-16T17:06:44.277922] [DEBUG] [LOCAL ] AIV: Trimmed 43626 records to 29849 (keeping 150 days of data) [2025-09-16T17:06:44.374955] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-05. [2025-09-16T17:06:44.451882] [DEBUG] [LOCAL ] AIV: Primary data saved (29849 records). [2025-09-16T17:06:44.452048] [INFO ] [MAIN ] AIV: Full history saved. [2025-09-16T17:06:44.452149] [INFO ] [MAIN ] Update Progress: 70/2369 symbols completed. AIV took 24.3s. Estimated remaining: 3h 39m 42s. Rate: 10.5 symbols/min [2025-09-16T17:06:44.625142] [INFO ] [MAIN ] AJG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:44.660799] [DEBUG] [LOCAL ] AJG: Loaded 34227 existing records from primary storage. [2025-09-16T17:06:44.660961] [INFO ] [MAIN ] AJG: Loaded 34227 existing records. [2025-09-16T17:06:44.840103] [WARN ] [MAIN ] AJG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:06:44.840384] [INFO ] [MAIN ] AJG: Wiping existing primary data before full refresh. [2025-09-16T17:06:44.840501] [INFO ] [LOCAL ] AJG: Starting deletion of primary data. [2025-09-16T17:06:44.841345] [DEBUG] [LOCAL ] AJG: Deleted primary data file. [2025-09-16T17:06:44.841397] [INFO ] [MAIN ] AJG: Starting full primary refresh. [2025-09-16T17:06:45.046571] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-09-16T17:06:45.128304] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-09-16T17:06:45.179948] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-06. [2025-09-16T17:06:45.235379] [INFO ] [REMOTE ] AJG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:45.790823] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-09-16T17:06:45.974923] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-02. [2025-09-16T17:06:46.774938] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-07. [2025-09-16T17:06:47.343803] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-09-16T17:06:47.400930] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-09-16T17:06:47.574912] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-03. [2025-09-16T17:06:48.374890] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-05. [2025-09-16T17:06:49.174864] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-09. [2025-09-16T17:06:49.285950] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T17:06:49.682373] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T17:06:49.871897] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-09-16T17:06:49.874867] [INFO ] [REMOTE ] AIT: Full fetch complete (30559 records). [2025-09-16T17:06:49.874943] [DEBUG] [VALIDATOR ] AIT: Validating primary data. [2025-09-16T17:06:49.978604] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-08. [2025-09-16T17:06:50.249490] [DEBUG] [VALIDATOR ] AIT: All expected trading days are present. [2025-09-16T17:06:50.249585] [DEBUG] [VALIDATOR ] AIT: Primary data validation passed. [2025-09-16T17:06:50.249965] [INFO ] [MAIN ] AIT: Fetched primary data is valid. [2025-09-16T17:06:50.250014] [DEBUG] [LOCAL ] AIT: Saving primary data. [2025-09-16T17:06:50.250959] [DEBUG] [LOCAL ] AIT: Trimmed 30559 records to 20432 (keeping 150 days of data) [2025-09-16T17:06:50.386714] [DEBUG] [LOCAL ] AIT: Primary data saved (20432 records). [2025-09-16T17:06:50.386867] [INFO ] [MAIN ] AIT: Full history saved. [2025-09-16T17:06:50.386968] [INFO ] [MAIN ] Update Progress: 71/2369 symbols completed. AIT took 31.5s. Estimated remaining: 3h 39m 43s. Rate: 10.5 symbols/min [2025-09-16T17:06:50.557496] [INFO ] [MAIN ] AKA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:50.557750] [DEBUG] [LOCAL ] AKA: No existing data file found in primary storage. [2025-09-16T17:06:50.557792] [WARN ] [MAIN ] AKA: No existing data found, performing full refresh instead. [2025-09-16T17:06:50.557917] [INFO ] [MAIN ] AKA: Wiping existing primary data before full refresh. [2025-09-16T17:06:50.557957] [INFO ] [LOCAL ] AKA: Starting deletion of primary data. [2025-09-16T17:06:50.557999] [DEBUG] [LOCAL ] AKA: No primary data file found to delete. [2025-09-16T17:06:50.558031] [INFO ] [MAIN ] AKA: Starting full primary refresh. [2025-09-16T17:06:50.784435] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-04. [2025-09-16T17:06:50.851897] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-09-16T17:06:50.934343] [INFO ] [REMOTE ] AKA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:51.574896] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-06. [2025-09-16T17:06:52.374898] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-09. [2025-09-16T17:06:52.759576] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-09-16T17:06:52.871393] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:06:52.877678] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-09-16T17:06:52.884936] [INFO ] [REMOTE ] AIZN: Full fetch complete (4222 records). [2025-09-16T17:06:52.885106] [DEBUG] [VALIDATOR ] AIZN: Validating primary data. [2025-09-16T17:06:53.179999] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-02. [2025-09-16T17:06:53.248290] [DEBUG] [VALIDATOR ] AIZN: All expected trading days are present. [2025-09-16T17:06:53.248363] [DEBUG] [VALIDATOR ] AIZN: Primary data validation passed. [2025-09-16T17:06:53.248511] [INFO ] [MAIN ] AIZN: Fetched primary data is valid. [2025-09-16T17:06:53.248552] [DEBUG] [LOCAL ] AIZN: Saving primary data. [2025-09-16T17:06:53.248946] [DEBUG] [LOCAL ] AIZN: Trimmed 4222 records to 2643 (keeping 150 days of data) [2025-09-16T17:06:53.265434] [DEBUG] [LOCAL ] AIZN: Primary data saved (2643 records). [2025-09-16T17:06:53.265500] [INFO ] [MAIN ] AIZN: Full history saved. [2025-09-16T17:06:53.265583] [INFO ] [MAIN ] Update Progress: 72/2369 symbols completed. AIZN took 18.4s. Estimated remaining: 3h 38m 6s. Rate: 10.5 symbols/min [2025-09-16T17:06:53.436874] [INFO ] [MAIN ] AKR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:06:53.476706] [DEBUG] [LOCAL ] AKR: Loaded 33836 existing records from primary storage. [2025-09-16T17:06:53.476879] [INFO ] [MAIN ] AKR: Loaded 33836 existing records. [2025-09-16T17:06:53.668395] [WARN ] [MAIN ] AKR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:06:53.668657] [INFO ] [MAIN ] AKR: Wiping existing primary data before full refresh. [2025-09-16T17:06:53.668738] [INFO ] [LOCAL ] AKR: Starting deletion of primary data. [2025-09-16T17:06:53.669552] [DEBUG] [LOCAL ] AKR: Deleted primary data file. [2025-09-16T17:06:53.669602] [INFO ] [MAIN ] AKR: Starting full primary refresh. [2025-09-16T17:06:53.785964] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-09-16T17:06:53.983808] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-07. [2025-09-16T17:06:54.101890] [INFO ] [REMOTE ] AKR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:06:54.774908] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-02. [2025-09-16T17:06:55.033416] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-09-16T17:06:55.574917] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-03. [2025-09-16T17:06:56.011090] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-09-16T17:06:56.025493] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-09-16T17:06:56.374915] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-04. [2025-09-16T17:06:56.853381] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-09-16T17:06:57.174908] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-05. [2025-09-16T17:06:57.721678] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-09-16T17:06:57.974920] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-03. [2025-09-16T17:06:58.774895] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-05. [2025-09-16T17:06:59.453237] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T17:06:59.574907] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-04. [2025-09-16T17:07:00.357590] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T17:07:00.374990] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-06. [2025-09-16T17:07:01.174926] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-08. [2025-09-16T17:07:01.181977] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-09-16T17:07:01.605161] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-09-16T17:07:01.974934] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-07. [2025-09-16T17:07:02.339217] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-09-16T17:07:02.774869] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-06. [2025-09-16T17:07:03.236463] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-09-16T17:07:03.574958] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-08. [2025-09-16T17:07:04.050789] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-09-16T17:07:04.374873] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-09. [2025-09-16T17:07:04.421266] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T17:07:05.174906] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-09. [2025-09-16T17:07:05.536325] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-09-16T17:07:05.539709] [INFO ] [REMOTE ] AIZ: Full fetch complete (33003 records). [2025-09-16T17:07:05.539781] [DEBUG] [VALIDATOR ] AIZ: Validating primary data. [2025-09-16T17:07:05.540409] [FATAL] [VALIDATOR ] AIZ: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:07:05.540500] [FATAL] [MAIN ] AIZ: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:07:05.540545] [WARN ] [MAIN ] AIZ: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:07:05.540585] [INFO ] [LOCAL ] AIZ: Starting deletion of primary data. [2025-09-16T17:07:05.540667] [DEBUG] [LOCAL ] AIZ: No primary data file found to delete. [2025-09-16T17:07:05.540717] [INFO ] [LOCAL ] AIZ: Starting deletion of derivative data. [2025-09-16T17:07:05.541096] [DEBUG] [LOCAL ] AIZ: Deleted derivative data file. [2025-09-16T17:07:05.541182] [INFO ] [MAIN ] Update Progress: 73/2369 symbols completed. AIZ took 39.3s. Estimated remaining: 3h 41m 27s. Rate: 10.4 symbols/min [2025-09-16T17:07:05.619713] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-09-16T17:07:05.644785] [INFO ] [REMOTE ] AKA: Full fetch complete (1762 records). [2025-09-16T17:07:05.647099] [DEBUG] [VALIDATOR ] AKA: Validating primary data. [2025-09-16T17:07:05.861553] [INFO ] [MAIN ] AL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:07:05.898261] [WARN ] [VALIDATOR ] AKA: Validation failed - insufficient historical data. Have 148 unique days, need 150 days [2025-09-16T17:07:05.898795] [FATAL] [MAIN ] AKA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:07:05.898976] [WARN ] [MAIN ] AKA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:07:05.899049] [INFO ] [LOCAL ] AKA: Starting deletion of primary data. [2025-09-16T17:07:05.899127] [DEBUG] [LOCAL ] AKA: No primary data file found to delete. [2025-09-16T17:07:05.899166] [INFO ] [LOCAL ] AKA: Starting deletion of derivative data. [2025-09-16T17:07:05.899205] [DEBUG] [LOCAL ] AKA: No derivative data file found to delete. [2025-09-16T17:07:05.899267] [INFO ] [MAIN ] Update Progress: 74/2369 symbols completed. AKA took 15.5s. Estimated remaining: 3h 38m 33s. Rate: 10.5 symbols/min [2025-09-16T17:07:05.979552] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-07. [2025-09-16T17:07:06.085898] [INFO ] [MAIN ] ALB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:07:06.097621] [DEBUG] [LOCAL ] AL: Loaded 32168 existing records from primary storage. [2025-09-16T17:07:06.097798] [INFO ] [MAIN ] AL: Loaded 32168 existing records. [2025-09-16T17:07:06.297275] [WARN ] [MAIN ] AL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:07:06.297555] [INFO ] [MAIN ] AL: Wiping existing primary data before full refresh. [2025-09-16T17:07:06.297606] [INFO ] [LOCAL ] AL: Starting deletion of primary data. [2025-09-16T17:07:06.298378] [DEBUG] [LOCAL ] AL: Deleted primary data file. [2025-09-16T17:07:06.298431] [INFO ] [MAIN ] AL: Starting full primary refresh. [2025-09-16T17:07:06.416409] [DEBUG] [LOCAL ] ALB: Loaded 39419 existing records from primary storage. [2025-09-16T17:07:06.427091] [INFO ] [MAIN ] ALB: Loaded 39419 existing records. [2025-09-16T17:07:06.774945] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-05. [2025-09-16T17:07:06.818670] [WARN ] [MAIN ] ALB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:07:06.845344] [INFO ] [MAIN ] ALB: Wiping existing primary data before full refresh. [2025-09-16T17:07:06.862854] [INFO ] [LOCAL ] ALB: Starting deletion of primary data. [2025-09-16T17:07:06.883413] [INFO ] [REMOTE ] AL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:07:06.884455] [DEBUG] [LOCAL ] ALB: Deleted primary data file. [2025-09-16T17:07:06.884521] [INFO ] [MAIN ] ALB: Starting full primary refresh. [2025-09-16T17:07:07.297275] [INFO ] [REMOTE ] ALB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:07:07.574899] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-02. [2025-09-16T17:07:07.873266] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T17:07:08.245544] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T17:07:08.374902] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-08. [2025-09-16T17:07:08.984028] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-09-16T17:07:09.174908] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-02. [2025-09-16T17:07:09.791809] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T17:07:09.974912] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-03. [2025-09-16T17:07:10.744857] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T17:07:10.774903] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-09. [2025-09-16T17:07:11.284083] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T17:07:11.574962] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-06. [2025-09-16T17:07:12.374937] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-03. [2025-09-16T17:07:12.618405] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-09-16T17:07:12.624270] [INFO ] [REMOTE ] AJG: Full fetch complete (51437 records). [2025-09-16T17:07:12.624423] [DEBUG] [VALIDATOR ] AJG: Validating primary data. [2025-09-16T17:07:13.031277] [DEBUG] [VALIDATOR ] AJG: All expected trading days are present. [2025-09-16T17:07:13.031441] [DEBUG] [VALIDATOR ] AJG: Primary data validation passed. [2025-09-16T17:07:13.032018] [INFO ] [MAIN ] AJG: Fetched primary data is valid. [2025-09-16T17:07:13.032122] [DEBUG] [LOCAL ] AJG: Saving primary data. [2025-09-16T17:07:13.033246] [DEBUG] [LOCAL ] AJG: Trimmed 51437 records to 34554 (keeping 150 days of data) [2025-09-16T17:07:13.157711] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T17:07:13.174993] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-04. [2025-09-16T17:07:13.292908] [DEBUG] [LOCAL ] AJG: Primary data saved (34554 records). [2025-09-16T17:07:13.293057] [INFO ] [MAIN ] AJG: Full history saved. [2025-09-16T17:07:13.295979] [INFO ] [MAIN ] Update Progress: 75/2369 symbols completed. AJG took 28.8s. Estimated remaining: 3h 39m 19s. Rate: 10.5 symbols/min [2025-09-16T17:07:13.464964] [INFO ] [MAIN ] ALC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:07:13.502942] [DEBUG] [LOCAL ] ALC: Loaded 37480 existing records from primary storage. [2025-09-16T17:07:13.503082] [INFO ] [MAIN ] ALC: Loaded 37480 existing records. [2025-09-16T17:07:13.678143] [WARN ] [MAIN ] ALC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:07:13.678395] [INFO ] [MAIN ] ALC: Wiping existing primary data before full refresh. [2025-09-16T17:07:13.678449] [INFO ] [LOCAL ] ALC: Starting deletion of primary data. [2025-09-16T17:07:13.679215] [DEBUG] [LOCAL ] ALC: Deleted primary data file. [2025-09-16T17:07:13.679270] [INFO ] [MAIN ] ALC: Starting full primary refresh. [2025-09-16T17:07:13.980038] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-07. [2025-09-16T17:07:14.037637] [INFO ] [REMOTE ] ALC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:07:14.178288] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T17:07:14.499966] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T17:07:14.774947] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-02. [2025-09-16T17:07:15.340119] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-09-16T17:07:15.574894] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-08. [2025-09-16T17:07:16.374912] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-05. [2025-09-16T17:07:16.525692] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T17:07:16.634703] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-09-16T17:07:17.174908] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-09. [2025-09-16T17:07:17.983717] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-03. [2025-09-16T17:07:17.988896] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T17:07:18.655986] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T17:07:18.661444] [INFO ] [REMOTE ] AKR: Full fetch complete (49441 records). [2025-09-16T17:07:18.661554] [DEBUG] [VALIDATOR ] AKR: Validating primary data. [2025-09-16T17:07:18.780059] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-06. [2025-09-16T17:07:19.045882] [DEBUG] [VALIDATOR ] AKR: All expected trading days are present. [2025-09-16T17:07:19.045990] [DEBUG] [VALIDATOR ] AKR: Primary data validation passed. [2025-09-16T17:07:19.046507] [INFO ] [MAIN ] AKR: Fetched primary data is valid. [2025-09-16T17:07:19.046549] [DEBUG] [LOCAL ] AKR: Saving primary data. [2025-09-16T17:07:19.048102] [DEBUG] [LOCAL ] AKR: Trimmed 49441 records to 34176 (keeping 150 days of data) [2025-09-16T17:07:19.254624] [DEBUG] [LOCAL ] AKR: Primary data saved (34176 records). [2025-09-16T17:07:19.254809] [INFO ] [MAIN ] AKR: Full history saved. [2025-09-16T17:07:19.254904] [INFO ] [MAIN ] Update Progress: 76/2369 symbols completed. AKR took 26.0s. Estimated remaining: 3h 39m 20s. Rate: 10.5 symbols/min [2025-09-16T17:07:19.437328] [INFO ] [MAIN ] ALE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:07:19.468415] [DEBUG] [LOCAL ] ALE: Loaded 22355 existing records from primary storage. [2025-09-16T17:07:19.468605] [INFO ] [MAIN ] ALE: Loaded 22355 existing records. [2025-09-16T17:07:19.575169] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-04. [2025-09-16T17:07:19.653332] [WARN ] [MAIN ] ALE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:07:19.653965] [INFO ] [MAIN ] ALE: Wiping existing primary data before full refresh. [2025-09-16T17:07:19.654060] [INFO ] [LOCAL ] ALE: Starting deletion of primary data. [2025-09-16T17:07:19.654736] [DEBUG] [LOCAL ] ALE: Deleted primary data file. [2025-09-16T17:07:19.654790] [INFO ] [MAIN ] ALE: Starting full primary refresh. [2025-09-16T17:07:19.990059] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-09-16T17:07:20.084275] [INFO ] [REMOTE ] ALE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:07:20.334923] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-09-16T17:07:20.374866] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-07. [2025-09-16T17:07:21.174908] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-04. [2025-09-16T17:07:21.320974] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-09-16T17:07:21.367590] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-09-16T17:07:21.974929] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-02. [2025-09-16T17:07:22.544111] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T17:07:22.774956] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-08. [2025-09-16T17:07:23.186743] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-09-16T17:07:23.574945] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-05. [2025-09-16T17:07:23.938016] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T17:07:24.374962] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-03. [2025-09-16T17:07:24.995728] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-09-16T17:07:25.165441] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T17:07:25.174839] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-04. [2025-09-16T17:07:25.974915] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-09. [2025-09-16T17:07:26.354538] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T17:07:26.774939] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-05. [2025-09-16T17:07:27.362956] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T17:07:27.369003] [INFO ] [REMOTE ] AL: Full fetch complete (47611 records). [2025-09-16T17:07:27.369095] [DEBUG] [VALIDATOR ] AL: Validating primary data. [2025-09-16T17:07:27.585944] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-06. [2025-09-16T17:07:27.744524] [DEBUG] [VALIDATOR ] AL: All expected trading days are present. [2025-09-16T17:07:27.744621] [DEBUG] [VALIDATOR ] AL: Primary data validation passed. [2025-09-16T17:07:27.745149] [INFO ] [MAIN ] AL: Fetched primary data is valid. [2025-09-16T17:07:27.745195] [DEBUG] [LOCAL ] AL: Saving primary data. [2025-09-16T17:07:27.746795] [DEBUG] [LOCAL ] AL: Trimmed 47611 records to 32517 (keeping 150 days of data) [2025-09-16T17:07:27.889962] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T17:07:27.976930] [DEBUG] [LOCAL ] AL: Primary data saved (32517 records). [2025-09-16T17:07:27.977083] [INFO ] [MAIN ] AL: Full history saved. [2025-09-16T17:07:27.977178] [INFO ] [MAIN ] Update Progress: 77/2369 symbols completed. AL took 22.4s. Estimated remaining: 3h 40m 43s. Rate: 10.4 symbols/min [2025-09-16T17:07:28.375004] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-06. [2025-09-16T17:07:29.133791] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-09-16T17:07:29.174873] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-07. [2025-09-16T17:07:29.450563] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T17:07:29.974916] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-05. [2025-09-16T17:07:30.774865] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-07. [2025-09-16T17:07:30.942606] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-09-16T17:07:31.574880] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-08. [2025-09-16T17:07:31.682998] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-09-16T17:07:31.772309] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T17:07:32.374901] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-06. [2025-09-16T17:07:33.174890] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-08. [2025-09-16T17:07:33.636032] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-09-16T17:07:33.674159] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:07:33.974948] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-09. [2025-09-16T17:07:34.354743] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T17:07:34.774909] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-09. [2025-09-16T17:07:35.574878] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-07. [2025-09-16T17:07:35.868037] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-09-16T17:07:35.874500] [INFO ] [REMOTE ] ALB: Full fetch complete (57868 records). [2025-09-16T17:07:35.874768] [DEBUG] [VALIDATOR ] ALB: Validating primary data. [2025-09-16T17:07:35.967390] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-09-16T17:07:36.016436] [INFO ] [REMOTE ] ALE: Full fetch complete (32061 records). [2025-09-16T17:07:36.026889] [DEBUG] [VALIDATOR ] ALE: Validating primary data. [2025-09-16T17:07:36.567678] [DEBUG] [VALIDATOR ] ALB: All expected trading days are present. [2025-09-16T17:07:36.583889] [DEBUG] [VALIDATOR ] ALB: Primary data validation passed. [2025-09-16T17:07:36.592541] [INFO ] [MAIN ] ALB: Fetched primary data is valid. [2025-09-16T17:07:36.594955] [DEBUG] [LOCAL ] ALB: Saving primary data. [2025-09-16T17:07:36.671071] [DEBUG] [LOCAL ] ALB: Trimmed 57868 records to 39803 (keeping 150 days of data) [2025-09-16T17:07:36.678975] [DEBUG] [VALIDATOR ] ALE: All expected trading days are present. [2025-09-16T17:07:36.679037] [DEBUG] [VALIDATOR ] ALE: Primary data validation passed. [2025-09-16T17:07:36.679682] [INFO ] [MAIN ] ALE: Fetched primary data is valid. [2025-09-16T17:07:36.679924] [DEBUG] [LOCAL ] ALE: Saving primary data. [2025-09-16T17:07:36.681287] [DEBUG] [LOCAL ] ALE: Trimmed 32061 records to 22661 (keeping 150 days of data) [2025-09-16T17:07:37.000911] [DEBUG] [LOCAL ] ALE: Primary data saved (22661 records). [2025-09-16T17:07:37.010321] [INFO ] [MAIN ] ALE: Full history saved. [2025-09-16T17:07:37.010634] [INFO ] [MAIN ] Update Progress: 78/2369 symbols completed. ALE took 17.8s. Estimated remaining: 3h 42m 13s. Rate: 10.3 symbols/min [2025-09-16T17:07:37.078167] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T17:07:37.091181] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-08. [2025-09-16T17:07:37.112765] [DEBUG] [LOCAL ] ALB: Primary data saved (39803 records). [2025-09-16T17:07:37.112914] [INFO ] [MAIN ] ALB: Full history saved. [2025-09-16T17:07:37.114019] [INFO ] [MAIN ] Update Progress: 79/2369 symbols completed. ALB took 31.2s. Estimated remaining: 3h 39m 21s. Rate: 10.4 symbols/min [2025-09-16T17:07:38.590608] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T17:07:38.601537] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-09. [2025-09-16T17:07:40.602743] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-09-16T17:07:40.608900] [INFO ] [REMOTE ] ALC: Full fetch complete (55382 records). [2025-09-16T17:07:40.609035] [DEBUG] [VALIDATOR ] ALC: Validating primary data. [2025-09-16T17:07:40.992109] [DEBUG] [VALIDATOR ] ALC: All expected trading days are present. [2025-09-16T17:07:40.992212] [DEBUG] [VALIDATOR ] ALC: Primary data validation passed. [2025-09-16T17:07:40.992972] [INFO ] [MAIN ] ALC: Fetched primary data is valid. [2025-09-16T17:07:40.993034] [DEBUG] [LOCAL ] ALC: Saving primary data. [2025-09-16T17:07:40.994400] [DEBUG] [LOCAL ] ALC: Trimmed 55382 records to 37854 (keeping 150 days of data) [2025-09-16T17:07:41.226617] [DEBUG] [LOCAL ] ALC: Primary data saved (37854 records). [2025-09-16T17:07:41.226769] [INFO ] [MAIN ] ALC: Full history saved. [2025-09-16T17:07:41.226880] [INFO ] [MAIN ] Update Progress: 80/2369 symbols completed. ALC took 27.9s. Estimated remaining: 3h 38m 29s. Rate: 10.5 symbols/min [2025-09-16T17:07:41.851289] [INFO ] [MAIN ] ALG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:07:41.932469] [INFO ] [MAIN ] ALEX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:07:41.949360] [INFO ] [MAIN ] ALIT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:07:41.950263] [INFO ] [MAIN ] ALK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:07:41.963407] [DEBUG] [LOCAL ] ALG: Loaded 7302 existing records from primary storage. [2025-09-16T17:07:41.963584] [INFO ] [MAIN ] ALG: Loaded 7302 existing records. [2025-09-16T17:07:42.123685] [DEBUG] [LOCAL ] ALEX: Loaded 21778 existing records from primary storage. [2025-09-16T17:07:42.181175] [INFO ] [MAIN ] ALEX: Loaded 21778 existing records. [2025-09-16T17:07:42.185563] [WARN ] [MAIN ] ALG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:07:42.187792] [DEBUG] [LOCAL ] ALK: Loaded 39100 existing records from primary storage. [2025-09-16T17:07:42.228539] [INFO ] [MAIN ] ALG: Wiping existing primary data before full refresh. [2025-09-16T17:07:42.228752] [INFO ] [MAIN ] ALK: Loaded 39100 existing records. [2025-09-16T17:07:42.254840] [INFO ] [LOCAL ] ALG: Starting deletion of primary data. [2025-09-16T17:07:42.295521] [DEBUG] [LOCAL ] ALG: Deleted primary data file. [2025-09-16T17:07:42.331682] [INFO ] [MAIN ] ALG: Starting full primary refresh. [2025-09-16T17:07:42.337424] [DEBUG] [LOCAL ] ALIT: Loaded 38839 existing records from primary storage. [2025-09-16T17:07:42.376535] [INFO ] [MAIN ] ALIT: Loaded 38839 existing records. [2025-09-16T17:07:42.704624] [WARN ] [MAIN ] ALEX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:07:42.781811] [INFO ] [MAIN ] ALEX: Wiping existing primary data before full refresh. [2025-09-16T17:07:42.781272] [WARN ] [MAIN ] ALK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:07:42.798172] [INFO ] [LOCAL ] ALEX: Starting deletion of primary data. [2025-09-16T17:07:42.836977] [INFO ] [MAIN ] ALK: Wiping existing primary data before full refresh. [2025-09-16T17:07:42.849050] [DEBUG] [LOCAL ] ALEX: Deleted primary data file. [2025-09-16T17:07:42.876051] [INFO ] [LOCAL ] ALK: Starting deletion of primary data. [2025-09-16T17:07:42.876155] [INFO ] [MAIN ] ALEX: Starting full primary refresh. [2025-09-16T17:07:42.897623] [DEBUG] [LOCAL ] ALK: Deleted primary data file. [2025-09-16T17:07:42.910736] [WARN ] [MAIN ] ALIT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:07:42.971538] [INFO ] [MAIN ] ALK: Starting full primary refresh. [2025-09-16T17:07:43.093153] [INFO ] [MAIN ] ALIT: Wiping existing primary data before full refresh. [2025-09-16T17:07:43.147306] [INFO ] [LOCAL ] ALIT: Starting deletion of primary data. [2025-09-16T17:07:43.209782] [DEBUG] [LOCAL ] ALIT: Deleted primary data file. [2025-09-16T17:07:43.266449] [INFO ] [MAIN ] ALIT: Starting full primary refresh. [2025-09-16T17:07:43.739833] [INFO ] [REMOTE ] ALG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:07:43.772386] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-02. [2025-09-16T17:07:44.064154] [INFO ] [REMOTE ] ALK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:07:44.119326] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-02. [2025-09-16T17:07:44.124716] [INFO ] [REMOTE ] ALEX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:07:44.159360] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-02. [2025-09-16T17:07:44.276178] [INFO ] [REMOTE ] ALIT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:07:44.276510] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-02. [2025-09-16T17:07:44.576446] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-09-16T17:07:44.579322] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-03. [2025-09-16T17:07:45.027683] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T17:07:45.035728] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-03. [2025-09-16T17:07:45.157783] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T17:07:45.161439] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-04. [2025-09-16T17:07:45.368532] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T17:07:45.378350] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-03. [2025-09-16T17:07:45.845708] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T17:07:45.848748] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-05. [2025-09-16T17:07:45.915427] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T17:07:45.925777] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-03. [2025-09-16T17:07:46.387301] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-09-16T17:07:46.390241] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-06. [2025-09-16T17:07:46.473215] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T17:07:46.733332] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T17:07:46.774893] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-04. [2025-09-16T17:07:47.466192] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-09-16T17:07:47.518418] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-09-16T17:07:47.574964] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-07. [2025-09-16T17:07:48.331609] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-09-16T17:07:48.374889] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-04. [2025-09-16T17:07:48.486921] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-09-16T17:07:49.174923] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-08. [2025-09-16T17:07:49.654377] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-09-16T17:07:49.887066] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-09-16T17:07:49.974903] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-05. [2025-09-16T17:07:50.774918] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-04. [2025-09-16T17:07:51.574896] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-05. [2025-09-16T17:07:51.655085] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T17:07:52.385880] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-09. [2025-09-16T17:07:52.431091] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T17:07:52.791098] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T17:07:53.041729] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-09-16T17:07:53.044201] [INFO ] [REMOTE ] ALG: Full fetch complete (11757 records). [2025-09-16T17:07:53.044318] [DEBUG] [VALIDATOR ] ALG: Validating primary data. [2025-09-16T17:07:53.180097] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-06. [2025-09-16T17:07:53.465275] [DEBUG] [VALIDATOR ] ALG: All expected trading days are present. [2025-09-16T17:07:53.465361] [DEBUG] [VALIDATOR ] ALG: Primary data validation passed. [2025-09-16T17:07:53.465613] [INFO ] [MAIN ] ALG: Fetched primary data is valid. [2025-09-16T17:07:53.465677] [DEBUG] [LOCAL ] ALG: Saving primary data. [2025-09-16T17:07:53.466274] [DEBUG] [LOCAL ] ALG: Trimmed 11757 records to 7360 (keeping 150 days of data) [2025-09-16T17:07:53.521119] [DEBUG] [LOCAL ] ALG: Primary data saved (7360 records). [2025-09-16T17:07:53.521270] [INFO ] [MAIN ] ALG: Full history saved. [2025-09-16T17:07:53.521404] [INFO ] [MAIN ] Update Progress: 81/2369 symbols completed. ALG took 12.3s. Estimated remaining: 3h 41m 28s. Rate: 10.3 symbols/min [2025-09-16T17:07:53.698759] [INFO ] [MAIN ] ALL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:07:53.738129] [DEBUG] [LOCAL ] ALL: Loaded 36718 existing records from primary storage. [2025-09-16T17:07:53.738300] [INFO ] [MAIN ] ALL: Loaded 36718 existing records. [2025-09-16T17:07:53.915840] [WARN ] [MAIN ] ALL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:07:53.916076] [INFO ] [MAIN ] ALL: Wiping existing primary data before full refresh. [2025-09-16T17:07:53.916148] [INFO ] [LOCAL ] ALL: Starting deletion of primary data. [2025-09-16T17:07:53.917318] [DEBUG] [LOCAL ] ALL: Deleted primary data file. [2025-09-16T17:07:53.917373] [INFO ] [MAIN ] ALL: Starting full primary refresh. [2025-09-16T17:07:53.984692] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-06. [2025-09-16T17:07:54.264058] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T17:07:54.325610] [INFO ] [REMOTE ] ALL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:07:54.774898] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-02. [2025-09-16T17:07:55.574915] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-05. [2025-09-16T17:07:55.859491] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-09-16T17:07:56.374904] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-07. [2025-09-16T17:07:56.452857] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T17:07:56.830176] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-09-16T17:07:57.174901] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-03. [2025-09-16T17:07:57.515370] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T17:07:57.974905] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-07. [2025-09-16T17:07:58.478853] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T17:07:58.774894] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-06. [2025-09-16T17:07:59.574905] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-08. [2025-09-16T17:07:59.820186] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T17:07:59.850341] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T17:08:00.374900] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-07. [2025-09-16T17:08:01.174912] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-04. [2025-09-16T17:08:01.317392] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-09-16T17:08:01.867498] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T17:08:01.974909] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-08. [2025-09-16T17:08:02.774908] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-09. [2025-09-16T17:08:03.414865] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-04 (2.24 seconds). [2025-09-16T17:08:03.556821] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-09-16T17:08:03.561008] [INFO ] [REMOTE ] ALEX: Full fetch complete (33729 records). [2025-09-16T17:08:03.561098] [DEBUG] [VALIDATOR ] ALEX: Validating primary data. [2025-09-16T17:08:03.582665] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-05. [2025-09-16T17:08:03.946400] [DEBUG] [VALIDATOR ] ALEX: All expected trading days are present. [2025-09-16T17:08:03.946499] [DEBUG] [VALIDATOR ] ALEX: Primary data validation passed. [2025-09-16T17:08:03.946946] [INFO ] [MAIN ] ALEX: Fetched primary data is valid. [2025-09-16T17:08:03.946992] [DEBUG] [LOCAL ] ALEX: Saving primary data. [2025-09-16T17:08:03.948035] [DEBUG] [LOCAL ] ALEX: Trimmed 33729 records to 22072 (keeping 150 days of data) [2025-09-16T17:08:04.117280] [DEBUG] [LOCAL ] ALEX: Primary data saved (22072 records). [2025-09-16T17:08:04.117465] [INFO ] [MAIN ] ALEX: Full history saved. [2025-09-16T17:08:04.118211] [INFO ] [MAIN ] Update Progress: 82/2369 symbols completed. ALEX took 22.9s. Estimated remaining: 3h 43m 36s. Rate: 10.2 symbols/min [2025-09-16T17:08:04.119729] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-09-16T17:08:04.299902] [INFO ] [MAIN ] ALLE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:08:04.340161] [DEBUG] [LOCAL ] ALLE: Loaded 32812 existing records from primary storage. [2025-09-16T17:08:04.340341] [INFO ] [MAIN ] ALLE: Loaded 32812 existing records. [2025-09-16T17:08:04.383214] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-09. [2025-09-16T17:08:04.520789] [WARN ] [MAIN ] ALLE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:08:04.521075] [INFO ] [MAIN ] ALLE: Wiping existing primary data before full refresh. [2025-09-16T17:08:04.521124] [INFO ] [LOCAL ] ALLE: Starting deletion of primary data. [2025-09-16T17:08:04.522102] [DEBUG] [LOCAL ] ALLE: Deleted primary data file. [2025-09-16T17:08:04.522153] [INFO ] [MAIN ] ALLE: Starting full primary refresh. [2025-09-16T17:08:04.875023] [INFO ] [REMOTE ] ALLE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:08:04.951143] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-09-16T17:08:05.174926] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-08. [2025-09-16T17:08:05.974957] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-06. [2025-09-16T17:08:05.995972] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T17:08:06.002306] [INFO ] [REMOTE ] ALIT: Full fetch complete (57360 records). [2025-09-16T17:08:06.002438] [DEBUG] [VALIDATOR ] ALIT: Validating primary data. [2025-09-16T17:08:06.395100] [DEBUG] [VALIDATOR ] ALIT: All expected trading days are present. [2025-09-16T17:08:06.395200] [DEBUG] [VALIDATOR ] ALIT: Primary data validation passed. [2025-09-16T17:08:06.395825] [INFO ] [MAIN ] ALIT: Fetched primary data is valid. [2025-09-16T17:08:06.395875] [DEBUG] [LOCAL ] ALIT: Saving primary data. [2025-09-16T17:08:06.397278] [DEBUG] [LOCAL ] ALIT: Trimmed 57360 records to 39226 (keeping 150 days of data) [2025-09-16T17:08:06.676925] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T17:08:06.692632] [DEBUG] [LOCAL ] ALIT: Primary data saved (39226 records). [2025-09-16T17:08:06.692821] [INFO ] [MAIN ] ALIT: Full history saved. [2025-09-16T17:08:06.693275] [INFO ] [MAIN ] Update Progress: 83/2369 symbols completed. ALIT took 25.5s. Estimated remaining: 3h 41m 60s. Rate: 10.3 symbols/min [2025-09-16T17:08:06.783200] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-02. [2025-09-16T17:08:06.862965] [INFO ] [MAIN ] ALLY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:08:06.904480] [DEBUG] [LOCAL ] ALLY: Loaded 39217 existing records from primary storage. [2025-09-16T17:08:06.904623] [INFO ] [MAIN ] ALLY: Loaded 39217 existing records. [2025-09-16T17:08:07.081410] [WARN ] [MAIN ] ALLY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:08:07.081703] [INFO ] [MAIN ] ALLY: Wiping existing primary data before full refresh. [2025-09-16T17:08:07.081749] [INFO ] [LOCAL ] ALLY: Starting deletion of primary data. [2025-09-16T17:08:07.082674] [DEBUG] [LOCAL ] ALLY: Deleted primary data file. [2025-09-16T17:08:07.082753] [INFO ] [MAIN ] ALLY: Starting full primary refresh. [2025-09-16T17:08:07.436990] [INFO ] [REMOTE ] ALLY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:08:07.574875] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-02. [2025-09-16T17:08:07.634587] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T17:08:08.374913] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-07. [2025-09-16T17:08:08.508069] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-09-16T17:08:08.893043] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-09-16T17:08:09.174924] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-03. [2025-09-16T17:08:09.725200] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T17:08:09.974954] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-08. [2025-09-16T17:08:10.689451] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-09-16T17:08:10.774901] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-04. [2025-09-16T17:08:11.300985] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-09-16T17:08:11.574947] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-09. [2025-09-16T17:08:12.293479] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T17:08:12.374865] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-05. [2025-09-16T17:08:12.789481] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-09-16T17:08:12.795420] [INFO ] [REMOTE ] ALK: Full fetch complete (57265 records). [2025-09-16T17:08:12.795564] [DEBUG] [VALIDATOR ] ALK: Validating primary data. [2025-09-16T17:08:12.796324] [FATAL] [VALIDATOR ] ALK: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:08:12.796407] [FATAL] [MAIN ] ALK: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:08:12.796446] [WARN ] [MAIN ] ALK: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:08:12.796502] [INFO ] [LOCAL ] ALK: Starting deletion of primary data. [2025-09-16T17:08:12.796575] [DEBUG] [LOCAL ] ALK: No primary data file found to delete. [2025-09-16T17:08:12.796614] [INFO ] [LOCAL ] ALK: Starting deletion of derivative data. [2025-09-16T17:08:12.797010] [DEBUG] [LOCAL ] ALK: Deleted derivative data file. [2025-09-16T17:08:12.797110] [INFO ] [MAIN ] Update Progress: 84/2369 symbols completed. ALK took 31.6s. Estimated remaining: 3h 42m 1s. Rate: 10.3 symbols/min [2025-09-16T17:08:12.975905] [INFO ] [MAIN ] ALSN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:08:13.011276] [DEBUG] [LOCAL ] ALSN: Loaded 31418 existing records from primary storage. [2025-09-16T17:08:13.011481] [INFO ] [MAIN ] ALSN: Loaded 31418 existing records. [2025-09-16T17:08:13.181900] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-03. [2025-09-16T17:08:13.196842] [WARN ] [MAIN ] ALSN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:08:13.197181] [INFO ] [MAIN ] ALSN: Wiping existing primary data before full refresh. [2025-09-16T17:08:13.197231] [INFO ] [LOCAL ] ALSN: Starting deletion of primary data. [2025-09-16T17:08:13.198184] [DEBUG] [LOCAL ] ALSN: Deleted primary data file. [2025-09-16T17:08:13.198246] [INFO ] [MAIN ] ALSN: Starting full primary refresh. [2025-09-16T17:08:13.578974] [INFO ] [REMOTE ] ALSN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:08:13.974921] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-09. [2025-09-16T17:08:14.043863] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-09-16T17:08:14.629806] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T17:08:14.774883] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-04. [2025-09-16T17:08:15.448466] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T17:08:15.454473] [INFO ] [REMOTE ] ALL: Full fetch complete (54492 records). [2025-09-16T17:08:15.454585] [DEBUG] [VALIDATOR ] ALL: Validating primary data. [2025-09-16T17:08:15.579996] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-02. [2025-09-16T17:08:15.837366] [DEBUG] [VALIDATOR ] ALL: All expected trading days are present. [2025-09-16T17:08:15.837447] [DEBUG] [VALIDATOR ] ALL: Primary data validation passed. [2025-09-16T17:08:15.838049] [INFO ] [MAIN ] ALL: Fetched primary data is valid. [2025-09-16T17:08:15.838099] [DEBUG] [LOCAL ] ALL: Saving primary data. [2025-09-16T17:08:15.839219] [DEBUG] [LOCAL ] ALL: Trimmed 54492 records to 37086 (keeping 150 days of data) [2025-09-16T17:08:16.077180] [DEBUG] [LOCAL ] ALL: Primary data saved (37086 records). [2025-09-16T17:08:16.077335] [INFO ] [MAIN ] ALL: Full history saved. [2025-09-16T17:08:16.077421] [INFO ] [MAIN ] Update Progress: 85/2369 symbols completed. ALL took 22.6s. Estimated remaining: 3h 40m 47s. Rate: 10.3 symbols/min [2025-09-16T17:08:16.251176] [INFO ] [MAIN ] ALTG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:08:16.251390] [DEBUG] [LOCAL ] ALTG: No existing data file found in primary storage. [2025-09-16T17:08:16.251431] [WARN ] [MAIN ] ALTG: No existing data found, performing full refresh instead. [2025-09-16T17:08:16.251551] [INFO ] [MAIN ] ALTG: Wiping existing primary data before full refresh. [2025-09-16T17:08:16.251591] [INFO ] [LOCAL ] ALTG: Starting deletion of primary data. [2025-09-16T17:08:16.251634] [DEBUG] [LOCAL ] ALTG: No primary data file found to delete. [2025-09-16T17:08:16.251689] [INFO ] [MAIN ] ALTG: Starting full primary refresh. [2025-09-16T17:08:16.395857] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-06. [2025-09-16T17:08:16.612371] [INFO ] [REMOTE ] ALTG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:08:16.721598] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-09-16T17:08:17.162729] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T17:08:17.174852] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-03. [2025-09-16T17:08:17.849841] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T17:08:17.974899] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-07. [2025-09-16T17:08:18.565252] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T17:08:18.774941] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-04. [2025-09-16T17:08:19.442956] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T17:08:19.574875] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-08. [2025-09-16T17:08:20.165839] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T17:08:20.374911] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-05. [2025-09-16T17:08:21.180021] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-02. [2025-09-16T17:08:21.193574] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-09-16T17:08:21.978960] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-09. [2025-09-16T17:08:21.998470] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T17:08:22.025164] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-09-16T17:08:22.774947] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-03. [2025-09-16T17:08:23.243457] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T17:08:23.248697] [INFO ] [REMOTE ] ALLE: Full fetch complete (49481 records). [2025-09-16T17:08:23.248844] [DEBUG] [VALIDATOR ] ALLE: Validating primary data. [2025-09-16T17:08:23.583257] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-06. [2025-09-16T17:08:23.627811] [DEBUG] [VALIDATOR ] ALLE: All expected trading days are present. [2025-09-16T17:08:23.628630] [DEBUG] [VALIDATOR ] ALLE: Primary data validation passed. [2025-09-16T17:08:23.629268] [INFO ] [MAIN ] ALLE: Fetched primary data is valid. [2025-09-16T17:08:23.629512] [DEBUG] [LOCAL ] ALLE: Saving primary data. [2025-09-16T17:08:23.647387] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-09-16T17:08:23.646697] [DEBUG] [LOCAL ] ALLE: Trimmed 49481 records to 33163 (keeping 150 days of data) [2025-09-16T17:08:23.871015] [DEBUG] [LOCAL ] ALLE: Primary data saved (33163 records). [2025-09-16T17:08:23.871165] [INFO ] [MAIN ] ALLE: Full history saved. [2025-09-16T17:08:23.874234] [INFO ] [MAIN ] Update Progress: 86/2369 symbols completed. ALLE took 19.8s. Estimated remaining: 3h 41m 34s. Rate: 10.3 symbols/min [2025-09-16T17:08:24.045985] [INFO ] [MAIN ] ALUR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:08:24.056206] [DEBUG] [LOCAL ] ALUR: Loaded 7335 existing records from primary storage. [2025-09-16T17:08:24.056335] [INFO ] [MAIN ] ALUR: Loaded 7335 existing records. [2025-09-16T17:08:24.235299] [WARN ] [MAIN ] ALUR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:08:24.235554] [INFO ] [MAIN ] ALUR: Wiping existing primary data before full refresh. [2025-09-16T17:08:24.235601] [INFO ] [LOCAL ] ALUR: Starting deletion of primary data. [2025-09-16T17:08:24.235945] [DEBUG] [LOCAL ] ALUR: Deleted primary data file. [2025-09-16T17:08:24.236005] [INFO ] [MAIN ] ALUR: Starting full primary refresh. [2025-09-16T17:08:24.384887] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-04. [2025-09-16T17:08:24.598112] [INFO ] [REMOTE ] ALUR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:08:25.174914] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-05. [2025-09-16T17:08:25.227266] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T17:08:25.698302] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T17:08:25.974892] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-07. [2025-09-16T17:08:26.417021] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-09-16T17:08:26.774911] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-06. [2025-09-16T17:08:27.575021] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-05. [2025-09-16T17:08:27.926367] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-09-16T17:08:28.300071] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-09-16T17:08:28.320806] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-09-16T17:08:28.374930] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-06. [2025-09-16T17:08:29.174910] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-07. [2025-09-16T17:08:29.358728] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T17:08:29.974927] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-08. [2025-09-16T17:08:30.774943] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-07. [2025-09-16T17:08:30.816781] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T17:08:31.405761] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-09-16T17:08:31.530071] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-09-16T17:08:31.574872] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-08. [2025-09-16T17:08:32.374918] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-09. [2025-09-16T17:08:32.702037] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T17:08:33.174920] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-02. [2025-09-16T17:08:33.974878] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-08. [2025-09-16T17:08:34.015254] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T17:08:34.021608] [INFO ] [REMOTE ] ALLY: Full fetch complete (57839 records). [2025-09-16T17:08:34.021738] [DEBUG] [VALIDATOR ] ALLY: Validating primary data. [2025-09-16T17:08:34.414399] [DEBUG] [VALIDATOR ] ALLY: All expected trading days are present. [2025-09-16T17:08:34.414600] [DEBUG] [VALIDATOR ] ALLY: Primary data validation passed. [2025-09-16T17:08:34.415254] [INFO ] [MAIN ] ALLY: Fetched primary data is valid. [2025-09-16T17:08:34.415337] [DEBUG] [LOCAL ] ALLY: Saving primary data. [2025-09-16T17:08:34.429699] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T17:08:34.434699] [DEBUG] [LOCAL ] ALLY: Trimmed 57839 records to 39603 (keeping 150 days of data) [2025-09-16T17:08:34.679763] [DEBUG] [LOCAL ] ALLY: Primary data saved (39603 records). [2025-09-16T17:08:34.679932] [INFO ] [MAIN ] ALLY: Full history saved. [2025-09-16T17:08:34.681476] [INFO ] [MAIN ] Update Progress: 87/2369 symbols completed. ALLY took 28.0s. Estimated remaining: 3h 43m 39s. Rate: 10.2 symbols/min [2025-09-16T17:08:34.780757] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-03. [2025-09-16T17:08:34.858540] [INFO ] [MAIN ] ALV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:08:34.895316] [DEBUG] [LOCAL ] ALV: Loaded 30747 existing records from primary storage. [2025-09-16T17:08:34.895471] [INFO ] [MAIN ] ALV: Loaded 30747 existing records. [2025-09-16T17:08:35.082032] [WARN ] [MAIN ] ALV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:08:35.082299] [INFO ] [MAIN ] ALV: Wiping existing primary data before full refresh. [2025-09-16T17:08:35.082348] [INFO ] [LOCAL ] ALV: Starting deletion of primary data. [2025-09-16T17:08:35.083164] [DEBUG] [LOCAL ] ALV: Deleted primary data file. [2025-09-16T17:08:35.083216] [INFO ] [MAIN ] ALV: Starting full primary refresh. [2025-09-16T17:08:35.446244] [INFO ] [REMOTE ] ALV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:08:35.499886] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T17:08:35.520121] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-09-16T17:08:35.574907] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-09. [2025-09-16T17:08:36.374921] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-09. [2025-09-16T17:08:36.960630] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T17:08:36.966139] [INFO ] [REMOTE ] ALSN: Full fetch complete (47486 records). [2025-09-16T17:08:36.966229] [DEBUG] [VALIDATOR ] ALSN: Validating primary data. [2025-09-16T17:08:37.183226] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-04. [2025-09-16T17:08:37.359773] [DEBUG] [VALIDATOR ] ALSN: All expected trading days are present. [2025-09-16T17:08:37.360628] [DEBUG] [VALIDATOR ] ALSN: Primary data validation passed. [2025-09-16T17:08:37.360991] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T17:08:37.361993] [INFO ] [MAIN ] ALSN: Fetched primary data is valid. [2025-09-16T17:08:37.364969] [DEBUG] [LOCAL ] ALSN: Saving primary data. [2025-09-16T17:08:37.366419] [INFO ] [REMOTE ] ALTG: Full fetch complete (26185 records). [2025-09-16T17:08:37.366742] [DEBUG] [VALIDATOR ] ALTG: Validating primary data. [2025-09-16T17:08:37.411702] [DEBUG] [LOCAL ] ALSN: Trimmed 47486 records to 31728 (keeping 150 days of data) [2025-09-16T17:08:37.868707] [DEBUG] [VALIDATOR ] ALTG: All expected trading days are present. [2025-09-16T17:08:37.869481] [DEBUG] [VALIDATOR ] ALTG: Primary data validation passed. [2025-09-16T17:08:37.871854] [INFO ] [MAIN ] ALTG: Fetched primary data is valid. [2025-09-16T17:08:37.872239] [DEBUG] [LOCAL ] ALTG: Saving primary data. [2025-09-16T17:08:37.881367] [DEBUG] [LOCAL ] ALTG: Trimmed 26185 records to 17512 (keeping 150 days of data) [2025-09-16T17:08:38.005220] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-02. [2025-09-16T17:08:38.054488] [DEBUG] [LOCAL ] ALSN: Primary data saved (31728 records). [2025-09-16T17:08:38.055458] [INFO ] [MAIN ] ALSN: Full history saved. [2025-09-16T17:08:38.055828] [INFO ] [MAIN ] Update Progress: 88/2369 symbols completed. ALSN took 25.3s. Estimated remaining: 3h 42m 28s. Rate: 10.3 symbols/min [2025-09-16T17:08:38.225399] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T17:08:38.253429] [INFO ] [MAIN ] ALX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:08:38.262616] [DEBUG] [LOCAL ] ALTG: Primary data saved (17512 records). [2025-09-16T17:08:38.262878] [INFO ] [MAIN ] ALTG: Full history saved. [2025-09-16T17:08:38.263024] [INFO ] [MAIN ] Update Progress: 89/2369 symbols completed. ALTG took 22.2s. Estimated remaining: 3h 39m 58s. Rate: 10.4 symbols/min [2025-09-16T17:08:38.266979] [DEBUG] [LOCAL ] ALX: Loaded 5174 existing records from primary storage. [2025-09-16T17:08:38.311094] [INFO ] [MAIN ] ALX: Loaded 5174 existing records. [2025-09-16T17:08:38.589994] [INFO ] [MAIN ] AM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:08:38.650071] [WARN ] [MAIN ] ALX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:08:38.650468] [INFO ] [MAIN ] ALX: Wiping existing primary data before full refresh. [2025-09-16T17:08:38.650533] [INFO ] [LOCAL ] ALX: Starting deletion of primary data. [2025-09-16T17:08:38.651050] [DEBUG] [LOCAL ] ALX: Deleted primary data file. [2025-09-16T17:08:38.651126] [INFO ] [MAIN ] ALX: Starting full primary refresh. [2025-09-16T17:08:38.776180] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-05. [2025-09-16T17:08:38.961313] [DEBUG] [LOCAL ] AM: Loaded 38259 existing records from primary storage. [2025-09-16T17:08:38.972116] [INFO ] [MAIN ] AM: Loaded 38259 existing records. [2025-09-16T17:08:39.194380] [INFO ] [REMOTE ] ALX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:08:39.256954] [WARN ] [MAIN ] AM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:08:39.257183] [INFO ] [MAIN ] AM: Wiping existing primary data before full refresh. [2025-09-16T17:08:39.257236] [INFO ] [LOCAL ] AM: Starting deletion of primary data. [2025-09-16T17:08:39.258143] [DEBUG] [LOCAL ] AM: Deleted primary data file. [2025-09-16T17:08:39.258191] [INFO ] [MAIN ] AM: Starting full primary refresh. [2025-09-16T17:08:39.582133] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-02. [2025-09-16T17:08:39.618065] [INFO ] [REMOTE ] AM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:08:39.649087] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T17:08:39.684577] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T17:08:40.133237] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-09-16T17:08:40.374909] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-03. [2025-09-16T17:08:40.877207] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-09-16T17:08:41.174929] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-06. [2025-09-16T17:08:41.948252] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T17:08:41.974849] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-07. [2025-09-16T17:08:42.597847] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-09-16T17:08:42.774913] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-08. [2025-09-16T17:08:43.508579] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-09-16T17:08:43.574905] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-09. [2025-09-16T17:08:44.205848] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-09-16T17:08:44.207128] [INFO ] [REMOTE ] ALUR: Full fetch complete (13072 records). [2025-09-16T17:08:44.207202] [DEBUG] [VALIDATOR ] ALUR: Validating primary data. [2025-09-16T17:08:44.378009] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-02. [2025-09-16T17:08:44.590458] [DEBUG] [VALIDATOR ] ALUR: All expected trading days are present. [2025-09-16T17:08:44.590534] [DEBUG] [VALIDATOR ] ALUR: Primary data validation passed. [2025-09-16T17:08:44.590776] [INFO ] [MAIN ] ALUR: Fetched primary data is valid. [2025-09-16T17:08:44.590819] [DEBUG] [LOCAL ] ALUR: Saving primary data. [2025-09-16T17:08:44.591279] [DEBUG] [LOCAL ] ALUR: Trimmed 13072 records to 7366 (keeping 150 days of data) [2025-09-16T17:08:44.634020] [DEBUG] [LOCAL ] ALUR: Primary data saved (7366 records). [2025-09-16T17:08:44.634124] [INFO ] [MAIN ] ALUR: Full history saved. [2025-09-16T17:08:44.634210] [INFO ] [MAIN ] Update Progress: 90/2369 symbols completed. ALUR took 20.8s. Estimated remaining: 3h 40m 7s. Rate: 10.4 symbols/min [2025-09-16T17:08:44.801284] [INFO ] [MAIN ] AMBC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:08:44.826790] [DEBUG] [LOCAL ] AMBC: Loaded 26441 existing records from primary storage. [2025-09-16T17:08:44.826945] [INFO ] [MAIN ] AMBC: Loaded 26441 existing records. [2025-09-16T17:08:45.003759] [WARN ] [MAIN ] AMBC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:08:45.004041] [INFO ] [MAIN ] AMBC: Wiping existing primary data before full refresh. [2025-09-16T17:08:45.004089] [INFO ] [LOCAL ] AMBC: Starting deletion of primary data. [2025-09-16T17:08:45.004770] [DEBUG] [LOCAL ] AMBC: Deleted primary data file. [2025-09-16T17:08:45.004822] [INFO ] [MAIN ] AMBC: Starting full primary refresh. [2025-09-16T17:08:45.180123] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-04. [2025-09-16T17:08:45.448871] [INFO ] [REMOTE ] AMBC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:08:45.734263] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T17:08:45.974988] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-02. [2025-09-16T17:08:46.239405] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-09-16T17:08:46.774920] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-03. [2025-09-16T17:08:46.996512] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T17:08:47.574910] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-03. [2025-09-16T17:08:48.236189] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T17:08:48.374929] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-04. [2025-09-16T17:08:49.027441] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T17:08:49.174945] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-03. [2025-09-16T17:08:49.974979] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-04. [2025-09-16T17:08:50.147187] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T17:08:50.746092] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-09-16T17:08:50.774883] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-04. [2025-09-16T17:08:51.000234] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T17:08:51.574993] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-05. [2025-09-16T17:08:52.027524] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-09-16T17:08:52.200655] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:08:52.374912] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-05. [2025-09-16T17:08:53.174906] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-05. [2025-09-16T17:08:53.846837] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T17:08:53.974920] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-06. [2025-09-16T17:08:54.628827] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T17:08:54.774910] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-06. [2025-09-16T17:08:54.846486] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-09-16T17:08:55.574903] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-06. [2025-09-16T17:08:56.283493] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-09-16T17:08:56.374897] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-05. [2025-09-16T17:08:57.042380] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-09-16T17:08:57.174879] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-07. [2025-09-16T17:08:57.974905] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-07. [2025-09-16T17:08:58.309622] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-09-16T17:08:58.642226] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T17:08:58.774901] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-08. [2025-09-16T17:08:59.439889] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T17:08:59.574879] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-08. [2025-09-16T17:09:00.374886] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-07. [2025-09-16T17:09:00.476100] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-09-16T17:09:01.174906] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-09. [2025-09-16T17:09:01.222741] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-09-16T17:09:01.411332] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-09-16T17:09:01.974893] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-08. [2025-09-16T17:09:02.687449] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-09-16T17:09:02.774914] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-09. [2025-09-16T17:09:02.856471] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T17:09:02.863039] [INFO ] [REMOTE ] AM: Full fetch complete (56821 records). [2025-09-16T17:09:02.863204] [DEBUG] [VALIDATOR ] AM: Validating primary data. [2025-09-16T17:09:03.259376] [DEBUG] [VALIDATOR ] AM: All expected trading days are present. [2025-09-16T17:09:03.259466] [DEBUG] [VALIDATOR ] AM: Primary data validation passed. [2025-09-16T17:09:03.260113] [INFO ] [MAIN ] AM: Fetched primary data is valid. [2025-09-16T17:09:03.260178] [DEBUG] [LOCAL ] AM: Saving primary data. [2025-09-16T17:09:03.261623] [DEBUG] [LOCAL ] AM: Trimmed 56821 records to 38636 (keeping 150 days of data) [2025-09-16T17:09:03.453804] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-09-16T17:09:03.480112] [INFO ] [REMOTE ] ALX: Full fetch complete (5953 records). [2025-09-16T17:09:03.480939] [DEBUG] [VALIDATOR ] ALX: Validating primary data. [2025-09-16T17:09:03.580018] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-09. [2025-09-16T17:09:03.863396] [DEBUG] [VALIDATOR ] ALX: All expected trading days are present. [2025-09-16T17:09:03.864257] [DEBUG] [VALIDATOR ] ALX: Primary data validation passed. [2025-09-16T17:09:03.865157] [INFO ] [MAIN ] ALX: Fetched primary data is valid. [2025-09-16T17:09:03.865512] [DEBUG] [LOCAL ] ALX: Saving primary data. [2025-09-16T17:09:03.870821] [DEBUG] [LOCAL ] ALX: Trimmed 5953 records to 5247 (keeping 150 days of data) [2025-09-16T17:09:03.904616] [DEBUG] [LOCAL ] AM: Primary data saved (38636 records). [2025-09-16T17:09:03.904990] [INFO ] [MAIN ] AM: Full history saved. [2025-09-16T17:09:03.905095] [INFO ] [MAIN ] Update Progress: 91/2369 symbols completed. AM took 25.6s. Estimated remaining: 3h 45m 38s. Rate: 10.1 symbols/min [2025-09-16T17:09:04.089814] [INFO ] [MAIN ] AMBP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:04.090729] [DEBUG] [LOCAL ] ALX: Primary data saved (5247 records). [2025-09-16T17:09:04.090913] [INFO ] [MAIN ] ALX: Full history saved. [2025-09-16T17:09:04.091265] [INFO ] [MAIN ] Update Progress: 92/2369 symbols completed. ALX took 26.0s. Estimated remaining: 3h 43m 10s. Rate: 10.2 symbols/min [2025-09-16T17:09:04.283018] [INFO ] [MAIN ] AMBQ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:04.283884] [DEBUG] [LOCAL ] AMBQ: No existing data file found in primary storage. [2025-09-16T17:09:04.292343] [WARN ] [MAIN ] AMBQ: No existing data found, performing full refresh instead. [2025-09-16T17:09:04.294189] [DEBUG] [LOCAL ] AMBP: Loaded 33476 existing records from primary storage. [2025-09-16T17:09:04.294260] [INFO ] [MAIN ] AMBQ: Wiping existing primary data before full refresh. [2025-09-16T17:09:04.294396] [INFO ] [MAIN ] AMBP: Loaded 33476 existing records. [2025-09-16T17:09:04.294450] [INFO ] [LOCAL ] AMBQ: Starting deletion of primary data. [2025-09-16T17:09:04.366519] [DEBUG] [LOCAL ] AMBQ: No primary data file found to delete. [2025-09-16T17:09:04.377606] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-06. [2025-09-16T17:09:04.382844] [INFO ] [MAIN ] AMBQ: Starting full primary refresh. [2025-09-16T17:09:04.589289] [WARN ] [MAIN ] AMBP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:09:04.601007] [INFO ] [MAIN ] AMBP: Wiping existing primary data before full refresh. [2025-09-16T17:09:04.616467] [INFO ] [LOCAL ] AMBP: Starting deletion of primary data. [2025-09-16T17:09:04.659831] [DEBUG] [LOCAL ] AMBP: Deleted primary data file. [2025-09-16T17:09:04.752243] [INFO ] [MAIN ] AMBP: Starting full primary refresh. [2025-09-16T17:09:04.981118] [INFO ] [REMOTE ] AMBQ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:05.175941] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-02. [2025-09-16T17:09:05.189330] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T17:09:05.245269] [INFO ] [REMOTE ] ALV: Full fetch complete (46927 records). [2025-09-16T17:09:05.245844] [DEBUG] [VALIDATOR ] ALV: Validating primary data. [2025-09-16T17:09:05.265397] [INFO ] [REMOTE ] AMBP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:05.450729] [DEBUG] [REMOTE ] AMBQ: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:09:05.479113] [WARN ] [REMOTE ] AMBQ: No data returned for month 2025-02 [2025-09-16T17:09:05.665347] [DEBUG] [VALIDATOR ] ALV: All expected trading days are present. [2025-09-16T17:09:05.665445] [DEBUG] [VALIDATOR ] ALV: Primary data validation passed. [2025-09-16T17:09:05.666047] [INFO ] [MAIN ] ALV: Fetched primary data is valid. [2025-09-16T17:09:05.666158] [DEBUG] [LOCAL ] ALV: Saving primary data. [2025-09-16T17:09:05.686663] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T17:09:05.693801] [DEBUG] [LOCAL ] ALV: Trimmed 46927 records to 31077 (keeping 150 days of data) [2025-09-16T17:09:05.896400] [DEBUG] [LOCAL ] ALV: Primary data saved (31077 records). [2025-09-16T17:09:05.896572] [INFO ] [MAIN ] ALV: Full history saved. [2025-09-16T17:09:05.897781] [INFO ] [MAIN ] Update Progress: 93/2369 symbols completed. ALV took 31.2s. Estimated remaining: 3h 41m 24s. Rate: 10.3 symbols/min [2025-09-16T17:09:05.980102] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-03. [2025-09-16T17:09:06.070564] [INFO ] [MAIN ] AMC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:06.111124] [DEBUG] [LOCAL ] AMC: Loaded 39733 existing records from primary storage. [2025-09-16T17:09:06.111285] [INFO ] [MAIN ] AMC: Loaded 39733 existing records. [2025-09-16T17:09:06.268638] [DEBUG] [REMOTE ] AMBQ: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:09:06.287377] [WARN ] [MAIN ] AMC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:09:06.287717] [INFO ] [MAIN ] AMC: Wiping existing primary data before full refresh. [2025-09-16T17:09:06.287844] [INFO ] [LOCAL ] AMC: Starting deletion of primary data. [2025-09-16T17:09:06.287447] [WARN ] [REMOTE ] AMBQ: No data returned for month 2025-03 [2025-09-16T17:09:06.288887] [DEBUG] [LOCAL ] AMC: Deleted primary data file. [2025-09-16T17:09:06.288961] [INFO ] [MAIN ] AMC: Starting full primary refresh. [2025-09-16T17:09:06.641203] [INFO ] [REMOTE ] AMC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:06.774901] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-02. [2025-09-16T17:09:07.574924] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-02. [2025-09-16T17:09:08.374953] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-04. [2025-09-16T17:09:08.610331] [DEBUG] [REMOTE ] AMBQ: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:09:08.610734] [WARN ] [REMOTE ] AMBQ: No data returned for month 2025-04 [2025-09-16T17:09:09.120821] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-09-16T17:09:09.159077] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T17:09:09.174844] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-03. [2025-09-16T17:09:09.974925] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-03. [2025-09-16T17:09:10.664512] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-09-16T17:09:10.774925] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-04. [2025-09-16T17:09:11.574939] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-05. [2025-09-16T17:09:12.086270] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T17:09:12.096363] [DEBUG] [REMOTE ] AMBQ: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:09:12.097229] [WARN ] [REMOTE ] AMBQ: No data returned for month 2025-05 [2025-09-16T17:09:12.143250] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-03 (2.17 seconds). [2025-09-16T17:09:12.374951] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-07. [2025-09-16T17:09:13.174911] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-05. [2025-09-16T17:09:13.881222] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T17:09:13.974917] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-08. [2025-09-16T17:09:14.658524] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T17:09:14.774909] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-04. [2025-09-16T17:09:14.910075] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-09-16T17:09:15.574923] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-09. [2025-09-16T17:09:16.374913] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-06. [2025-09-16T17:09:16.431745] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T17:09:16.435469] [INFO ] [REMOTE ] AMBC: Full fetch complete (40401 records). [2025-09-16T17:09:16.435571] [DEBUG] [VALIDATOR ] AMBC: Validating primary data. [2025-09-16T17:09:16.436193] [FATAL] [VALIDATOR ] AMBC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:09:16.436275] [FATAL] [MAIN ] AMBC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:09:16.436313] [WARN ] [MAIN ] AMBC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:09:16.436351] [INFO ] [LOCAL ] AMBC: Starting deletion of primary data. [2025-09-16T17:09:16.436409] [DEBUG] [LOCAL ] AMBC: No primary data file found to delete. [2025-09-16T17:09:16.436467] [INFO ] [LOCAL ] AMBC: Starting deletion of derivative data. [2025-09-16T17:09:16.436769] [DEBUG] [LOCAL ] AMBC: Deleted derivative data file. [2025-09-16T17:09:16.436853] [INFO ] [MAIN ] Update Progress: 94/2369 symbols completed. AMBC took 31.8s. Estimated remaining: 3h 43m 12s. Rate: 10.2 symbols/min [2025-09-16T17:09:16.615198] [INFO ] [MAIN ] AMCR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:16.655438] [DEBUG] [LOCAL ] AMCR: Loaded 39632 existing records from primary storage. [2025-09-16T17:09:16.655584] [INFO ] [MAIN ] AMCR: Loaded 39632 existing records. [2025-09-16T17:09:16.841074] [WARN ] [MAIN ] AMCR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:09:16.841316] [INFO ] [MAIN ] AMCR: Wiping existing primary data before full refresh. [2025-09-16T17:09:16.841363] [INFO ] [LOCAL ] AMCR: Starting deletion of primary data. [2025-09-16T17:09:16.842344] [DEBUG] [LOCAL ] AMCR: Deleted primary data file. [2025-09-16T17:09:16.842404] [INFO ] [MAIN ] AMCR: Starting full primary refresh. [2025-09-16T17:09:17.185283] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-06. [2025-09-16T17:09:17.212277] [INFO ] [REMOTE ] AMCR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:17.246517] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-04 (2.47 seconds). [2025-09-16T17:09:17.405711] [DEBUG] [REMOTE ] AMBQ: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:09:17.406064] [WARN ] [REMOTE ] AMBQ: No data returned for month 2025-06 [2025-09-16T17:09:17.473369] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-09-16T17:09:17.974913] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-02. [2025-09-16T17:09:18.774916] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-05. [2025-09-16T17:09:19.574932] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-07. [2025-09-16T17:09:19.658417] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T17:09:20.132272] [INFO ] [REMOTE ] AMBQ: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-09-16T17:09:20.374906] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-07. [2025-09-16T17:09:21.094634] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-05 (2.32 seconds). [2025-09-16T17:09:21.174894] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-06. [2025-09-16T17:09:21.856943] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T17:09:21.974948] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-08. [2025-09-16T17:09:22.774930] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-08. [2025-09-16T17:09:23.026232] [INFO ] [REMOTE ] AMBQ: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T17:09:23.217006] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-06 (2.04 seconds). [2025-09-16T17:09:23.574977] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-07. [2025-09-16T17:09:24.159746] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T17:09:24.374926] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-03. [2025-09-16T17:09:25.174922] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-09. [2025-09-16T17:09:25.592107] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-09-16T17:09:25.974914] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-09. [2025-09-16T17:09:26.066166] [INFO ] [REMOTE ] AMBQ: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T17:09:26.069486] [INFO ] [REMOTE ] AMBQ: Full fetch complete (6990 records). [2025-09-16T17:09:26.069555] [DEBUG] [VALIDATOR ] AMBQ: Validating primary data. [2025-09-16T17:09:26.248757] [WARN ] [VALIDATOR ] AMBQ: Validation failed - insufficient historical data. Have 34 unique days, need 150 days [2025-09-16T17:09:26.248988] [FATAL] [MAIN ] AMBQ: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:09:26.249032] [WARN ] [MAIN ] AMBQ: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:09:26.249073] [INFO ] [LOCAL ] AMBQ: Starting deletion of primary data. [2025-09-16T17:09:26.249131] [DEBUG] [LOCAL ] AMBQ: No primary data file found to delete. [2025-09-16T17:09:26.249169] [INFO ] [LOCAL ] AMBQ: Starting deletion of derivative data. [2025-09-16T17:09:26.249213] [DEBUG] [LOCAL ] AMBQ: No derivative data file found to delete. [2025-09-16T17:09:26.249275] [INFO ] [MAIN ] Update Progress: 95/2369 symbols completed. AMBQ took 22.2s. Estimated remaining: 3h 44m 40s. Rate: 10.1 symbols/min [2025-09-16T17:09:26.428252] [INFO ] [MAIN ] AME: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:26.464612] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-09-16T17:09:26.505145] [DEBUG] [LOCAL ] AME: Loaded 35658 existing records from primary storage. [2025-09-16T17:09:26.505308] [INFO ] [MAIN ] AME: Loaded 35658 existing records. [2025-09-16T17:09:26.679694] [WARN ] [MAIN ] AME: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:09:26.679921] [INFO ] [MAIN ] AME: Wiping existing primary data before full refresh. [2025-09-16T17:09:26.679966] [INFO ] [LOCAL ] AME: Starting deletion of primary data. [2025-09-16T17:09:26.681256] [DEBUG] [LOCAL ] AME: Deleted primary data file. [2025-09-16T17:09:26.681307] [INFO ] [MAIN ] AME: Starting full primary refresh. [2025-09-16T17:09:26.779980] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-04. [2025-09-16T17:09:27.031868] [INFO ] [REMOTE ] AME: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:27.419856] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T17:09:27.424981] [INFO ] [REMOTE ] AMBP: Full fetch complete (49425 records). [2025-09-16T17:09:27.425100] [DEBUG] [VALIDATOR ] AMBP: Validating primary data. [2025-09-16T17:09:27.583714] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-02. [2025-09-16T17:09:27.825982] [DEBUG] [VALIDATOR ] AMBP: All expected trading days are present. [2025-09-16T17:09:27.826068] [DEBUG] [VALIDATOR ] AMBP: Primary data validation passed. [2025-09-16T17:09:27.826591] [INFO ] [MAIN ] AMBP: Fetched primary data is valid. [2025-09-16T17:09:27.826636] [DEBUG] [LOCAL ] AMBP: Saving primary data. [2025-09-16T17:09:27.827721] [DEBUG] [LOCAL ] AMBP: Trimmed 49425 records to 33802 (keeping 150 days of data) [2025-09-16T17:09:28.022287] [DEBUG] [LOCAL ] AMBP: Primary data saved (33802 records). [2025-09-16T17:09:28.022437] [INFO ] [MAIN ] AMBP: Full history saved. [2025-09-16T17:09:28.022523] [INFO ] [MAIN ] Update Progress: 96/2369 symbols completed. AMBP took 24.1s. Estimated remaining: 3h 42m 56s. Rate: 10.2 symbols/min [2025-09-16T17:09:28.197261] [INFO ] [MAIN ] AMG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:28.214296] [DEBUG] [LOCAL ] AMG: Loaded 14410 existing records from primary storage. [2025-09-16T17:09:28.214431] [INFO ] [MAIN ] AMG: Loaded 14410 existing records. [2025-09-16T17:09:28.380898] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-08. [2025-09-16T17:09:28.402346] [WARN ] [MAIN ] AMG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:09:28.403423] [INFO ] [MAIN ] AMG: Wiping existing primary data before full refresh. [2025-09-16T17:09:28.403485] [INFO ] [LOCAL ] AMG: Starting deletion of primary data. [2025-09-16T17:09:28.404251] [DEBUG] [LOCAL ] AMG: Deleted primary data file. [2025-09-16T17:09:28.404324] [INFO ] [MAIN ] AMG: Starting full primary refresh. [2025-09-16T17:09:28.780578] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-09-16T17:09:28.821909] [INFO ] [REMOTE ] AMG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:29.058745] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T17:09:29.174966] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-05. [2025-09-16T17:09:29.974919] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-03. [2025-09-16T17:09:30.342347] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-09-16T17:09:30.774893] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-09. [2025-09-16T17:09:30.836156] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T17:09:31.416321] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T17:09:31.574883] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-04. [2025-09-16T17:09:32.374942] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-02. [2025-09-16T17:09:33.174875] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-06. [2025-09-16T17:09:33.200207] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-09 (2.43 seconds). [2025-09-16T17:09:33.206877] [INFO ] [REMOTE ] AMC: Full fetch complete (58433 records). [2025-09-16T17:09:33.206989] [DEBUG] [VALIDATOR ] AMC: Validating primary data. [2025-09-16T17:09:33.595933] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T17:09:33.668925] [DEBUG] [VALIDATOR ] AMC: All expected trading days are present. [2025-09-16T17:09:33.669318] [DEBUG] [VALIDATOR ] AMC: Primary data validation passed. [2025-09-16T17:09:33.670372] [INFO ] [MAIN ] AMC: Fetched primary data is valid. [2025-09-16T17:09:33.671661] [DEBUG] [LOCAL ] AMC: Saving primary data. [2025-09-16T17:09:33.671557] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-09-16T17:09:33.683544] [DEBUG] [LOCAL ] AMC: Trimmed 58433 records to 40123 (keeping 150 days of data) [2025-09-16T17:09:33.921465] [DEBUG] [LOCAL ] AMC: Primary data saved (40123 records). [2025-09-16T17:09:33.921599] [INFO ] [MAIN ] AMC: Full history saved. [2025-09-16T17:09:33.922601] [INFO ] [MAIN ] Update Progress: 97/2369 symbols completed. AMC took 28.0s. Estimated remaining: 3h 42m 50s. Rate: 10.2 symbols/min [2025-09-16T17:09:33.982168] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-03. [2025-09-16T17:09:34.104630] [INFO ] [MAIN ] AMH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:34.147604] [DEBUG] [LOCAL ] AMH: Loaded 38075 existing records from primary storage. [2025-09-16T17:09:34.147772] [INFO ] [MAIN ] AMH: Loaded 38075 existing records. [2025-09-16T17:09:34.325792] [WARN ] [MAIN ] AMH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:09:34.326077] [INFO ] [MAIN ] AMH: Wiping existing primary data before full refresh. [2025-09-16T17:09:34.326136] [INFO ] [LOCAL ] AMH: Starting deletion of primary data. [2025-09-16T17:09:34.327083] [DEBUG] [LOCAL ] AMH: Deleted primary data file. [2025-09-16T17:09:34.327151] [INFO ] [MAIN ] AMH: Starting full primary refresh. [2025-09-16T17:09:34.689813] [INFO ] [REMOTE ] AMH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:34.746027] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T17:09:34.774870] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-02. [2025-09-16T17:09:35.231974] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-09-16T17:09:35.574882] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-07. [2025-09-16T17:09:36.147623] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T17:09:36.374917] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-05. [2025-09-16T17:09:37.174940] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-03. [2025-09-16T17:09:37.647328] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T17:09:37.701387] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-09-16T17:09:37.974917] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-04. [2025-09-16T17:09:38.769501] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-09-16T17:09:38.779755] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-06. [2025-09-16T17:09:39.179094] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-09-16T17:09:39.574915] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-04. [2025-09-16T17:09:40.227572] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T17:09:40.374857] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-07. [2025-09-16T17:09:41.174910] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-08. [2025-09-16T17:09:41.337686] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T17:09:41.768822] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-09-16T17:09:41.974902] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-08. [2025-09-16T17:09:42.774923] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-05. [2025-09-16T17:09:43.164654] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-09-16T17:09:43.574907] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-09. [2025-09-16T17:09:43.613927] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T17:09:44.149383] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-09-16T17:09:44.374902] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-06. [2025-09-16T17:09:45.174895] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-05. [2025-09-16T17:09:45.494020] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-09-16T17:09:45.500813] [INFO ] [REMOTE ] AMCR: Full fetch complete (58347 records). [2025-09-16T17:09:45.500947] [DEBUG] [VALIDATOR ] AMCR: Validating primary data. [2025-09-16T17:09:45.884349] [DEBUG] [VALIDATOR ] AMCR: All expected trading days are present. [2025-09-16T17:09:45.884546] [DEBUG] [VALIDATOR ] AMCR: Primary data validation passed. [2025-09-16T17:09:45.885165] [INFO ] [MAIN ] AMCR: Fetched primary data is valid. [2025-09-16T17:09:45.885241] [DEBUG] [LOCAL ] AMCR: Saving primary data. [2025-09-16T17:09:45.887876] [DEBUG] [LOCAL ] AMCR: Trimmed 58347 records to 40022 (keeping 150 days of data) [2025-09-16T17:09:45.983993] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-09. [2025-09-16T17:09:46.036403] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T17:09:46.191086] [DEBUG] [LOCAL ] AMCR: Primary data saved (40022 records). [2025-09-16T17:09:46.191242] [INFO ] [MAIN ] AMCR: Full history saved. [2025-09-16T17:09:46.192323] [INFO ] [MAIN ] Update Progress: 98/2369 symbols completed. AMCR took 29.8s. Estimated remaining: 3h 45m 13s. Rate: 10.1 symbols/min [2025-09-16T17:09:46.364038] [INFO ] [MAIN ] AMN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:46.383010] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-09-16T17:09:46.422409] [DEBUG] [LOCAL ] AMN: Loaded 31022 existing records from primary storage. [2025-09-16T17:09:46.422578] [INFO ] [MAIN ] AMN: Loaded 31022 existing records. [2025-09-16T17:09:46.599389] [WARN ] [MAIN ] AMN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:09:46.599686] [INFO ] [MAIN ] AMN: Wiping existing primary data before full refresh. [2025-09-16T17:09:46.599736] [INFO ] [LOCAL ] AMN: Starting deletion of primary data. [2025-09-16T17:09:46.600520] [DEBUG] [LOCAL ] AMN: Deleted primary data file. [2025-09-16T17:09:46.600585] [INFO ] [MAIN ] AMN: Starting full primary refresh. [2025-09-16T17:09:46.780156] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-06. [2025-09-16T17:09:46.962309] [INFO ] [REMOTE ] AMN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:47.574918] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-07. [2025-09-16T17:09:47.723986] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-09-16T17:09:47.730068] [INFO ] [REMOTE ] AME: Full fetch complete (52766 records). [2025-09-16T17:09:47.730164] [DEBUG] [VALIDATOR ] AME: Validating primary data. [2025-09-16T17:09:48.051219] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-09-16T17:09:48.141329] [DEBUG] [VALIDATOR ] AME: All expected trading days are present. [2025-09-16T17:09:48.141408] [DEBUG] [VALIDATOR ] AME: Primary data validation passed. [2025-09-16T17:09:48.142035] [INFO ] [MAIN ] AME: Fetched primary data is valid. [2025-09-16T17:09:48.142083] [DEBUG] [LOCAL ] AME: Saving primary data. [2025-09-16T17:09:48.143294] [DEBUG] [LOCAL ] AME: Trimmed 52766 records to 36016 (keeping 150 days of data) [2025-09-16T17:09:48.374940] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-07. [2025-09-16T17:09:48.378733] [DEBUG] [LOCAL ] AME: Primary data saved (36016 records). [2025-09-16T17:09:48.378852] [INFO ] [MAIN ] AME: Full history saved. [2025-09-16T17:09:48.378959] [INFO ] [MAIN ] Update Progress: 99/2369 symbols completed. AME took 22.1s. Estimated remaining: 3h 43m 40s. Rate: 10.1 symbols/min [2025-09-16T17:09:48.548524] [INFO ] [MAIN ] AMP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:48.572429] [DEBUG] [LOCAL ] AMP: Loaded 23065 existing records from primary storage. [2025-09-16T17:09:48.572560] [INFO ] [MAIN ] AMP: Loaded 23065 existing records. [2025-09-16T17:09:48.751141] [WARN ] [MAIN ] AMP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:09:48.751405] [INFO ] [MAIN ] AMP: Wiping existing primary data before full refresh. [2025-09-16T17:09:48.751489] [INFO ] [LOCAL ] AMP: Starting deletion of primary data. [2025-09-16T17:09:48.752218] [DEBUG] [LOCAL ] AMP: Deleted primary data file. [2025-09-16T17:09:48.752266] [INFO ] [MAIN ] AMP: Starting full primary refresh. [2025-09-16T17:09:49.112204] [INFO ] [REMOTE ] AMP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:49.180749] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-02. [2025-09-16T17:09:49.194093] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-09-16T17:09:49.511022] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T17:09:49.974903] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-02. [2025-09-16T17:09:50.767448] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T17:09:50.774833] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-03. [2025-09-16T17:09:51.258134] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-09-16T17:09:51.574910] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-08. [2025-09-16T17:09:52.223202] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T17:09:52.374904] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-04. [2025-09-16T17:09:52.558705] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-09-16T17:09:53.174911] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-09. [2025-09-16T17:09:53.839613] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T17:09:53.974896] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-05. [2025-09-16T17:09:54.078096] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-09-16T17:09:54.080998] [INFO ] [REMOTE ] AMG: Full fetch complete (22495 records). [2025-09-16T17:09:54.081084] [DEBUG] [VALIDATOR ] AMG: Validating primary data. [2025-09-16T17:09:54.454229] [DEBUG] [VALIDATOR ] AMG: All expected trading days are present. [2025-09-16T17:09:54.454320] [DEBUG] [VALIDATOR ] AMG: Primary data validation passed. [2025-09-16T17:09:54.454622] [INFO ] [MAIN ] AMG: Fetched primary data is valid. [2025-09-16T17:09:54.454682] [DEBUG] [LOCAL ] AMG: Saving primary data. [2025-09-16T17:09:54.455450] [DEBUG] [LOCAL ] AMG: Trimmed 22495 records to 14556 (keeping 150 days of data) [2025-09-16T17:09:54.552316] [DEBUG] [LOCAL ] AMG: Primary data saved (14556 records). [2025-09-16T17:09:54.552463] [INFO ] [MAIN ] AMG: Full history saved. [2025-09-16T17:09:54.552561] [INFO ] [MAIN ] Update Progress: 100/2369 symbols completed. AMG took 26.5s. Estimated remaining: 3h 43m 40s. Rate: 10.1 symbols/min [2025-09-16T17:09:54.722936] [INFO ] [MAIN ] AMPX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:09:54.766519] [DEBUG] [LOCAL ] AMPX: Loaded 38648 existing records from primary storage. [2025-09-16T17:09:54.766689] [INFO ] [MAIN ] AMPX: Loaded 38648 existing records. [2025-09-16T17:09:54.780032] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-03. [2025-09-16T17:09:54.949156] [WARN ] [MAIN ] AMPX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:09:54.949376] [INFO ] [MAIN ] AMPX: Wiping existing primary data before full refresh. [2025-09-16T17:09:54.949422] [INFO ] [LOCAL ] AMPX: Starting deletion of primary data. [2025-09-16T17:09:54.950360] [DEBUG] [LOCAL ] AMPX: Deleted primary data file. [2025-09-16T17:09:54.950411] [INFO ] [MAIN ] AMPX: Starting full primary refresh. [2025-09-16T17:09:55.318220] [INFO ] [REMOTE ] AMPX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:09:55.338520] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-09-16T17:09:55.574897] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-02. [2025-09-16T17:09:55.833186] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-09-16T17:09:56.374891] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-04. [2025-09-16T17:09:56.952884] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T17:09:57.174900] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-08. [2025-09-16T17:09:57.765613] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T17:09:57.974902] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-05. [2025-09-16T17:09:58.417676] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T17:09:58.774941] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-06. [2025-09-16T17:09:59.316025] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T17:09:59.574978] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-06. [2025-09-16T17:09:59.755218] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T17:10:00.374899] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-07. [2025-09-16T17:10:01.045739] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-09-16T17:10:01.174893] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-07. [2025-09-16T17:10:01.974906] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-03. [2025-09-16T17:10:02.719754] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-07 (2.34 seconds). [2025-09-16T17:10:02.774867] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-08. [2025-09-16T17:10:03.395374] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-07 (2.22 seconds). [2025-09-16T17:10:03.574918] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-08. [2025-09-16T17:10:03.729941] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T17:10:03.874411] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-09-16T17:10:04.374909] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-04. [2025-09-16T17:10:05.020517] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T17:10:05.174892] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-09. [2025-09-16T17:10:05.974902] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-09. [2025-09-16T17:10:06.307553] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-09-16T17:10:06.416501] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T17:10:06.421821] [INFO ] [REMOTE ] AMN: Full fetch complete (46344 records). [2025-09-16T17:10:06.421965] [DEBUG] [VALIDATOR ] AMN: Validating primary data. [2025-09-16T17:10:06.778267] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-05. [2025-09-16T17:10:06.810738] [DEBUG] [VALIDATOR ] AMN: All expected trading days are present. [2025-09-16T17:10:06.810810] [DEBUG] [VALIDATOR ] AMN: Primary data validation passed. [2025-09-16T17:10:06.811343] [INFO ] [MAIN ] AMN: Fetched primary data is valid. [2025-09-16T17:10:06.811386] [DEBUG] [LOCAL ] AMN: Saving primary data. [2025-09-16T17:10:06.812718] [DEBUG] [LOCAL ] AMN: Trimmed 46344 records to 31354 (keeping 150 days of data) [2025-09-16T17:10:07.003056] [DEBUG] [LOCAL ] AMN: Primary data saved (31354 records). [2025-09-16T17:10:07.003205] [INFO ] [MAIN ] AMN: Full history saved. [2025-09-16T17:10:07.006342] [INFO ] [MAIN ] Update Progress: 101/2369 symbols completed. AMN took 20.8s. Estimated remaining: 3h 46m 1s. Rate: 10.0 symbols/min [2025-09-16T17:10:07.177482] [INFO ] [MAIN ] AMPY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:07.207820] [DEBUG] [LOCAL ] AMPY: Loaded 27646 existing records from primary storage. [2025-09-16T17:10:07.207995] [INFO ] [MAIN ] AMPY: Loaded 27646 existing records. [2025-09-16T17:10:07.391797] [WARN ] [MAIN ] AMPY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:10:07.392181] [INFO ] [MAIN ] AMPY: Wiping existing primary data before full refresh. [2025-09-16T17:10:07.392278] [INFO ] [LOCAL ] AMPY: Starting deletion of primary data. [2025-09-16T17:10:07.402429] [DEBUG] [LOCAL ] AMPY: Deleted primary data file. [2025-09-16T17:10:07.415262] [INFO ] [MAIN ] AMPY: Starting full primary refresh. [2025-09-16T17:10:07.496251] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T17:10:07.531734] [INFO ] [REMOTE ] AMH: Full fetch complete (56207 records). [2025-09-16T17:10:07.557999] [DEBUG] [VALIDATOR ] AMH: Validating primary data. [2025-09-16T17:10:07.581394] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-09. [2025-09-16T17:10:07.965874] [INFO ] [REMOTE ] AMPY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:10:08.186451] [DEBUG] [VALIDATOR ] AMH: All expected trading days are present. [2025-09-16T17:10:08.186534] [DEBUG] [VALIDATOR ] AMH: Primary data validation passed. [2025-09-16T17:10:08.187154] [INFO ] [MAIN ] AMH: Fetched primary data is valid. [2025-09-16T17:10:08.187202] [DEBUG] [LOCAL ] AMH: Saving primary data. [2025-09-16T17:10:08.188390] [DEBUG] [LOCAL ] AMH: Trimmed 56207 records to 38458 (keeping 150 days of data) [2025-09-16T17:10:08.398382] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-02. [2025-09-16T17:10:08.431964] [DEBUG] [LOCAL ] AMH: Primary data saved (38458 records). [2025-09-16T17:10:08.432093] [INFO ] [MAIN ] AMH: Full history saved. [2025-09-16T17:10:08.434736] [INFO ] [MAIN ] Update Progress: 102/2369 symbols completed. AMH took 34.5s. Estimated remaining: 3h 44m 14s. Rate: 10.1 symbols/min [2025-09-16T17:10:08.612825] [INFO ] [MAIN ] AMR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:08.635829] [DEBUG] [LOCAL ] AMR: Loaded 21277 existing records from primary storage. [2025-09-16T17:10:08.635969] [INFO ] [MAIN ] AMR: Loaded 21277 existing records. [2025-09-16T17:10:08.826294] [WARN ] [MAIN ] AMR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:10:08.826751] [INFO ] [MAIN ] AMR: Wiping existing primary data before full refresh. [2025-09-16T17:10:08.826821] [INFO ] [LOCAL ] AMR: Starting deletion of primary data. [2025-09-16T17:10:08.827679] [DEBUG] [LOCAL ] AMR: Deleted primary data file. [2025-09-16T17:10:08.843164] [INFO ] [MAIN ] AMR: Starting full primary refresh. [2025-09-16T17:10:08.913175] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-09-16T17:10:09.179613] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-06. [2025-09-16T17:10:09.232917] [INFO ] [REMOTE ] AMR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:10:09.245178] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-09-16T17:10:09.249563] [INFO ] [REMOTE ] AMP: Full fetch complete (35692 records). [2025-09-16T17:10:09.249680] [DEBUG] [VALIDATOR ] AMP: Validating primary data. [2025-09-16T17:10:09.650634] [DEBUG] [VALIDATOR ] AMP: All expected trading days are present. [2025-09-16T17:10:09.651810] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T17:10:09.651212] [DEBUG] [VALIDATOR ] AMP: Primary data validation passed. [2025-09-16T17:10:09.657898] [INFO ] [MAIN ] AMP: Fetched primary data is valid. [2025-09-16T17:10:09.657969] [DEBUG] [LOCAL ] AMP: Saving primary data. [2025-09-16T17:10:09.658777] [DEBUG] [LOCAL ] AMP: Trimmed 35692 records to 23302 (keeping 150 days of data) [2025-09-16T17:10:09.810598] [DEBUG] [LOCAL ] AMP: Primary data saved (23302 records). [2025-09-16T17:10:09.810755] [INFO ] [MAIN ] AMP: Full history saved. [2025-09-16T17:10:09.810861] [INFO ] [MAIN ] Update Progress: 103/2369 symbols completed. AMP took 21.4s. Estimated remaining: 3h 42m 28s. Rate: 10.2 symbols/min [2025-09-16T17:10:09.983338] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-03. [2025-09-16T17:10:09.986693] [INFO ] [MAIN ] AMRC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:10.015978] [DEBUG] [LOCAL ] AMRC: Loaded 27480 existing records from primary storage. [2025-09-16T17:10:10.016124] [INFO ] [MAIN ] AMRC: Loaded 27480 existing records. [2025-09-16T17:10:10.199717] [WARN ] [MAIN ] AMRC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:10:10.199970] [INFO ] [MAIN ] AMRC: Wiping existing primary data before full refresh. [2025-09-16T17:10:10.200016] [INFO ] [LOCAL ] AMRC: Starting deletion of primary data. [2025-09-16T17:10:10.200953] [DEBUG] [LOCAL ] AMRC: Deleted primary data file. [2025-09-16T17:10:10.201005] [INFO ] [MAIN ] AMRC: Starting full primary refresh. [2025-09-16T17:10:10.567320] [INFO ] [REMOTE ] AMRC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:10:10.774910] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-02. [2025-09-16T17:10:10.983693] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-09-16T17:10:11.164928] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-09-16T17:10:11.574917] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-07. [2025-09-16T17:10:12.374902] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-04. [2025-09-16T17:10:12.502116] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T17:10:13.174959] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-02. [2025-09-16T17:10:13.824075] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T17:10:13.974948] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-05. [2025-09-16T17:10:14.079263] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-07 (2.50 seconds). [2025-09-16T17:10:14.533253] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T17:10:14.774901] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-03. [2025-09-16T17:10:15.097253] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T17:10:15.574908] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-08. [2025-09-16T17:10:15.926929] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T17:10:16.374894] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-03. [2025-09-16T17:10:17.174919] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-06. [2025-09-16T17:10:17.896627] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-08 (2.32 seconds). [2025-09-16T17:10:17.938495] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T17:10:17.974929] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-04. [2025-09-16T17:10:18.362334] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-09-16T17:10:18.774994] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-07. [2025-09-16T17:10:19.132900] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-09-16T17:10:19.574974] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-09. [2025-09-16T17:10:19.947216] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T17:10:20.374989] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-08. [2025-09-16T17:10:21.174943] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-04. [2025-09-16T17:10:21.438795] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-09-16T17:10:21.540365] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-09-16T17:10:21.549341] [INFO ] [REMOTE ] AMPX: Full fetch complete (56252 records). [2025-09-16T17:10:21.549671] [DEBUG] [VALIDATOR ] AMPX: Validating primary data. [2025-09-16T17:10:21.963952] [DEBUG] [VALIDATOR ] AMPX: All expected trading days are present. [2025-09-16T17:10:21.964044] [DEBUG] [VALIDATOR ] AMPX: Primary data validation passed. [2025-09-16T17:10:21.964617] [INFO ] [MAIN ] AMPX: Fetched primary data is valid. [2025-09-16T17:10:21.964692] [DEBUG] [LOCAL ] AMPX: Saving primary data. [2025-09-16T17:10:21.965899] [DEBUG] [LOCAL ] AMPX: Trimmed 56252 records to 39038 (keeping 150 days of data) [2025-09-16T17:10:22.017583] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-09. [2025-09-16T17:10:22.207524] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T17:10:22.229530] [DEBUG] [LOCAL ] AMPX: Primary data saved (39038 records). [2025-09-16T17:10:22.229720] [INFO ] [MAIN ] AMPX: Full history saved. [2025-09-16T17:10:22.229815] [INFO ] [MAIN ] Update Progress: 104/2369 symbols completed. AMPX took 27.7s. Estimated remaining: 3h 44m 44s. Rate: 10.1 symbols/min [2025-09-16T17:10:22.398397] [INFO ] [MAIN ] AMRZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:22.398592] [DEBUG] [LOCAL ] AMRZ: No existing data file found in primary storage. [2025-09-16T17:10:22.398632] [WARN ] [MAIN ] AMRZ: No existing data found, performing full refresh instead. [2025-09-16T17:10:22.398774] [INFO ] [MAIN ] AMRZ: Wiping existing primary data before full refresh. [2025-09-16T17:10:22.398814] [INFO ] [LOCAL ] AMRZ: Starting deletion of primary data. [2025-09-16T17:10:22.398855] [DEBUG] [LOCAL ] AMRZ: No primary data file found to delete. [2025-09-16T17:10:22.398889] [INFO ] [MAIN ] AMRZ: Starting full primary refresh. [2025-09-16T17:10:22.742048] [INFO ] [REMOTE ] AMRZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:10:22.774844] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-02. [2025-09-16T17:10:22.973803] [DEBUG] [REMOTE ] AMRZ: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:10:22.974274] [WARN ] [REMOTE ] AMRZ: No data returned for month 2025-02 [2025-09-16T17:10:23.329625] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T17:10:23.333889] [INFO ] [REMOTE ] AMPY: Full fetch complete (39702 records). [2025-09-16T17:10:23.334032] [DEBUG] [VALIDATOR ] AMPY: Validating primary data. [2025-09-16T17:10:23.580138] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-03. [2025-09-16T17:10:23.761133] [DEBUG] [VALIDATOR ] AMPY: All expected trading days are present. [2025-09-16T17:10:23.761226] [DEBUG] [VALIDATOR ] AMPY: Primary data validation passed. [2025-09-16T17:10:23.761694] [INFO ] [MAIN ] AMPY: Fetched primary data is valid. [2025-09-16T17:10:23.761742] [DEBUG] [LOCAL ] AMPY: Saving primary data. [2025-09-16T17:10:23.762740] [DEBUG] [LOCAL ] AMPY: Trimmed 39702 records to 27915 (keeping 150 days of data) [2025-09-16T17:10:23.864406] [DEBUG] [REMOTE ] AMRZ: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:10:23.867597] [WARN ] [REMOTE ] AMRZ: No data returned for month 2025-03 [2025-09-16T17:10:23.927536] [DEBUG] [LOCAL ] AMPY: Primary data saved (27915 records). [2025-09-16T17:10:23.927770] [INFO ] [MAIN ] AMPY: Full history saved. [2025-09-16T17:10:23.927886] [INFO ] [MAIN ] Update Progress: 105/2369 symbols completed. AMPY took 16.9s. Estimated remaining: 3h 43m 7s. Rate: 10.1 symbols/min [2025-09-16T17:10:24.100681] [INFO ] [MAIN ] AMT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:24.142131] [DEBUG] [LOCAL ] AMT: Loaded 38572 existing records from primary storage. [2025-09-16T17:10:24.142299] [INFO ] [MAIN ] AMT: Loaded 38572 existing records. [2025-09-16T17:10:24.322929] [WARN ] [MAIN ] AMT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:10:24.323188] [INFO ] [MAIN ] AMT: Wiping existing primary data before full refresh. [2025-09-16T17:10:24.323235] [INFO ] [LOCAL ] AMT: Starting deletion of primary data. [2025-09-16T17:10:24.324357] [DEBUG] [LOCAL ] AMT: Deleted primary data file. [2025-09-16T17:10:24.324409] [INFO ] [MAIN ] AMT: Starting full primary refresh. [2025-09-16T17:10:24.380070] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-04. [2025-09-16T17:10:24.654575] [DEBUG] [REMOTE ] AMRZ: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:10:24.671899] [WARN ] [REMOTE ] AMRZ: No data returned for month 2025-04 [2025-09-16T17:10:24.691995] [INFO ] [REMOTE ] AMT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:10:25.174920] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-02. [2025-09-16T17:10:25.974929] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-05. [2025-09-16T17:10:26.741601] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T17:10:26.774856] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-03. [2025-09-16T17:10:27.187339] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-09-16T17:10:27.575008] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-05. [2025-09-16T17:10:28.057784] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-09-16T17:10:28.374919] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-06. [2025-09-16T17:10:28.977294] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T17:10:29.174945] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-06. [2025-09-16T17:10:29.558878] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-09-16T17:10:29.974894] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-05. [2025-09-16T17:10:30.011587] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-09-16T17:10:30.196010] [DEBUG] [REMOTE ] AMRZ: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:10:30.196291] [WARN ] [REMOTE ] AMRZ: No data returned for month 2025-05 [2025-09-16T17:10:30.774904] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-07. [2025-09-16T17:10:31.574906] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-06. [2025-09-16T17:10:31.802415] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-09-16T17:10:32.374908] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-04. [2025-09-16T17:10:32.590566] [INFO ] [REMOTE ] AMRZ: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-09-16T17:10:33.174941] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-07. [2025-09-16T17:10:33.745841] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T17:10:33.974918] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-05. [2025-09-16T17:10:34.704318] [INFO ] [REMOTE ] AMRZ: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-09-16T17:10:34.774871] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-07. [2025-09-16T17:10:35.574906] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-08. [2025-09-16T17:10:35.742887] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-09-16T17:10:36.267128] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T17:10:36.374876] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-08. [2025-09-16T17:10:36.834740] [INFO ] [REMOTE ] AMRZ: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-09-16T17:10:37.174898] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-06. [2025-09-16T17:10:37.264903] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-09-16T17:10:37.974906] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-09. [2025-09-16T17:10:38.774930] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-08. [2025-09-16T17:10:38.996948] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-09-16T17:10:39.469557] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T17:10:39.475287] [INFO ] [REMOTE ] AMRC: Full fetch complete (41400 records). [2025-09-16T17:10:39.475482] [DEBUG] [VALIDATOR ] AMRC: Validating primary data. [2025-09-16T17:10:39.580028] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-09. [2025-09-16T17:10:39.872617] [DEBUG] [VALIDATOR ] AMRC: All expected trading days are present. [2025-09-16T17:10:39.872854] [DEBUG] [VALIDATOR ] AMRC: Primary data validation passed. [2025-09-16T17:10:39.873511] [INFO ] [MAIN ] AMRC: Fetched primary data is valid. [2025-09-16T17:10:39.873593] [DEBUG] [LOCAL ] AMRC: Saving primary data. [2025-09-16T17:10:39.875366] [DEBUG] [LOCAL ] AMRC: Trimmed 41400 records to 27742 (keeping 150 days of data) [2025-09-16T17:10:39.957598] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T17:10:40.074883] [DEBUG] [LOCAL ] AMRC: Primary data saved (27742 records). [2025-09-16T17:10:40.075029] [INFO ] [MAIN ] AMRC: Full history saved. [2025-09-16T17:10:40.075114] [INFO ] [MAIN ] Update Progress: 106/2369 symbols completed. AMRC took 30.3s. Estimated remaining: 3h 46m 39s. Rate: 10.0 symbols/min [2025-09-16T17:10:40.247727] [INFO ] [MAIN ] AMTB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:40.266166] [DEBUG] [LOCAL ] AMTB: Loaded 15250 existing records from primary storage. [2025-09-16T17:10:40.266316] [INFO ] [MAIN ] AMTB: Loaded 15250 existing records. [2025-09-16T17:10:40.380024] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-09. [2025-09-16T17:10:40.444952] [WARN ] [MAIN ] AMTB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:10:40.445228] [INFO ] [MAIN ] AMTB: Wiping existing primary data before full refresh. [2025-09-16T17:10:40.445282] [INFO ] [LOCAL ] AMTB: Starting deletion of primary data. [2025-09-16T17:10:40.445952] [DEBUG] [LOCAL ] AMTB: Deleted primary data file. [2025-09-16T17:10:40.446003] [INFO ] [MAIN ] AMTB: Starting full primary refresh. [2025-09-16T17:10:40.839000] [INFO ] [REMOTE ] AMTB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:10:41.008235] [INFO ] [REMOTE ] AMRZ: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-09-16T17:10:41.014924] [INFO ] [REMOTE ] AMRZ: Full fetch complete (22645 records). [2025-09-16T17:10:41.015022] [DEBUG] [VALIDATOR ] AMRZ: Validating primary data. [2025-09-16T17:10:41.180591] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-02. [2025-09-16T17:10:41.213516] [WARN ] [VALIDATOR ] AMRZ: Validation failed - insufficient historical data. Have 60 unique days, need 150 days [2025-09-16T17:10:41.214613] [FATAL] [MAIN ] AMRZ: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:10:41.214690] [WARN ] [MAIN ] AMRZ: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:10:41.214739] [INFO ] [LOCAL ] AMRZ: Starting deletion of primary data. [2025-09-16T17:10:41.214793] [DEBUG] [LOCAL ] AMRZ: No primary data file found to delete. [2025-09-16T17:10:41.214831] [INFO ] [LOCAL ] AMRZ: Starting deletion of derivative data. [2025-09-16T17:10:41.214882] [DEBUG] [LOCAL ] AMRZ: No derivative data file found to delete. [2025-09-16T17:10:41.214946] [INFO ] [MAIN ] Update Progress: 107/2369 symbols completed. AMRZ took 19.0s. Estimated remaining: 3h 44m 50s. Rate: 10.1 symbols/min [2025-09-16T17:10:41.394256] [INFO ] [MAIN ] AMTD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:41.404550] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T17:10:41.409320] [INFO ] [REMOTE ] AMR: Full fetch complete (31750 records). [2025-09-16T17:10:41.411808] [DEBUG] [VALIDATOR ] AMR: Validating primary data. [2025-09-16T17:10:41.411737] [DEBUG] [LOCAL ] AMTD: Loaded 2875 existing records from primary storage. [2025-09-16T17:10:41.412039] [INFO ] [MAIN ] AMTD: Loaded 2875 existing records. [2025-09-16T17:10:41.771765] [WARN ] [MAIN ] AMTD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:10:41.848477] [INFO ] [MAIN ] AMTD: Wiping existing primary data before full refresh. [2025-09-16T17:10:41.854185] [INFO ] [LOCAL ] AMTD: Starting deletion of primary data. [2025-09-16T17:10:41.880230] [DEBUG] [LOCAL ] AMTD: Deleted primary data file. [2025-09-16T17:10:41.880346] [INFO ] [MAIN ] AMTD: Starting full primary refresh. [2025-09-16T17:10:41.977516] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-07. [2025-09-16T17:10:42.069773] [DEBUG] [VALIDATOR ] AMR: All expected trading days are present. [2025-09-16T17:10:42.075417] [DEBUG] [VALIDATOR ] AMR: Primary data validation passed. [2025-09-16T17:10:42.091220] [INFO ] [MAIN ] AMR: Fetched primary data is valid. [2025-09-16T17:10:42.130562] [DEBUG] [LOCAL ] AMR: Saving primary data. [2025-09-16T17:10:42.253767] [DEBUG] [LOCAL ] AMR: Trimmed 31750 records to 21471 (keeping 150 days of data) [2025-09-16T17:10:42.430670] [INFO ] [REMOTE ] AMTD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:10:42.476624] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-09-16T17:10:42.505028] [DEBUG] [LOCAL ] AMR: Primary data saved (21471 records). [2025-09-16T17:10:42.505201] [INFO ] [MAIN ] AMR: Full history saved. [2025-09-16T17:10:42.505295] [INFO ] [MAIN ] Update Progress: 108/2369 symbols completed. AMR took 34.1s. Estimated remaining: 3h 43m 6s. Rate: 10.1 symbols/min [2025-09-16T17:10:42.680929] [INFO ] [MAIN ] AMTM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:42.681131] [DEBUG] [LOCAL ] AMTM: No existing data file found in primary storage. [2025-09-16T17:10:42.681170] [WARN ] [MAIN ] AMTM: No existing data found, performing full refresh instead. [2025-09-16T17:10:42.681287] [INFO ] [MAIN ] AMTM: Wiping existing primary data before full refresh. [2025-09-16T17:10:42.681325] [INFO ] [LOCAL ] AMTM: Starting deletion of primary data. [2025-09-16T17:10:42.681366] [DEBUG] [LOCAL ] AMTM: No primary data file found to delete. [2025-09-16T17:10:42.681398] [INFO ] [MAIN ] AMTM: Starting full primary refresh. [2025-09-16T17:10:42.779947] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-03. [2025-09-16T17:10:43.044856] [INFO ] [REMOTE ] AMTM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:10:43.574929] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-02. [2025-09-16T17:10:43.801127] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T17:10:43.823451] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-09-16T17:10:44.374883] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-04. [2025-09-16T17:10:45.174897] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-02. [2025-09-16T17:10:45.241704] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T17:10:45.584871] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-09-16T17:10:45.692406] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-09-16T17:10:45.974917] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-08. [2025-09-16T17:10:46.774912] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-05. [2025-09-16T17:10:47.571297] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-09-16T17:10:47.581718] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-03. [2025-09-16T17:10:47.601491] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T17:10:48.374901] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-03. [2025-09-16T17:10:48.845618] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T17:10:48.892085] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-09-16T17:10:49.174905] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-04. [2025-09-16T17:10:49.618024] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-09-16T17:10:49.974901] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-06. [2025-09-16T17:10:50.774896] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-05. [2025-09-16T17:10:50.899991] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-09-16T17:10:51.310018] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-09-16T17:10:51.574900] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-06. [2025-09-16T17:10:52.086392] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-09-16T17:10:52.374909] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-07. [2025-09-16T17:10:53.174910] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-04. [2025-09-16T17:10:53.331037] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-09-16T17:10:53.974922] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-07. [2025-09-16T17:10:54.367106] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:10:54.480815] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-09-16T17:10:54.775018] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-08. [2025-09-16T17:10:55.574911] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-09. [2025-09-16T17:10:55.852383] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T17:10:56.374931] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-05. [2025-09-16T17:10:57.174906] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-09. [2025-09-16T17:10:57.311421] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-09-16T17:10:57.317714] [INFO ] [REMOTE ] AMT: Full fetch complete (57037 records). [2025-09-16T17:10:57.317833] [DEBUG] [VALIDATOR ] AMT: Validating primary data. [2025-09-16T17:10:57.700084] [DEBUG] [VALIDATOR ] AMT: All expected trading days are present. [2025-09-16T17:10:57.700168] [DEBUG] [VALIDATOR ] AMT: Primary data validation passed. [2025-09-16T17:10:57.700744] [INFO ] [MAIN ] AMT: Fetched primary data is valid. [2025-09-16T17:10:57.700789] [DEBUG] [LOCAL ] AMT: Saving primary data. [2025-09-16T17:10:57.702005] [DEBUG] [LOCAL ] AMT: Trimmed 57037 records to 38959 (keeping 150 days of data) [2025-09-16T17:10:57.979048] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-08. [2025-09-16T17:10:57.979242] [DEBUG] [LOCAL ] AMT: Primary data saved (38959 records). [2025-09-16T17:10:57.979510] [INFO ] [MAIN ] AMT: Full history saved. [2025-09-16T17:10:57.981747] [INFO ] [MAIN ] Update Progress: 109/2369 symbols completed. AMT took 34.1s. Estimated remaining: 3h 46m 19s. Rate: 10.0 symbols/min [2025-09-16T17:10:57.989665] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-09-16T17:10:58.054706] [INFO ] [REMOTE ] AMTB: Full fetch complete (22382 records). [2025-09-16T17:10:58.078388] [DEBUG] [VALIDATOR ] AMTB: Validating primary data. [2025-09-16T17:10:58.195082] [INFO ] [MAIN ] AMWL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:58.321379] [DEBUG] [LOCAL ] AMWL: Loaded 6494 existing records from primary storage. [2025-09-16T17:10:58.373893] [INFO ] [MAIN ] AMWL: Loaded 6494 existing records. [2025-09-16T17:10:58.628722] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-09-16T17:10:58.726851] [DEBUG] [VALIDATOR ] AMTB: All expected trading days are present. [2025-09-16T17:10:58.730567] [WARN ] [MAIN ] AMWL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:10:58.741066] [DEBUG] [VALIDATOR ] AMTB: Primary data validation passed. [2025-09-16T17:10:58.755965] [INFO ] [MAIN ] AMWL: Wiping existing primary data before full refresh. [2025-09-16T17:10:58.755850] [INFO ] [MAIN ] AMTB: Fetched primary data is valid. [2025-09-16T17:10:58.756159] [INFO ] [LOCAL ] AMWL: Starting deletion of primary data. [2025-09-16T17:10:58.756246] [DEBUG] [LOCAL ] AMTB: Saving primary data. [2025-09-16T17:10:58.760056] [DEBUG] [LOCAL ] AMWL: Deleted primary data file. [2025-09-16T17:10:58.762198] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-05 (2.39 seconds). [2025-09-16T17:10:58.762338] [INFO ] [MAIN ] AMWL: Starting full primary refresh. [2025-09-16T17:10:58.773118] [DEBUG] [LOCAL ] AMTB: Trimmed 22382 records to 15340 (keeping 150 days of data) [2025-09-16T17:10:58.778585] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-09. [2025-09-16T17:10:59.216869] [INFO ] [REMOTE ] AMWL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:10:59.231043] [DEBUG] [LOCAL ] AMTB: Primary data saved (15340 records). [2025-09-16T17:10:59.231139] [INFO ] [MAIN ] AMTB: Full history saved. [2025-09-16T17:10:59.231215] [INFO ] [MAIN ] Update Progress: 110/2369 symbols completed. AMTB took 19.2s. Estimated remaining: 3h 44m 35s. Rate: 10.1 symbols/min [2025-09-16T17:10:59.399380] [INFO ] [MAIN ] AMX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:10:59.435036] [DEBUG] [LOCAL ] AMX: Loaded 36088 existing records from primary storage. [2025-09-16T17:10:59.435173] [INFO ] [MAIN ] AMX: Loaded 36088 existing records. [2025-09-16T17:10:59.537796] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-09-16T17:10:59.541526] [INFO ] [REMOTE ] AMTD: Full fetch complete (3767 records). [2025-09-16T17:10:59.552110] [DEBUG] [VALIDATOR ] AMTD: Validating primary data. [2025-09-16T17:10:59.590534] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-06. [2025-09-16T17:10:59.676979] [WARN ] [MAIN ] AMX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:10:59.697486] [INFO ] [MAIN ] AMX: Wiping existing primary data before full refresh. [2025-09-16T17:10:59.714277] [INFO ] [LOCAL ] AMX: Starting deletion of primary data. [2025-09-16T17:10:59.731503] [DEBUG] [LOCAL ] AMX: Deleted primary data file. [2025-09-16T17:10:59.742013] [INFO ] [MAIN ] AMX: Starting full primary refresh. [2025-09-16T17:11:00.265753] [DEBUG] [VALIDATOR ] AMTD: All expected trading days are present. [2025-09-16T17:11:00.266026] [DEBUG] [VALIDATOR ] AMTD: Primary data validation passed. [2025-09-16T17:11:00.276526] [INFO ] [MAIN ] AMTD: Fetched primary data is valid. [2025-09-16T17:11:00.291894] [DEBUG] [LOCAL ] AMTD: Saving primary data. [2025-09-16T17:11:00.342064] [INFO ] [REMOTE ] AMX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:00.338958] [DEBUG] [LOCAL ] AMTD: Trimmed 3767 records to 2906 (keeping 150 days of data) [2025-09-16T17:11:00.359325] [DEBUG] [LOCAL ] AMTD: Primary data saved (2906 records). [2025-09-16T17:11:00.359391] [INFO ] [MAIN ] AMTD: Full history saved. [2025-09-16T17:11:00.359470] [INFO ] [MAIN ] Update Progress: 111/2369 symbols completed. AMTD took 19.1s. Estimated remaining: 3h 42m 50s. Rate: 10.1 symbols/min [2025-09-16T17:11:00.380058] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-02. [2025-09-16T17:11:00.532911] [INFO ] [MAIN ] AN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:00.555440] [DEBUG] [LOCAL ] AN: Loaded 23001 existing records from primary storage. [2025-09-16T17:11:00.555576] [INFO ] [MAIN ] AN: Loaded 23001 existing records. [2025-09-16T17:11:00.736211] [WARN ] [MAIN ] AN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:11:00.736463] [INFO ] [MAIN ] AN: Wiping existing primary data before full refresh. [2025-09-16T17:11:00.736508] [INFO ] [LOCAL ] AN: Starting deletion of primary data. [2025-09-16T17:11:00.737230] [DEBUG] [LOCAL ] AN: Deleted primary data file. [2025-09-16T17:11:00.737280] [INFO ] [MAIN ] AN: Starting full primary refresh. [2025-09-16T17:11:01.094905] [INFO ] [REMOTE ] AN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:01.174861] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-02. [2025-09-16T17:11:01.695613] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-06 (2.11 seconds). [2025-09-16T17:11:01.974903] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-07. [2025-09-16T17:11:02.654758] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-02 (2.27 seconds). [2025-09-16T17:11:02.774899] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-03. [2025-09-16T17:11:02.959370] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T17:11:03.311397] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-09-16T17:11:03.574890] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-04. [2025-09-16T17:11:04.042592] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-09-16T17:11:04.053450] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-09-16T17:11:04.374906] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-05. [2025-09-16T17:11:05.167989] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-09-16T17:11:05.174833] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-06. [2025-09-16T17:11:05.905429] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-09-16T17:11:05.974914] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-03. [2025-09-16T17:11:06.774913] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-02. [2025-09-16T17:11:07.122700] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T17:11:07.574909] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-04. [2025-09-16T17:11:08.072733] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-09-16T17:11:08.374907] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-03. [2025-09-16T17:11:08.942064] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T17:11:09.174931] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-08. [2025-09-16T17:11:09.809701] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-09-16T17:11:09.974917] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-04. [2025-09-16T17:11:10.774879] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-05. [2025-09-16T17:11:11.248901] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-09-16T17:11:11.574891] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-07. [2025-09-16T17:11:12.018012] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-09-16T17:11:12.214894] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T17:11:12.230061] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-09-16T17:11:12.374893] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-06. [2025-09-16T17:11:13.174918] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-05. [2025-09-16T17:11:13.799367] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T17:11:13.974957] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-07. [2025-09-16T17:11:14.487415] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T17:11:14.774918] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-09. [2025-09-16T17:11:15.248113] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-09-16T17:11:15.574905] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-08. [2025-09-16T17:11:16.201902] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-09-16T17:11:16.207808] [INFO ] [REMOTE ] AMTM: Full fetch complete (55177 records). [2025-09-16T17:11:16.207926] [DEBUG] [VALIDATOR ] AMTM: Validating primary data. [2025-09-16T17:11:16.382308] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-06. [2025-09-16T17:11:16.544699] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-09-16T17:11:16.639568] [DEBUG] [VALIDATOR ] AMTM: All expected trading days are present. [2025-09-16T17:11:16.639666] [DEBUG] [VALIDATOR ] AMTM: Primary data validation passed. [2025-09-16T17:11:16.640210] [INFO ] [MAIN ] AMTM: Fetched primary data is valid. [2025-09-16T17:11:16.640263] [DEBUG] [LOCAL ] AMTM: Saving primary data. [2025-09-16T17:11:16.641404] [DEBUG] [LOCAL ] AMTM: Trimmed 55177 records to 37550 (keeping 150 days of data) [2025-09-16T17:11:16.868097] [DEBUG] [LOCAL ] AMTM: Primary data saved (37550 records). [2025-09-16T17:11:16.868272] [INFO ] [MAIN ] AMTM: Full history saved. [2025-09-16T17:11:16.868368] [INFO ] [MAIN ] Update Progress: 112/2369 symbols completed. AMTM took 34.4s. Estimated remaining: 3h 46m 18s. Rate: 10.0 symbols/min [2025-09-16T17:11:17.040674] [INFO ] [MAIN ] ANET: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:17.081279] [DEBUG] [LOCAL ] ANET: Loaded 39726 existing records from primary storage. [2025-09-16T17:11:17.081448] [INFO ] [MAIN ] ANET: Loaded 39726 existing records. [2025-09-16T17:11:17.180340] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-09. [2025-09-16T17:11:17.263384] [WARN ] [MAIN ] ANET: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:11:17.263600] [INFO ] [MAIN ] ANET: Wiping existing primary data before full refresh. [2025-09-16T17:11:17.263703] [INFO ] [LOCAL ] ANET: Starting deletion of primary data. [2025-09-16T17:11:17.264868] [DEBUG] [LOCAL ] ANET: Deleted primary data file. [2025-09-16T17:11:17.264931] [INFO ] [MAIN ] ANET: Starting full primary refresh. [2025-09-16T17:11:17.631143] [INFO ] [REMOTE ] ANET: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:17.917998] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-09-16T17:11:17.920081] [INFO ] [REMOTE ] AMWL: Full fetch complete (10072 records). [2025-09-16T17:11:17.920186] [DEBUG] [VALIDATOR ] AMWL: Validating primary data. [2025-09-16T17:11:17.920582] [FATAL] [VALIDATOR ] AMWL: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:11:17.920685] [FATAL] [MAIN ] AMWL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:11:17.920744] [WARN ] [MAIN ] AMWL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:11:17.920807] [INFO ] [LOCAL ] AMWL: Starting deletion of primary data. [2025-09-16T17:11:17.920882] [DEBUG] [LOCAL ] AMWL: No primary data file found to delete. [2025-09-16T17:11:17.920947] [INFO ] [LOCAL ] AMWL: Starting deletion of derivative data. [2025-09-16T17:11:17.921285] [DEBUG] [LOCAL ] AMWL: Deleted derivative data file. [2025-09-16T17:11:17.921398] [INFO ] [MAIN ] Update Progress: 113/2369 symbols completed. AMWL took 19.9s. Estimated remaining: 3h 44m 33s. Rate: 10.0 symbols/min [2025-09-16T17:11:17.988363] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-08. [2025-09-16T17:11:18.123450] [INFO ] [MAIN ] ANF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:18.125121] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T17:11:18.175448] [DEBUG] [LOCAL ] ANF: Loaded 38417 existing records from primary storage. [2025-09-16T17:11:18.175591] [INFO ] [MAIN ] ANF: Loaded 38417 existing records. [2025-09-16T17:11:18.350690] [WARN ] [MAIN ] ANF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:11:18.350938] [INFO ] [MAIN ] ANF: Wiping existing primary data before full refresh. [2025-09-16T17:11:18.350999] [INFO ] [LOCAL ] ANF: Starting deletion of primary data. [2025-09-16T17:11:18.352291] [DEBUG] [LOCAL ] ANF: Deleted primary data file. [2025-09-16T17:11:18.352367] [INFO ] [MAIN ] ANF: Starting full primary refresh. [2025-09-16T17:11:18.739888] [INFO ] [REMOTE ] ANF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:18.774901] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-02. [2025-09-16T17:11:19.245577] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-09-16T17:11:19.574949] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-07. [2025-09-16T17:11:20.374888] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-02. [2025-09-16T17:11:20.635986] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-09-16T17:11:21.013009] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T17:11:21.174870] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-08. [2025-09-16T17:11:21.974926] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-03. [2025-09-16T17:11:22.749287] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T17:11:22.779962] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-09. [2025-09-16T17:11:22.797596] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-02 (2.42 seconds). [2025-09-16T17:11:23.574887] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-09. [2025-09-16T17:11:23.651278] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-09-16T17:11:24.348705] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T17:11:24.355569] [INFO ] [REMOTE ] AMX: Full fetch complete (53231 records). [2025-09-16T17:11:24.355799] [DEBUG] [VALIDATOR ] AMX: Validating primary data. [2025-09-16T17:11:24.380087] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-03. [2025-09-16T17:11:24.741282] [DEBUG] [VALIDATOR ] AMX: All expected trading days are present. [2025-09-16T17:11:24.741383] [DEBUG] [VALIDATOR ] AMX: Primary data validation passed. [2025-09-16T17:11:24.741987] [INFO ] [MAIN ] AMX: Fetched primary data is valid. [2025-09-16T17:11:24.742056] [DEBUG] [LOCAL ] AMX: Saving primary data. [2025-09-16T17:11:24.743186] [DEBUG] [LOCAL ] AMX: Trimmed 53231 records to 36405 (keeping 150 days of data) [2025-09-16T17:11:24.959201] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T17:11:24.965315] [INFO ] [REMOTE ] AN: Full fetch complete (35890 records). [2025-09-16T17:11:24.966175] [DEBUG] [VALIDATOR ] AN: Validating primary data. [2025-09-16T17:11:25.179972] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-04. [2025-09-16T17:11:25.276809] [DEBUG] [LOCAL ] AMX: Primary data saved (36405 records). [2025-09-16T17:11:25.277035] [INFO ] [MAIN ] AMX: Full history saved. [2025-09-16T17:11:25.277422] [INFO ] [MAIN ] Update Progress: 114/2369 symbols completed. AMX took 26.0s. Estimated remaining: 3h 44m 54s. Rate: 10.0 symbols/min [2025-09-16T17:11:25.489776] [DEBUG] [VALIDATOR ] AN: All expected trading days are present. [2025-09-16T17:11:25.490193] [DEBUG] [VALIDATOR ] AN: Primary data validation passed. [2025-09-16T17:11:25.496123] [INFO ] [MAIN ] AN: Fetched primary data is valid. [2025-09-16T17:11:25.501326] [DEBUG] [LOCAL ] AN: Saving primary data. [2025-09-16T17:11:25.565461] [DEBUG] [LOCAL ] AN: Trimmed 35890 records to 23233 (keeping 150 days of data) [2025-09-16T17:11:25.583923] [INFO ] [MAIN ] ANGX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:25.584996] [DEBUG] [LOCAL ] ANGX: No existing data file found in primary storage. [2025-09-16T17:11:25.585079] [WARN ] [MAIN ] ANGX: No existing data found, performing full refresh instead. [2025-09-16T17:11:25.585217] [INFO ] [MAIN ] ANGX: Wiping existing primary data before full refresh. [2025-09-16T17:11:25.585259] [INFO ] [LOCAL ] ANGX: Starting deletion of primary data. [2025-09-16T17:11:25.585300] [DEBUG] [LOCAL ] ANGX: No primary data file found to delete. [2025-09-16T17:11:25.585362] [INFO ] [MAIN ] ANGX: Starting full primary refresh. [2025-09-16T17:11:26.081013] [INFO ] [REMOTE ] ANGX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:26.085813] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-02. [2025-09-16T17:11:26.105969] [DEBUG] [LOCAL ] AN: Primary data saved (23233 records). [2025-09-16T17:11:26.106111] [INFO ] [MAIN ] AN: Full history saved. [2025-09-16T17:11:26.106207] [INFO ] [MAIN ] Update Progress: 115/2369 symbols completed. AN took 25.7s. Estimated remaining: 3h 43m 7s. Rate: 10.1 symbols/min [2025-09-16T17:11:26.317504] [INFO ] [MAIN ] ANRO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:26.333677] [DEBUG] [LOCAL ] ANRO: Loaded 13456 existing records from primary storage. [2025-09-16T17:11:26.333856] [INFO ] [MAIN ] ANRO: Loaded 13456 existing records. [2025-09-16T17:11:26.380375] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:26.405887] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-02 [2025-09-16T17:11:26.544501] [WARN ] [MAIN ] ANRO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:11:26.544823] [INFO ] [MAIN ] ANRO: Wiping existing primary data before full refresh. [2025-09-16T17:11:26.544919] [INFO ] [LOCAL ] ANRO: Starting deletion of primary data. [2025-09-16T17:11:26.545521] [DEBUG] [LOCAL ] ANRO: Deleted primary data file. [2025-09-16T17:11:26.545585] [INFO ] [MAIN ] ANRO: Starting full primary refresh. [2025-09-16T17:11:26.786104] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-03. [2025-09-16T17:11:26.936685] [INFO ] [REMOTE ] ANRO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:26.971983] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-03 (2.59 seconds). [2025-09-16T17:11:27.046441] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-09-16T17:11:27.066525] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:27.066837] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-03 [2025-09-16T17:11:27.574909] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-04. [2025-09-16T17:11:27.769338] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:27.769691] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-04 [2025-09-16T17:11:28.374897] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-04. [2025-09-16T17:11:29.174909] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-05. [2025-09-16T17:11:29.343816] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:29.344144] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-05 [2025-09-16T17:11:29.974934] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-02. [2025-09-16T17:11:30.644311] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-04 (2.27 seconds). [2025-09-16T17:11:30.774901] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-05. [2025-09-16T17:11:31.139155] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-09-16T17:11:31.574913] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-03. [2025-09-16T17:11:32.374892] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-05. [2025-09-16T17:11:32.761697] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-09-16T17:11:32.929771] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-09-16T17:11:33.174930] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-04. [2025-09-16T17:11:33.974885] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-06. [2025-09-16T17:11:34.059418] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T17:11:34.215210] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:34.215535] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-06 [2025-09-16T17:11:34.332145] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-09-16T17:11:34.774933] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-07. [2025-09-16T17:11:34.960362] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-07: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:34.960717] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-07 [2025-09-16T17:11:35.574897] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-08. [2025-09-16T17:11:35.764107] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-08: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:35.764488] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-08 [2025-09-16T17:11:36.374914] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-06. [2025-09-16T17:11:37.174938] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-09. [2025-09-16T17:11:37.894835] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T17:11:37.974923] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-07. [2025-09-16T17:11:38.129820] [INFO ] [REMOTE ] ANGX: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T17:11:38.132285] [INFO ] [REMOTE ] ANGX: Full fetch complete (1325 records). [2025-09-16T17:11:38.132360] [DEBUG] [VALIDATOR ] ANGX: Validating primary data. [2025-09-16T17:11:38.325018] [WARN ] [VALIDATOR ] ANGX: Validation failed - insufficient historical data. Have 4 unique days, need 150 days [2025-09-16T17:11:38.325240] [FATAL] [MAIN ] ANGX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:11:38.325296] [WARN ] [MAIN ] ANGX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:11:38.325355] [INFO ] [LOCAL ] ANGX: Starting deletion of primary data. [2025-09-16T17:11:38.325414] [DEBUG] [LOCAL ] ANGX: No primary data file found to delete. [2025-09-16T17:11:38.325459] [INFO ] [LOCAL ] ANGX: Starting deletion of derivative data. [2025-09-16T17:11:38.325507] [DEBUG] [LOCAL ] ANGX: No derivative data file found to delete. [2025-09-16T17:11:38.325584] [INFO ] [MAIN ] Update Progress: 116/2369 symbols completed. ANGX took 13.0s. Estimated remaining: 3h 45m 3s. Rate: 10.0 symbols/min [2025-09-16T17:11:38.501209] [INFO ] [MAIN ] ANVS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:38.520983] [DEBUG] [LOCAL ] ANVS: Loaded 17585 existing records from primary storage. [2025-09-16T17:11:38.521125] [INFO ] [MAIN ] ANVS: Loaded 17585 existing records. [2025-09-16T17:11:38.703845] [WARN ] [MAIN ] ANVS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:11:38.704117] [INFO ] [MAIN ] ANVS: Wiping existing primary data before full refresh. [2025-09-16T17:11:38.704165] [INFO ] [LOCAL ] ANVS: Starting deletion of primary data. [2025-09-16T17:11:38.704875] [DEBUG] [LOCAL ] ANVS: Deleted primary data file. [2025-09-16T17:11:38.704926] [INFO ] [MAIN ] ANVS: Starting full primary refresh. [2025-09-16T17:11:38.783228] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-06. [2025-09-16T17:11:39.077108] [INFO ] [REMOTE ] ANVS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:39.487383] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T17:11:39.574902] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-08. [2025-09-16T17:11:40.374936] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-02. [2025-09-16T17:11:41.172310] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-09-16T17:11:41.182584] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-05. [2025-09-16T17:11:41.238933] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-06 (2.46 seconds). [2025-09-16T17:11:41.624074] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T17:11:41.974900] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-03. [2025-09-16T17:11:42.228265] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-09-16T17:11:42.774880] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-06. [2025-09-16T17:11:43.168585] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-09-16T17:11:43.574928] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-07. [2025-09-16T17:11:43.916248] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T17:11:44.374910] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-09. [2025-09-16T17:11:45.174905] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-07. [2025-09-16T17:11:45.803471] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-09-16T17:11:45.826940] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-09-16T17:11:45.974857] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-08. [2025-09-16T17:11:46.234452] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-09-16T17:11:46.240527] [INFO ] [REMOTE ] ANF: Full fetch complete (56855 records). [2025-09-16T17:11:46.240636] [DEBUG] [VALIDATOR ] ANF: Validating primary data. [2025-09-16T17:11:46.625230] [DEBUG] [VALIDATOR ] ANF: All expected trading days are present. [2025-09-16T17:11:46.625351] [DEBUG] [VALIDATOR ] ANF: Primary data validation passed. [2025-09-16T17:11:46.626130] [INFO ] [MAIN ] ANF: Fetched primary data is valid. [2025-09-16T17:11:46.626293] [DEBUG] [LOCAL ] ANF: Saving primary data. [2025-09-16T17:11:46.638193] [DEBUG] [LOCAL ] ANF: Trimmed 56855 records to 38799 (keeping 150 days of data) [2025-09-16T17:11:46.638046] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-09-16T17:11:46.813817] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-09. [2025-09-16T17:11:46.885975] [DEBUG] [LOCAL ] ANF: Primary data saved (38799 records). [2025-09-16T17:11:46.886193] [INFO ] [MAIN ] ANF: Full history saved. [2025-09-16T17:11:46.886529] [INFO ] [MAIN ] Update Progress: 117/2369 symbols completed. ANF took 29.0s. Estimated remaining: 3h 45m 47s. Rate: 10.0 symbols/min [2025-09-16T17:11:47.515960] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-09-16T17:11:47.518337] [INFO ] [REMOTE ] ANRO: Full fetch complete (22405 records). [2025-09-16T17:11:47.518492] [DEBUG] [VALIDATOR ] ANRO: Validating primary data. [2025-09-16T17:11:47.580145] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-04. [2025-09-16T17:11:48.059976] [DEBUG] [VALIDATOR ] ANRO: All expected trading days are present. [2025-09-16T17:11:48.060107] [DEBUG] [VALIDATOR ] ANRO: Primary data validation passed. [2025-09-16T17:11:48.060803] [INFO ] [MAIN ] ANRO: Fetched primary data is valid. [2025-09-16T17:11:48.060896] [DEBUG] [LOCAL ] ANRO: Saving primary data. [2025-09-16T17:11:48.062563] [DEBUG] [LOCAL ] ANRO: Trimmed 22405 records to 13497 (keeping 150 days of data) [2025-09-16T17:11:48.160021] [DEBUG] [LOCAL ] ANRO: Primary data saved (13497 records). [2025-09-16T17:11:48.160189] [INFO ] [MAIN ] ANRO: Full history saved. [2025-09-16T17:11:48.160345] [INFO ] [MAIN ] Update Progress: 118/2369 symbols completed. ANRO took 22.1s. Estimated remaining: 3h 44m 10s. Rate: 10.0 symbols/min [2025-09-16T17:11:48.374998] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-08. [2025-09-16T17:11:48.767293] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:11:49.174956] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-05. [2025-09-16T17:11:49.999885] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T17:11:50.005599] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-06. [2025-09-16T17:11:50.697521] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-08 (2.32 seconds). [2025-09-16T17:11:50.774874] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-09. [2025-09-16T17:11:51.144827] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T17:11:51.574932] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-07. [2025-09-16T17:11:52.837634] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-09-16T17:11:52.843969] [INFO ] [REMOTE ] ANET: Full fetch complete (58446 records). [2025-09-16T17:11:52.844088] [DEBUG] [VALIDATOR ] ANET: Validating primary data. [2025-09-16T17:11:53.191000] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-09-16T17:11:53.261524] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-08. [2025-09-16T17:11:53.264449] [DEBUG] [VALIDATOR ] ANET: All expected trading days are present. [2025-09-16T17:11:53.265477] [DEBUG] [VALIDATOR ] ANET: Primary data validation passed. [2025-09-16T17:11:53.266139] [INFO ] [MAIN ] ANET: Fetched primary data is valid. [2025-09-16T17:11:53.266190] [DEBUG] [LOCAL ] ANET: Saving primary data. [2025-09-16T17:11:53.267546] [DEBUG] [LOCAL ] ANET: Trimmed 58446 records to 40116 (keeping 150 days of data) [2025-09-16T17:11:53.515334] [DEBUG] [LOCAL ] ANET: Primary data saved (40116 records). [2025-09-16T17:11:53.515463] [INFO ] [MAIN ] ANET: Full history saved. [2025-09-16T17:11:53.516721] [INFO ] [MAIN ] Update Progress: 119/2369 symbols completed. ANET took 36.6s. Estimated remaining: 3h 43m 52s. Rate: 10.1 symbols/min [2025-09-16T17:11:54.440595] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T17:11:54.445910] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-09. [2025-09-16T17:11:55.581464] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T17:11:55.584007] [INFO ] [REMOTE ] ANVS: Full fetch complete (26385 records). [2025-09-16T17:11:55.584096] [DEBUG] [VALIDATOR ] ANVS: Validating primary data. [2025-09-16T17:11:55.956905] [DEBUG] [VALIDATOR ] ANVS: All expected trading days are present. [2025-09-16T17:11:55.957014] [DEBUG] [VALIDATOR ] ANVS: Primary data validation passed. [2025-09-16T17:11:55.957376] [INFO ] [MAIN ] ANVS: Fetched primary data is valid. [2025-09-16T17:11:55.957427] [DEBUG] [LOCAL ] ANVS: Saving primary data. [2025-09-16T17:11:55.958267] [DEBUG] [LOCAL ] ANVS: Trimmed 26385 records to 17667 (keeping 150 days of data) [2025-09-16T17:11:56.066246] [DEBUG] [LOCAL ] ANVS: Primary data saved (17667 records). [2025-09-16T17:11:56.066398] [INFO ] [MAIN ] ANVS: Full history saved. [2025-09-16T17:11:56.066481] [INFO ] [MAIN ] Update Progress: 120/2369 symbols completed. ANVS took 17.7s. Estimated remaining: 3h 42m 42s. Rate: 10.1 symbols/min [2025-09-16T17:11:56.619061] [INFO ] [MAIN ] AOMR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:56.758784] [INFO ] [MAIN ] AOMD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:56.765389] [DEBUG] [LOCAL ] AOMD: No existing data file found in primary storage. [2025-09-16T17:11:56.746823] [INFO ] [MAIN ] AON: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:56.784826] [WARN ] [MAIN ] AOMD: No existing data found, performing full refresh instead. [2025-09-16T17:11:56.784747] [INFO ] [MAIN ] AOD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:11:56.785581] [INFO ] [MAIN ] AOMD: Wiping existing primary data before full refresh. [2025-09-16T17:11:56.785661] [DEBUG] [LOCAL ] AOD: No existing data file found in primary storage. [2025-09-16T17:11:56.785752] [INFO ] [LOCAL ] AOMD: Starting deletion of primary data. [2025-09-16T17:11:56.785810] [WARN ] [MAIN ] AOD: No existing data found, performing full refresh instead. [2025-09-16T17:11:56.785902] [DEBUG] [LOCAL ] AOMD: No primary data file found to delete. [2025-09-16T17:11:56.786175] [INFO ] [MAIN ] AOD: Wiping existing primary data before full refresh. [2025-09-16T17:11:56.786234] [INFO ] [LOCAL ] AOD: Starting deletion of primary data. [2025-09-16T17:11:56.786303] [INFO ] [MAIN ] AOMD: Starting full primary refresh. [2025-09-16T17:11:56.786356] [DEBUG] [LOCAL ] AOD: No primary data file found to delete. [2025-09-16T17:11:56.798104] [INFO ] [MAIN ] AOD: Starting full primary refresh. [2025-09-16T17:11:56.919380] [DEBUG] [LOCAL ] AOMR: Loaded 12648 existing records from primary storage. [2025-09-16T17:11:56.954477] [INFO ] [MAIN ] AOMR: Loaded 12648 existing records. [2025-09-16T17:11:57.362909] [DEBUG] [LOCAL ] AON: Loaded 33409 existing records from primary storage. [2025-09-16T17:11:57.397628] [INFO ] [MAIN ] AON: Loaded 33409 existing records. [2025-09-16T17:11:57.609375] [WARN ] [MAIN ] AOMR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:11:57.770684] [INFO ] [MAIN ] AOMR: Wiping existing primary data before full refresh. [2025-09-16T17:11:57.807589] [INFO ] [REMOTE ] AOD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:57.833864] [INFO ] [LOCAL ] AOMR: Starting deletion of primary data. [2025-09-16T17:11:57.828689] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-02. [2025-09-16T17:11:57.880472] [INFO ] [REMOTE ] AOMD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:57.897885] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-02. [2025-09-16T17:11:57.897626] [DEBUG] [LOCAL ] AOMR: Deleted primary data file. [2025-09-16T17:11:57.898043] [INFO ] [MAIN ] AOMR: Starting full primary refresh. [2025-09-16T17:11:57.930370] [WARN ] [MAIN ] AON: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:11:57.984749] [INFO ] [MAIN ] AON: Wiping existing primary data before full refresh. [2025-09-16T17:11:58.013343] [INFO ] [LOCAL ] AON: Starting deletion of primary data. [2025-09-16T17:11:58.031680] [DEBUG] [LOCAL ] AON: Deleted primary data file. [2025-09-16T17:11:58.064614] [INFO ] [MAIN ] AON: Starting full primary refresh. [2025-09-16T17:11:58.176054] [DEBUG] [REMOTE ] AOMD: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:58.245088] [WARN ] [REMOTE ] AOMD: No data returned for month 2025-02 [2025-09-16T17:11:58.270305] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-03. [2025-09-16T17:11:58.535665] [INFO ] [REMOTE ] AOMR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:58.547399] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-02. [2025-09-16T17:11:58.635677] [DEBUG] [REMOTE ] AOMD: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:58.656759] [INFO ] [REMOTE ] AON: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:11:58.657232] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-02. [2025-09-16T17:11:58.657019] [WARN ] [REMOTE ] AOMD: No data returned for month 2025-03 [2025-09-16T17:11:58.658282] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-04. [2025-09-16T17:11:58.922185] [DEBUG] [REMOTE ] AOMD: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:11:58.922503] [WARN ] [REMOTE ] AOMD: No data returned for month 2025-04 [2025-09-16T17:11:58.922996] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-05. [2025-09-16T17:11:59.329354] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-09-16T17:11:59.432854] [INFO ] [REMOTE ] AOMD: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-09-16T17:11:59.574921] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-06. [2025-09-16T17:11:59.590451] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T17:12:00.025701] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T17:12:00.206995] [INFO ] [REMOTE ] AOMD: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-09-16T17:12:00.374895] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-07. [2025-09-16T17:12:01.174903] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-03. [2025-09-16T17:12:01.217855] [INFO ] [REMOTE ] AOMD: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-09-16T17:12:01.974901] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-03. [2025-09-16T17:12:02.635241] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T17:12:02.774919] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-08. [2025-09-16T17:12:03.092233] [INFO ] [REMOTE ] AOMD: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-09-16T17:12:03.214745] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-09-16T17:12:03.574901] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-04. [2025-09-16T17:12:04.374896] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-09. [2025-09-16T17:12:04.583582] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-09-16T17:12:04.749678] [INFO ] [REMOTE ] AOMD: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-09-16T17:12:04.750519] [INFO ] [REMOTE ] AOMD: Full fetch complete (1684 records). [2025-09-16T17:12:04.750586] [DEBUG] [VALIDATOR ] AOMD: Validating primary data. [2025-09-16T17:12:04.942388] [WARN ] [VALIDATOR ] AOMD: Validation failed - insufficient historical data. Have 78 unique days, need 150 days [2025-09-16T17:12:04.942566] [FATAL] [MAIN ] AOMD: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:12:04.942635] [WARN ] [MAIN ] AOMD: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:12:04.942721] [INFO ] [LOCAL ] AOMD: Starting deletion of primary data. [2025-09-16T17:12:04.942788] [DEBUG] [LOCAL ] AOMD: No primary data file found to delete. [2025-09-16T17:12:04.942824] [INFO ] [LOCAL ] AOMD: Starting deletion of derivative data. [2025-09-16T17:12:04.942874] [DEBUG] [LOCAL ] AOMD: No derivative data file found to delete. [2025-09-16T17:12:04.942934] [INFO ] [MAIN ] Update Progress: 121/2369 symbols completed. AOMD took 8.9s. Estimated remaining: 3h 43m 31s. Rate: 10.1 symbols/min [2025-09-16T17:12:05.113056] [INFO ] [MAIN ] AORT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:12:05.137175] [DEBUG] [LOCAL ] AORT: Loaded 21502 existing records from primary storage. [2025-09-16T17:12:05.137315] [INFO ] [MAIN ] AORT: Loaded 21502 existing records. [2025-09-16T17:12:05.179992] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-05. [2025-09-16T17:12:05.318742] [WARN ] [MAIN ] AORT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:12:05.318986] [INFO ] [MAIN ] AORT: Wiping existing primary data before full refresh. [2025-09-16T17:12:05.319034] [INFO ] [LOCAL ] AORT: Starting deletion of primary data. [2025-09-16T17:12:05.319805] [DEBUG] [LOCAL ] AORT: Deleted primary data file. [2025-09-16T17:12:05.319885] [INFO ] [MAIN ] AORT: Starting full primary refresh. [2025-09-16T17:12:05.676446] [INFO ] [REMOTE ] AORT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:12:05.947949] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-09-16T17:12:05.974851] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-06. [2025-09-16T17:12:06.751900] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T17:12:06.774882] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-07. [2025-09-16T17:12:07.274253] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-09-16T17:12:07.574936] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-03. [2025-09-16T17:12:08.374884] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-04. [2025-09-16T17:12:09.174911] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-08. [2025-09-16T17:12:09.296885] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T17:12:09.806097] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:12:09.974908] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-05. [2025-09-16T17:12:10.168471] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T17:12:10.774868] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-09. [2025-09-16T17:12:11.231966] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-09-16T17:12:11.574915] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-06. [2025-09-16T17:12:11.652018] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-09-16T17:12:11.654475] [INFO ] [REMOTE ] AOMR: Full fetch complete (18977 records). [2025-09-16T17:12:11.654563] [DEBUG] [VALIDATOR ] AOMR: Validating primary data. [2025-09-16T17:12:12.017830] [DEBUG] [VALIDATOR ] AOMR: All expected trading days are present. [2025-09-16T17:12:12.017908] [DEBUG] [VALIDATOR ] AOMR: Primary data validation passed. [2025-09-16T17:12:12.018182] [INFO ] [MAIN ] AOMR: Fetched primary data is valid. [2025-09-16T17:12:12.018223] [DEBUG] [LOCAL ] AOMR: Saving primary data. [2025-09-16T17:12:12.018776] [DEBUG] [LOCAL ] AOMR: Trimmed 18977 records to 12747 (keeping 150 days of data) [2025-09-16T17:12:12.096271] [DEBUG] [LOCAL ] AOMR: Primary data saved (12747 records). [2025-09-16T17:12:12.096426] [INFO ] [MAIN ] AOMR: Full history saved. [2025-09-16T17:12:12.096514] [INFO ] [MAIN ] Update Progress: 122/2369 symbols completed. AOMR took 16.0s. Estimated remaining: 3h 43m 47s. Rate: 10.0 symbols/min [2025-09-16T17:12:12.269718] [INFO ] [MAIN ] AOS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:12:12.312785] [DEBUG] [LOCAL ] AOS: Loaded 36309 existing records from primary storage. [2025-09-16T17:12:12.312920] [INFO ] [MAIN ] AOS: Loaded 36309 existing records. [2025-09-16T17:12:12.380045] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-02. [2025-09-16T17:12:12.496442] [WARN ] [MAIN ] AOS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:12:12.496742] [INFO ] [MAIN ] AOS: Wiping existing primary data before full refresh. [2025-09-16T17:12:12.496795] [INFO ] [LOCAL ] AOS: Starting deletion of primary data. [2025-09-16T17:12:12.498410] [DEBUG] [LOCAL ] AOS: Deleted primary data file. [2025-09-16T17:12:12.498478] [INFO ] [MAIN ] AOS: Starting full primary refresh. [2025-09-16T17:12:12.860452] [INFO ] [REMOTE ] AOS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:12:12.876152] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:12:13.174930] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-04. [2025-09-16T17:12:13.384902] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-09-16T17:12:13.974926] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-07. [2025-09-16T17:12:14.774900] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-02. [2025-09-16T17:12:14.964700] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-09-16T17:12:15.002940] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T17:12:15.574920] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-05. [2025-09-16T17:12:16.269875] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T17:12:16.374951] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-08. [2025-09-16T17:12:16.970964] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T17:12:17.174863] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-03. [2025-09-16T17:12:17.627139] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-09-16T17:12:17.974918] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-09. [2025-09-16T17:12:18.774865] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-06. [2025-09-16T17:12:18.882130] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T17:12:19.029258] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T17:12:19.033190] [INFO ] [REMOTE ] AOD: Full fetch complete (32982 records). [2025-09-16T17:12:19.033264] [DEBUG] [VALIDATOR ] AOD: Validating primary data. [2025-09-16T17:12:19.400465] [DEBUG] [VALIDATOR ] AOD: All expected trading days are present. [2025-09-16T17:12:19.400542] [DEBUG] [VALIDATOR ] AOD: Primary data validation passed. [2025-09-16T17:12:19.400940] [INFO ] [MAIN ] AOD: Fetched primary data is valid. [2025-09-16T17:12:19.400983] [DEBUG] [LOCAL ] AOD: Saving primary data. [2025-09-16T17:12:19.401761] [DEBUG] [LOCAL ] AOD: Trimmed 32982 records to 22340 (keeping 150 days of data) [2025-09-16T17:12:19.536833] [DEBUG] [LOCAL ] AOD: Primary data saved (22340 records). [2025-09-16T17:12:19.536965] [INFO ] [MAIN ] AOD: Full history saved. [2025-09-16T17:12:19.537040] [INFO ] [MAIN ] Update Progress: 123/2369 symbols completed. AOD took 23.5s. Estimated remaining: 3h 44m 8s. Rate: 10.0 symbols/min [2025-09-16T17:12:19.580038] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-04. [2025-09-16T17:12:19.713373] [INFO ] [MAIN ] AP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:12:19.719961] [DEBUG] [LOCAL ] AP: Loaded 5760 existing records from primary storage. [2025-09-16T17:12:19.720047] [INFO ] [MAIN ] AP: Loaded 5760 existing records. [2025-09-16T17:12:19.894742] [WARN ] [MAIN ] AP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:12:19.895015] [INFO ] [MAIN ] AP: Wiping existing primary data before full refresh. [2025-09-16T17:12:19.895068] [INFO ] [LOCAL ] AP: Starting deletion of primary data. [2025-09-16T17:12:19.895330] [DEBUG] [LOCAL ] AP: Deleted primary data file. [2025-09-16T17:12:19.895374] [INFO ] [MAIN ] AP: Starting full primary refresh. [2025-09-16T17:12:20.251264] [INFO ] [REMOTE ] AP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:12:20.380404] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-03. [2025-09-16T17:12:20.394433] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T17:12:21.174902] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-02. [2025-09-16T17:12:21.418134] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-09-16T17:12:21.443251] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-09-16T17:12:21.722362] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-09-16T17:12:21.974919] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-04. [2025-09-16T17:12:22.775006] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-03. [2025-09-16T17:12:23.276869] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-09-16T17:12:23.277359] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-09-16T17:12:23.574899] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-05. [2025-09-16T17:12:24.374906] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-05. [2025-09-16T17:12:24.922345] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T17:12:25.174907] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-06. [2025-09-16T17:12:25.914769] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T17:12:25.974865] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-06. [2025-09-16T17:12:26.423341] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T17:12:26.774945] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-07. [2025-09-16T17:12:27.563934] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T17:12:27.574847] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-07. [2025-09-16T17:12:27.791031] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T17:12:28.374891] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-07. [2025-09-16T17:12:29.174934] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-08. [2025-09-16T17:12:29.313103] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T17:12:29.834438] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T17:12:29.974916] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-08. [2025-09-16T17:12:30.572051] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-09-16T17:12:30.774914] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-04. [2025-09-16T17:12:31.147697] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-09-16T17:12:31.219549] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T17:12:31.574960] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-05. [2025-09-16T17:12:32.374931] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-09. [2025-09-16T17:12:32.399227] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T17:12:33.174917] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-09. [2025-09-16T17:12:33.712894] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T17:12:33.717334] [INFO ] [REMOTE ] AORT: Full fetch complete (32583 records). [2025-09-16T17:12:33.717459] [DEBUG] [VALIDATOR ] AORT: Validating primary data. [2025-09-16T17:12:33.983527] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-08. [2025-09-16T17:12:34.132844] [DEBUG] [VALIDATOR ] AORT: All expected trading days are present. [2025-09-16T17:12:34.132931] [DEBUG] [VALIDATOR ] AORT: Primary data validation passed. [2025-09-16T17:12:34.133345] [INFO ] [MAIN ] AORT: Fetched primary data is valid. [2025-09-16T17:12:34.133391] [DEBUG] [LOCAL ] AORT: Saving primary data. [2025-09-16T17:12:34.134318] [DEBUG] [LOCAL ] AORT: Trimmed 32583 records to 21726 (keeping 150 days of data) [2025-09-16T17:12:34.262481] [DEBUG] [LOCAL ] AORT: Primary data saved (21726 records). [2025-09-16T17:12:34.262612] [INFO ] [MAIN ] AORT: Full history saved. [2025-09-16T17:12:34.262713] [INFO ] [MAIN ] Update Progress: 124/2369 symbols completed. AORT took 29.3s. Estimated remaining: 3h 46m 40s. Rate: 9.9 symbols/min [2025-09-16T17:12:34.438330] [INFO ] [MAIN ] APAM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:12:34.464695] [DEBUG] [LOCAL ] APAM: Loaded 26710 existing records from primary storage. [2025-09-16T17:12:34.464853] [INFO ] [MAIN ] APAM: Loaded 26710 existing records. [2025-09-16T17:12:34.643042] [WARN ] [MAIN ] APAM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:12:34.643441] [INFO ] [MAIN ] APAM: Wiping existing primary data before full refresh. [2025-09-16T17:12:34.643587] [INFO ] [LOCAL ] APAM: Starting deletion of primary data. [2025-09-16T17:12:34.649935] [DEBUG] [LOCAL ] APAM: Deleted primary data file. [2025-09-16T17:12:34.658218] [INFO ] [MAIN ] APAM: Starting full primary refresh. [2025-09-16T17:12:34.664334] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T17:12:34.751549] [INFO ] [REMOTE ] AOS: Full fetch complete (53702 records). [2025-09-16T17:12:34.768003] [DEBUG] [VALIDATOR ] AOS: Validating primary data. [2025-09-16T17:12:34.790777] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-06. [2025-09-16T17:12:35.334567] [INFO ] [REMOTE ] APAM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:12:35.426564] [DEBUG] [VALIDATOR ] AOS: All expected trading days are present. [2025-09-16T17:12:35.426757] [DEBUG] [VALIDATOR ] AOS: Primary data validation passed. [2025-09-16T17:12:35.427363] [INFO ] [MAIN ] AOS: Fetched primary data is valid. [2025-09-16T17:12:35.427477] [DEBUG] [LOCAL ] AOS: Saving primary data. [2025-09-16T17:12:35.428744] [DEBUG] [LOCAL ] AOS: Trimmed 53702 records to 36605 (keeping 150 days of data) [2025-09-16T17:12:35.538707] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-09-16T17:12:35.570792] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T17:12:35.579989] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-02. [2025-09-16T17:12:35.710968] [DEBUG] [LOCAL ] AOS: Primary data saved (36605 records). [2025-09-16T17:12:35.711123] [INFO ] [MAIN ] AOS: Full history saved. [2025-09-16T17:12:35.714623] [INFO ] [MAIN ] Update Progress: 125/2369 symbols completed. AOS took 23.6s. Estimated remaining: 3h 45m 11s. Rate: 10.0 symbols/min [2025-09-16T17:12:35.889094] [INFO ] [MAIN ] APD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:12:35.923946] [DEBUG] [LOCAL ] APD: Loaded 33036 existing records from primary storage. [2025-09-16T17:12:35.924094] [INFO ] [MAIN ] APD: Loaded 33036 existing records. [2025-09-16T17:12:36.099169] [WARN ] [MAIN ] APD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:12:36.099434] [INFO ] [MAIN ] APD: Wiping existing primary data before full refresh. [2025-09-16T17:12:36.099479] [INFO ] [LOCAL ] APD: Starting deletion of primary data. [2025-09-16T17:12:36.100829] [DEBUG] [LOCAL ] APD: Deleted primary data file. [2025-09-16T17:12:36.100881] [INFO ] [MAIN ] APD: Starting full primary refresh. [2025-09-16T17:12:36.380007] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-09. [2025-09-16T17:12:36.465695] [INFO ] [REMOTE ] APD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:12:37.096523] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T17:12:37.174893] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-03. [2025-09-16T17:12:37.728448] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T17:12:37.733785] [INFO ] [REMOTE ] AON: Full fetch complete (49473 records). [2025-09-16T17:12:37.733889] [DEBUG] [VALIDATOR ] AON: Validating primary data. [2025-09-16T17:12:37.986148] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-07. [2025-09-16T17:12:38.121214] [DEBUG] [VALIDATOR ] AON: All expected trading days are present. [2025-09-16T17:12:38.121300] [DEBUG] [VALIDATOR ] AON: Primary data validation passed. [2025-09-16T17:12:38.121858] [INFO ] [MAIN ] AON: Fetched primary data is valid. [2025-09-16T17:12:38.121905] [DEBUG] [LOCAL ] AON: Saving primary data. [2025-09-16T17:12:38.122968] [DEBUG] [LOCAL ] AON: Trimmed 49473 records to 33740 (keeping 150 days of data) [2025-09-16T17:12:38.348391] [DEBUG] [LOCAL ] AON: Primary data saved (33740 records). [2025-09-16T17:12:38.348548] [INFO ] [MAIN ] AON: Full history saved. [2025-09-16T17:12:38.351020] [INFO ] [MAIN ] Update Progress: 126/2369 symbols completed. AON took 42.3s. Estimated remaining: 3h 44m 5s. Rate: 10.0 symbols/min [2025-09-16T17:12:38.532524] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T17:12:38.549494] [INFO ] [MAIN ] APG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:12:38.603329] [DEBUG] [LOCAL ] APG: Loaded 37503 existing records from primary storage. [2025-09-16T17:12:38.603500] [INFO ] [MAIN ] APG: Loaded 37503 existing records. [2025-09-16T17:12:38.735129] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-09-16T17:12:38.781426] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-08. [2025-09-16T17:12:38.804164] [WARN ] [MAIN ] APG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:12:38.804434] [INFO ] [MAIN ] APG: Wiping existing primary data before full refresh. [2025-09-16T17:12:38.804482] [INFO ] [LOCAL ] APG: Starting deletion of primary data. [2025-09-16T17:12:38.805979] [DEBUG] [LOCAL ] APG: Deleted primary data file. [2025-09-16T17:12:38.806038] [INFO ] [MAIN ] APG: Starting full primary refresh. [2025-09-16T17:12:39.167987] [INFO ] [REMOTE ] APG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:12:39.306279] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-09-16T17:12:39.574965] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-02. [2025-09-16T17:12:40.374977] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-02. [2025-09-16T17:12:40.935776] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T17:12:41.174914] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-03. [2025-09-16T17:12:41.933852] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T17:12:41.974887] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-03. [2025-09-16T17:12:42.638226] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T17:12:42.774876] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-04. [2025-09-16T17:12:43.574873] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-09. [2025-09-16T17:12:43.665414] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-09-16T17:12:44.371816] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T17:12:44.381418] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-04. [2025-09-16T17:12:44.386368] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-09-16T17:12:44.387985] [INFO ] [REMOTE ] AP: Full fetch complete (6999 records). [2025-09-16T17:12:44.388286] [DEBUG] [VALIDATOR ] AP: Validating primary data. [2025-09-16T17:12:44.764702] [DEBUG] [VALIDATOR ] AP: All expected trading days are present. [2025-09-16T17:12:44.764900] [DEBUG] [VALIDATOR ] AP: Primary data validation passed. [2025-09-16T17:12:44.765085] [INFO ] [MAIN ] AP: Fetched primary data is valid. [2025-09-16T17:12:44.765125] [DEBUG] [LOCAL ] AP: Saving primary data. [2025-09-16T17:12:44.765523] [DEBUG] [LOCAL ] AP: Trimmed 6999 records to 5815 (keeping 150 days of data) [2025-09-16T17:12:44.799605] [DEBUG] [LOCAL ] AP: Primary data saved (5815 records). [2025-09-16T17:12:44.799712] [INFO ] [MAIN ] AP: Full history saved. [2025-09-16T17:12:44.799793] [INFO ] [MAIN ] Update Progress: 127/2369 symbols completed. AP took 25.3s. Estimated remaining: 3h 44m 7s. Rate: 10.0 symbols/min [2025-09-16T17:12:44.975775] [INFO ] [MAIN ] APH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:12:45.016760] [DEBUG] [LOCAL ] APH: Loaded 39651 existing records from primary storage. [2025-09-16T17:12:45.016923] [INFO ] [MAIN ] APH: Loaded 39651 existing records. [2025-09-16T17:12:45.177028] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-04. [2025-09-16T17:12:45.200168] [WARN ] [MAIN ] APH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:12:45.200368] [INFO ] [MAIN ] APH: Wiping existing primary data before full refresh. [2025-09-16T17:12:45.200414] [INFO ] [LOCAL ] APH: Starting deletion of primary data. [2025-09-16T17:12:45.202013] [DEBUG] [LOCAL ] APH: Deleted primary data file. [2025-09-16T17:12:45.202068] [INFO ] [MAIN ] APH: Starting full primary refresh. [2025-09-16T17:12:45.572583] [INFO ] [REMOTE ] APH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:12:45.962393] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T17:12:45.974843] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-02. [2025-09-16T17:12:46.774934] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-05. [2025-09-16T17:12:46.943910] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T17:12:47.574903] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-05. [2025-09-16T17:12:47.935971] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-09-16T17:12:48.017112] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-09-16T17:12:48.374885] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-06. [2025-09-16T17:12:48.947125] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-09-16T17:12:49.174898] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-03. [2025-09-16T17:12:49.359404] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T17:12:49.974950] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-06. [2025-09-16T17:12:50.774935] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-07. [2025-09-16T17:12:51.042471] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-09-16T17:12:51.217981] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T17:12:51.574900] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-05. [2025-09-16T17:12:52.233301] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T17:12:52.374885] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-08. [2025-09-16T17:12:53.060024] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T17:12:53.174915] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-04. [2025-09-16T17:12:53.669164] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T17:12:53.974907] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-07. [2025-09-16T17:12:54.774902] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-06. [2025-09-16T17:12:55.051805] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T17:12:55.212343] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-09-16T17:12:55.574905] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-08. [2025-09-16T17:12:56.205371] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T17:12:56.374892] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-07. [2025-09-16T17:12:56.683875] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-09-16T17:12:57.174872] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-05. [2025-09-16T17:12:57.649384] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-09-16T17:12:57.974896] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-09. [2025-09-16T17:12:58.774925] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-08. [2025-09-16T17:12:59.115434] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-09-16T17:12:59.139030] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-09-16T17:12:59.143871] [INFO ] [REMOTE ] APD: Full fetch complete (49638 records). [2025-09-16T17:12:59.143961] [DEBUG] [VALIDATOR ] APD: Validating primary data. [2025-09-16T17:12:59.547550] [DEBUG] [VALIDATOR ] APD: All expected trading days are present. [2025-09-16T17:12:59.547654] [DEBUG] [VALIDATOR ] APD: Primary data validation passed. [2025-09-16T17:12:59.548174] [INFO ] [MAIN ] APD: Fetched primary data is valid. [2025-09-16T17:12:59.548218] [DEBUG] [LOCAL ] APD: Saving primary data. [2025-09-16T17:12:59.549410] [DEBUG] [LOCAL ] APD: Trimmed 49638 records to 33341 (keeping 150 days of data) [2025-09-16T17:12:59.612921] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-06. [2025-09-16T17:12:59.766839] [DEBUG] [LOCAL ] APD: Primary data saved (33341 records). [2025-09-16T17:12:59.767010] [INFO ] [MAIN ] APD: Full history saved. [2025-09-16T17:12:59.768132] [INFO ] [MAIN ] Update Progress: 128/2369 symbols completed. APD took 24.1s. Estimated remaining: 3h 46m 38s. Rate: 9.9 symbols/min [2025-09-16T17:12:59.941779] [INFO ] [MAIN ] APLE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:12:59.985744] [DEBUG] [LOCAL ] APLE: Loaded 38334 existing records from primary storage. [2025-09-16T17:12:59.985885] [INFO ] [MAIN ] APLE: Loaded 38334 existing records. [2025-09-16T17:13:00.169788] [WARN ] [MAIN ] APLE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:00.170107] [INFO ] [MAIN ] APLE: Wiping existing primary data before full refresh. [2025-09-16T17:13:00.170162] [INFO ] [LOCAL ] APLE: Starting deletion of primary data. [2025-09-16T17:13:00.171844] [DEBUG] [LOCAL ] APLE: Deleted primary data file. [2025-09-16T17:13:00.171899] [INFO ] [MAIN ] APLE: Starting full primary refresh. [2025-09-16T17:13:00.380812] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-09. [2025-09-16T17:13:00.389595] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-09-16T17:13:00.576626] [INFO ] [REMOTE ] APLE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:01.174889] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-09. [2025-09-16T17:13:01.493539] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-09-16T17:13:01.974901] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-07. [2025-09-16T17:13:02.071124] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-09-16T17:13:02.075072] [INFO ] [REMOTE ] APAM: Full fetch complete (39730 records). [2025-09-16T17:13:02.075188] [DEBUG] [VALIDATOR ] APAM: Validating primary data. [2025-09-16T17:13:02.075729] [FATAL] [VALIDATOR ] APAM: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:13:02.075806] [FATAL] [MAIN ] APAM: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:13:02.075843] [WARN ] [MAIN ] APAM: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:13:02.075889] [INFO ] [LOCAL ] APAM: Starting deletion of primary data. [2025-09-16T17:13:02.075946] [DEBUG] [LOCAL ] APAM: No primary data file found to delete. [2025-09-16T17:13:02.075983] [INFO ] [LOCAL ] APAM: Starting deletion of derivative data. [2025-09-16T17:13:02.076292] [DEBUG] [LOCAL ] APAM: Deleted derivative data file. [2025-09-16T17:13:02.076390] [INFO ] [MAIN ] Update Progress: 129/2369 symbols completed. APAM took 27.8s. Estimated remaining: 3h 45m 27s. Rate: 9.9 symbols/min [2025-09-16T17:13:02.248744] [INFO ] [MAIN ] APO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:13:02.293124] [DEBUG] [LOCAL ] APO: Loaded 39369 existing records from primary storage. [2025-09-16T17:13:02.293299] [INFO ] [MAIN ] APO: Loaded 39369 existing records. [2025-09-16T17:13:02.484770] [WARN ] [MAIN ] APO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:02.485059] [INFO ] [MAIN ] APO: Wiping existing primary data before full refresh. [2025-09-16T17:13:02.485109] [INFO ] [LOCAL ] APO: Starting deletion of primary data. [2025-09-16T17:13:02.487023] [DEBUG] [LOCAL ] APO: Deleted primary data file. [2025-09-16T17:13:02.487077] [INFO ] [MAIN ] APO: Starting full primary refresh. [2025-09-16T17:13:02.780061] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-02. [2025-09-16T17:13:02.848403] [INFO ] [REMOTE ] APO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:03.250283] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-09-16T17:13:03.256485] [INFO ] [REMOTE ] APG: Full fetch complete (54475 records). [2025-09-16T17:13:03.256590] [DEBUG] [VALIDATOR ] APG: Validating primary data. [2025-09-16T17:13:03.579367] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-02. [2025-09-16T17:13:03.673872] [DEBUG] [VALIDATOR ] APG: All expected trading days are present. [2025-09-16T17:13:03.673973] [DEBUG] [VALIDATOR ] APG: Primary data validation passed. [2025-09-16T17:13:03.674581] [INFO ] [MAIN ] APG: Fetched primary data is valid. [2025-09-16T17:13:03.674680] [DEBUG] [LOCAL ] APG: Saving primary data. [2025-09-16T17:13:03.676184] [DEBUG] [LOCAL ] APG: Trimmed 54475 records to 37860 (keeping 150 days of data) [2025-09-16T17:13:03.905393] [DEBUG] [LOCAL ] APG: Primary data saved (37860 records). [2025-09-16T17:13:03.905544] [INFO ] [MAIN ] APG: Full history saved. [2025-09-16T17:13:03.905655] [INFO ] [MAIN ] Update Progress: 130/2369 symbols completed. APG took 25.6s. Estimated remaining: 3h 44m 8s. Rate: 10.0 symbols/min [2025-09-16T17:13:04.089963] [INFO ] [MAIN ] APOS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:13:04.112433] [DEBUG] [LOCAL ] APOS: Loaded 5181 existing records from primary storage. [2025-09-16T17:13:04.122192] [INFO ] [MAIN ] APOS: Loaded 5181 existing records. [2025-09-16T17:13:04.129507] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-09-16T17:13:04.294782] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-07 (2.32 seconds). [2025-09-16T17:13:04.385260] [WARN ] [MAIN ] APOS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:04.379990] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-08. [2025-09-16T17:13:04.386432] [INFO ] [MAIN ] APOS: Wiping existing primary data before full refresh. [2025-09-16T17:13:04.386498] [INFO ] [LOCAL ] APOS: Starting deletion of primary data. [2025-09-16T17:13:04.386869] [DEBUG] [LOCAL ] APOS: Deleted primary data file. [2025-09-16T17:13:04.386929] [INFO ] [MAIN ] APOS: Starting full primary refresh. [2025-09-16T17:13:04.776899] [INFO ] [REMOTE ] APOS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:05.162553] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T17:13:05.174853] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-03. [2025-09-16T17:13:05.974873] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-03. [2025-09-16T17:13:06.558583] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-09-16T17:13:06.774893] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-09. [2025-09-16T17:13:06.819746] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T17:13:07.336265] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T17:13:07.574906] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-02. [2025-09-16T17:13:08.336069] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T17:13:08.374887] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-04. [2025-09-16T17:13:08.697987] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-09-16T17:13:08.704907] [INFO ] [REMOTE ] APH: Full fetch complete (58371 records). [2025-09-16T17:13:08.705076] [DEBUG] [VALIDATOR ] APH: Validating primary data. [2025-09-16T17:13:09.083538] [DEBUG] [VALIDATOR ] APH: All expected trading days are present. [2025-09-16T17:13:09.083630] [DEBUG] [VALIDATOR ] APH: Primary data validation passed. [2025-09-16T17:13:09.084306] [INFO ] [MAIN ] APH: Fetched primary data is valid. [2025-09-16T17:13:09.084353] [DEBUG] [LOCAL ] APH: Saving primary data. [2025-09-16T17:13:09.085675] [DEBUG] [LOCAL ] APH: Trimmed 58371 records to 40041 (keeping 150 days of data) [2025-09-16T17:13:09.177328] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-03. [2025-09-16T17:13:09.331948] [DEBUG] [LOCAL ] APH: Primary data saved (40041 records). [2025-09-16T17:13:09.332083] [INFO ] [MAIN ] APH: Full history saved. [2025-09-16T17:13:09.332857] [INFO ] [MAIN ] Update Progress: 131/2369 symbols completed. APH took 24.5s. Estimated remaining: 3h 43m 52s. Rate: 10.0 symbols/min [2025-09-16T17:13:09.506718] [INFO ] [MAIN ] APTV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:13:09.545678] [DEBUG] [LOCAL ] APTV: Loaded 38330 existing records from primary storage. [2025-09-16T17:13:09.545822] [INFO ] [MAIN ] APTV: Loaded 38330 existing records. [2025-09-16T17:13:09.720789] [WARN ] [MAIN ] APTV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:09.721048] [INFO ] [MAIN ] APTV: Wiping existing primary data before full refresh. [2025-09-16T17:13:09.721095] [INFO ] [LOCAL ] APTV: Starting deletion of primary data. [2025-09-16T17:13:09.722145] [DEBUG] [LOCAL ] APTV: Deleted primary data file. [2025-09-16T17:13:09.722195] [INFO ] [MAIN ] APTV: Starting full primary refresh. [2025-09-16T17:13:09.977161] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-04. [2025-09-16T17:13:10.092731] [INFO ] [REMOTE ] APTV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:10.135096] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T17:13:10.153097] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T17:13:10.774915] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-02. [2025-09-16T17:13:11.574914] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-05. [2025-09-16T17:13:11.816138] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T17:13:12.374909] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-05. [2025-09-16T17:13:12.461123] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T17:13:13.174971] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-04. [2025-09-16T17:13:13.520954] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-09-16T17:13:13.817196] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T17:13:13.916121] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-09-16T17:13:13.974870] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-05. [2025-09-16T17:13:14.691840] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-09-16T17:13:14.774898] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-06. [2025-09-16T17:13:15.574905] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-06. [2025-09-16T17:13:15.654546] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-09-16T17:13:16.374898] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-03. [2025-09-16T17:13:16.998466] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T17:13:17.174921] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-07. [2025-09-16T17:13:17.653070] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-09-16T17:13:17.850299] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-09-16T17:13:17.974891] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-06. [2025-09-16T17:13:18.774932] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-08. [2025-09-16T17:13:19.563161] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T17:13:19.574925] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-07. [2025-09-16T17:13:19.893998] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-09-16T17:13:20.374928] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-07. [2025-09-16T17:13:21.174933] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-04. [2025-09-16T17:13:21.585658] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-09-16T17:13:21.667715] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-09-16T17:13:21.974947] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-08. [2025-09-16T17:13:22.674537] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T17:13:22.774891] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-05. [2025-09-16T17:13:23.574979] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-08. [2025-09-16T17:13:23.622090] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T17:13:23.919394] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T17:13:24.374936] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-06. [2025-09-16T17:13:25.174890] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-09. [2025-09-16T17:13:25.262818] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T17:13:25.732263] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-09-16T17:13:25.974940] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-09. [2025-09-16T17:13:26.497981] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T17:13:26.504990] [INFO ] [REMOTE ] APO: Full fetch complete (58036 records). [2025-09-16T17:13:26.505201] [DEBUG] [VALIDATOR ] APO: Validating primary data. [2025-09-16T17:13:26.777998] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-07. [2025-09-16T17:13:26.817369] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T17:13:26.864108] [INFO ] [REMOTE ] APOS: Full fetch complete (8087 records). [2025-09-16T17:13:26.891970] [DEBUG] [VALIDATOR ] APOS: Validating primary data. [2025-09-16T17:13:26.905920] [DEBUG] [VALIDATOR ] APO: All expected trading days are present. [2025-09-16T17:13:26.906165] [DEBUG] [VALIDATOR ] APO: Primary data validation passed. [2025-09-16T17:13:26.979813] [INFO ] [MAIN ] APO: Fetched primary data is valid. [2025-09-16T17:13:26.991267] [DEBUG] [LOCAL ] APO: Saving primary data. [2025-09-16T17:13:27.035352] [DEBUG] [LOCAL ] APO: Trimmed 58036 records to 39757 (keeping 150 days of data) [2025-09-16T17:13:27.394860] [DEBUG] [VALIDATOR ] APOS: All expected trading days are present. [2025-09-16T17:13:27.396833] [DEBUG] [VALIDATOR ] APOS: Primary data validation passed. [2025-09-16T17:13:27.399304] [INFO ] [MAIN ] APOS: Fetched primary data is valid. [2025-09-16T17:13:27.400851] [DEBUG] [LOCAL ] APOS: Saving primary data. [2025-09-16T17:13:27.406705] [DEBUG] [LOCAL ] APOS: Trimmed 8087 records to 5235 (keeping 150 days of data) [2025-09-16T17:13:27.451674] [DEBUG] [LOCAL ] APOS: Primary data saved (5235 records). [2025-09-16T17:13:27.456035] [INFO ] [MAIN ] APOS: Full history saved. [2025-09-16T17:13:27.457630] [INFO ] [MAIN ] Update Progress: 132/2369 symbols completed. APOS took 23.6s. Estimated remaining: 3h 47m 12s. Rate: 9.8 symbols/min [2025-09-16T17:13:27.632354] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-09. [2025-09-16T17:13:27.728870] [INFO ] [MAIN ] AQN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:13:27.780333] [DEBUG] [LOCAL ] AQN: Loaded 38142 existing records from primary storage. [2025-09-16T17:13:27.782353] [INFO ] [MAIN ] AQN: Loaded 38142 existing records. [2025-09-16T17:13:27.855486] [DEBUG] [LOCAL ] APO: Primary data saved (39757 records). [2025-09-16T17:13:27.877134] [INFO ] [MAIN ] APO: Full history saved. [2025-09-16T17:13:27.887660] [INFO ] [MAIN ] Update Progress: 133/2369 symbols completed. APO took 25.8s. Estimated remaining: 3h 45m 30s. Rate: 9.9 symbols/min [2025-09-16T17:13:28.075207] [WARN ] [MAIN ] AQN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:28.080598] [INFO ] [MAIN ] AQN: Wiping existing primary data before full refresh. [2025-09-16T17:13:28.091100] [INFO ] [LOCAL ] AQN: Starting deletion of primary data. [2025-09-16T17:13:28.107736] [DEBUG] [LOCAL ] AQN: Deleted primary data file. [2025-09-16T17:13:28.149445] [INFO ] [MAIN ] AQN: Starting full primary refresh. [2025-09-16T17:13:28.150835] [INFO ] [MAIN ] AQNB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:13:28.256084] [DEBUG] [LOCAL ] AQNB: Loaded 2896 existing records from primary storage. [2025-09-16T17:13:28.266739] [INFO ] [MAIN ] AQNB: Loaded 2896 existing records. [2025-09-16T17:13:28.602604] [WARN ] [MAIN ] AQNB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:28.613552] [INFO ] [MAIN ] AQNB: Wiping existing primary data before full refresh. [2025-09-16T17:13:28.613731] [INFO ] [LOCAL ] AQNB: Starting deletion of primary data. [2025-09-16T17:13:28.639776] [DEBUG] [LOCAL ] AQNB: Deleted primary data file. [2025-09-16T17:13:28.664206] [INFO ] [MAIN ] AQNB: Starting full primary refresh. [2025-09-16T17:13:28.696279] [INFO ] [REMOTE ] AQN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:28.696749] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-02. [2025-09-16T17:13:29.083971] [INFO ] [REMOTE ] AQNB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:29.088005] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-07 (2.31 seconds). [2025-09-16T17:13:29.174930] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-02. [2025-09-16T17:13:29.410166] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T17:13:29.416509] [INFO ] [REMOTE ] APLE: Full fetch complete (56360 records). [2025-09-16T17:13:29.416671] [DEBUG] [VALIDATOR ] APLE: Validating primary data. [2025-09-16T17:13:29.740763] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-09-16T17:13:29.833011] [DEBUG] [VALIDATOR ] APLE: All expected trading days are present. [2025-09-16T17:13:29.833100] [DEBUG] [VALIDATOR ] APLE: Primary data validation passed. [2025-09-16T17:13:29.833727] [INFO ] [MAIN ] APLE: Fetched primary data is valid. [2025-09-16T17:13:29.833779] [DEBUG] [LOCAL ] APLE: Saving primary data. [2025-09-16T17:13:29.835093] [DEBUG] [LOCAL ] APLE: Trimmed 56360 records to 38699 (keeping 150 days of data) [2025-09-16T17:13:30.014872] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-08. [2025-09-16T17:13:30.079405] [DEBUG] [LOCAL ] APLE: Primary data saved (38699 records). [2025-09-16T17:13:30.079549] [INFO ] [MAIN ] APLE: Full history saved. [2025-09-16T17:13:30.080706] [INFO ] [MAIN ] Update Progress: 134/2369 symbols completed. APLE took 30.3s. Estimated remaining: 3h 44m 20s. Rate: 10.0 symbols/min [2025-09-16T17:13:30.258695] [INFO ] [MAIN ] AR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:13:30.305822] [DEBUG] [LOCAL ] AR: Loaded 39562 existing records from primary storage. [2025-09-16T17:13:30.306009] [INFO ] [MAIN ] AR: Loaded 39562 existing records. [2025-09-16T17:13:30.493501] [WARN ] [MAIN ] AR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:30.493814] [INFO ] [MAIN ] AR: Wiping existing primary data before full refresh. [2025-09-16T17:13:30.493874] [INFO ] [LOCAL ] AR: Starting deletion of primary data. [2025-09-16T17:13:30.495513] [DEBUG] [LOCAL ] AR: Deleted primary data file. [2025-09-16T17:13:30.495572] [INFO ] [MAIN ] AR: Starting full primary refresh. [2025-09-16T17:13:30.783891] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-03. [2025-09-16T17:13:30.909539] [INFO ] [REMOTE ] AR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:30.937844] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-09-16T17:13:31.517562] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T17:13:31.515223] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-09-16T17:13:31.574921] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-09. [2025-09-16T17:13:32.374911] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-04. [2025-09-16T17:13:32.969549] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-09-16T17:13:33.036439] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-09-16T17:13:33.042941] [INFO ] [REMOTE ] APTV: Full fetch complete (56894 records). [2025-09-16T17:13:33.043076] [DEBUG] [VALIDATOR ] APTV: Validating primary data. [2025-09-16T17:13:33.190758] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-05. [2025-09-16T17:13:33.442119] [DEBUG] [VALIDATOR ] APTV: All expected trading days are present. [2025-09-16T17:13:33.442224] [DEBUG] [VALIDATOR ] APTV: Primary data validation passed. [2025-09-16T17:13:33.442928] [INFO ] [MAIN ] APTV: Fetched primary data is valid. [2025-09-16T17:13:33.442994] [DEBUG] [LOCAL ] APTV: Saving primary data. [2025-09-16T17:13:33.444872] [DEBUG] [LOCAL ] APTV: Trimmed 56894 records to 38716 (keeping 150 days of data) [2025-09-16T17:13:33.679272] [DEBUG] [LOCAL ] APTV: Primary data saved (38716 records). [2025-09-16T17:13:33.679409] [INFO ] [MAIN ] APTV: Full history saved. [2025-09-16T17:13:33.680094] [INFO ] [MAIN ] Update Progress: 135/2369 symbols completed. APTV took 24.3s. Estimated remaining: 3h 43m 34s. Rate: 10.0 symbols/min [2025-09-16T17:13:33.856327] [INFO ] [MAIN ] ARCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:13:33.857247] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-09-16T17:13:33.893051] [DEBUG] [LOCAL ] ARCO: Loaded 32872 existing records from primary storage. [2025-09-16T17:13:33.893209] [INFO ] [MAIN ] ARCO: Loaded 32872 existing records. [2025-09-16T17:13:33.980206] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-06. [2025-09-16T17:13:34.075933] [WARN ] [MAIN ] ARCO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:34.076173] [INFO ] [MAIN ] ARCO: Wiping existing primary data before full refresh. [2025-09-16T17:13:34.076220] [INFO ] [LOCAL ] ARCO: Starting deletion of primary data. [2025-09-16T17:13:34.077392] [DEBUG] [LOCAL ] ARCO: Deleted primary data file. [2025-09-16T17:13:34.077446] [INFO ] [MAIN ] ARCO: Starting full primary refresh. [2025-09-16T17:13:34.483540] [INFO ] [REMOTE ] ARCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:34.488746] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-09-16T17:13:34.774894] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-02. [2025-09-16T17:13:35.574902] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-07. [2025-09-16T17:13:35.920263] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-09-16T17:13:36.092328] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-09-16T17:13:36.374938] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-02. [2025-09-16T17:13:37.174910] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-08. [2025-09-16T17:13:37.730820] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-09-16T17:13:37.974905] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-09. [2025-09-16T17:13:38.075159] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-09-16T17:13:38.463357] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-09-16T17:13:38.464342] [INFO ] [REMOTE ] AQNB: Full fetch complete (4410 records). [2025-09-16T17:13:38.464410] [DEBUG] [VALIDATOR ] AQNB: Validating primary data. [2025-09-16T17:13:38.780006] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-03. [2025-09-16T17:13:38.818947] [DEBUG] [VALIDATOR ] AQNB: All expected trading days are present. [2025-09-16T17:13:38.819035] [DEBUG] [VALIDATOR ] AQNB: Primary data validation passed. [2025-09-16T17:13:38.819238] [INFO ] [MAIN ] AQNB: Fetched primary data is valid. [2025-09-16T17:13:38.819289] [DEBUG] [LOCAL ] AQNB: Saving primary data. [2025-09-16T17:13:38.819916] [DEBUG] [LOCAL ] AQNB: Trimmed 4410 records to 2917 (keeping 150 days of data) [2025-09-16T17:13:38.838957] [DEBUG] [LOCAL ] AQNB: Primary data saved (2917 records). [2025-09-16T17:13:38.839026] [INFO ] [MAIN ] AQNB: Full history saved. [2025-09-16T17:13:38.839122] [INFO ] [MAIN ] Update Progress: 136/2369 symbols completed. AQNB took 11.0s. Estimated remaining: 3h 43m 14s. Rate: 10.0 symbols/min [2025-09-16T17:13:39.008085] [INFO ] [MAIN ] ARDC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:13:39.023277] [DEBUG] [LOCAL ] ARDC: Loaded 10313 existing records from primary storage. [2025-09-16T17:13:39.023397] [INFO ] [MAIN ] ARDC: Loaded 10313 existing records. [2025-09-16T17:13:39.199525] [WARN ] [MAIN ] ARDC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:39.199767] [INFO ] [MAIN ] ARDC: Wiping existing primary data before full refresh. [2025-09-16T17:13:39.199812] [INFO ] [LOCAL ] ARDC: Starting deletion of primary data. [2025-09-16T17:13:39.200363] [DEBUG] [LOCAL ] ARDC: Deleted primary data file. [2025-09-16T17:13:39.200409] [INFO ] [MAIN ] ARDC: Starting full primary refresh. [2025-09-16T17:13:39.547631] [INFO ] [REMOTE ] ARDC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:39.574861] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-03. [2025-09-16T17:13:40.374905] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-02. [2025-09-16T17:13:40.521449] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T17:13:41.174906] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-03. [2025-09-16T17:13:41.381558] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T17:13:41.401402] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-09-16T17:13:41.974908] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-03. [2025-09-16T17:13:42.461337] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-09-16T17:13:42.779877] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-04. [2025-09-16T17:13:42.784487] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-09-16T17:13:43.574900] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-04. [2025-09-16T17:13:44.273409] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T17:13:44.316256] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-09-16T17:13:44.374882] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-04. [2025-09-16T17:13:45.174900] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-05. [2025-09-16T17:13:45.781690] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-09-16T17:13:45.974908] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-05. [2025-09-16T17:13:46.774913] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-04. [2025-09-16T17:13:46.864278] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-09-16T17:13:47.444685] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T17:13:47.574918] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-06. [2025-09-16T17:13:48.374905] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-05. [2025-09-16T17:13:48.649977] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T17:13:49.179997] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-06. [2025-09-16T17:13:49.196972] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T17:13:49.234454] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T17:13:49.974904] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-05. [2025-09-16T17:13:50.657674] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T17:13:50.774907] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-07. [2025-09-16T17:13:51.311416] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T17:13:51.574972] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-07. [2025-09-16T17:13:52.374971] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-06. [2025-09-16T17:13:52.448747] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-09-16T17:13:53.133241] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-09-16T17:13:53.174858] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-08. [2025-09-16T17:13:53.974907] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-06. [2025-09-16T17:13:54.078067] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T17:13:54.456028] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-09-16T17:13:54.714269] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-09-16T17:13:54.774891] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-07. [2025-09-16T17:13:55.574913] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-09. [2025-09-16T17:13:55.831388] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-09-16T17:13:56.374896] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-08. [2025-09-16T17:13:56.991766] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T17:13:56.997251] [INFO ] [REMOTE ] ARCO: Full fetch complete (48541 records). [2025-09-16T17:13:56.997362] [DEBUG] [VALIDATOR ] ARCO: Validating primary data. [2025-09-16T17:13:57.175623] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-08. [2025-09-16T17:13:57.386607] [DEBUG] [VALIDATOR ] ARCO: All expected trading days are present. [2025-09-16T17:13:57.386742] [DEBUG] [VALIDATOR ] ARCO: Primary data validation passed. [2025-09-16T17:13:57.387365] [INFO ] [MAIN ] ARCO: Fetched primary data is valid. [2025-09-16T17:13:57.387418] [DEBUG] [LOCAL ] ARCO: Saving primary data. [2025-09-16T17:13:57.388708] [DEBUG] [LOCAL ] ARCO: Trimmed 48541 records to 33170 (keeping 150 days of data) [2025-09-16T17:13:57.590236] [DEBUG] [LOCAL ] ARCO: Primary data saved (33170 records). [2025-09-16T17:13:57.590391] [INFO ] [MAIN ] ARCO: Full history saved. [2025-09-16T17:13:57.591467] [INFO ] [MAIN ] Update Progress: 137/2369 symbols completed. ARCO took 23.9s. Estimated remaining: 3h 46m 36s. Rate: 9.9 symbols/min [2025-09-16T17:13:57.773715] [INFO ] [MAIN ] ARDT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:13:57.807243] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-09-16T17:13:57.831114] [DEBUG] [LOCAL ] ARDT: Loaded 19778 existing records from primary storage. [2025-09-16T17:13:57.831270] [INFO ] [MAIN ] ARDT: Loaded 19778 existing records. [2025-09-16T17:13:57.980491] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-07. [2025-09-16T17:13:58.018780] [WARN ] [MAIN ] ARDT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:13:58.018986] [INFO ] [MAIN ] ARDT: Wiping existing primary data before full refresh. [2025-09-16T17:13:58.019048] [INFO ] [LOCAL ] ARDT: Starting deletion of primary data. [2025-09-16T17:13:58.019684] [DEBUG] [LOCAL ] ARDT: Deleted primary data file. [2025-09-16T17:13:58.019741] [INFO ] [MAIN ] ARDT: Starting full primary refresh. [2025-09-16T17:13:58.365064] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-09-16T17:13:58.388401] [INFO ] [REMOTE ] ARDT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:13:58.774907] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-02. [2025-09-16T17:13:59.574875] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-09. [2025-09-16T17:13:59.776818] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-09-16T17:14:00.041772] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T17:14:00.374907] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-09. [2025-09-16T17:14:01.174921] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-08. [2025-09-16T17:14:01.594171] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-09-16T17:14:01.601235] [INFO ] [REMOTE ] AQN: Full fetch complete (56582 records). [2025-09-16T17:14:01.601395] [DEBUG] [VALIDATOR ] AQN: Validating primary data. [2025-09-16T17:14:01.740156] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T17:14:01.810609] [INFO ] [REMOTE ] ARDC: Full fetch complete (16030 records). [2025-09-16T17:14:01.817745] [DEBUG] [VALIDATOR ] ARDC: Validating primary data. [2025-09-16T17:14:01.993921] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-03. [2025-09-16T17:14:02.325194] [DEBUG] [VALIDATOR ] AQN: All expected trading days are present. [2025-09-16T17:14:02.351211] [DEBUG] [VALIDATOR ] AQN: Primary data validation passed. [2025-09-16T17:14:02.354387] [INFO ] [MAIN ] AQN: Fetched primary data is valid. [2025-09-16T17:14:02.354707] [DEBUG] [LOCAL ] AQN: Saving primary data. [2025-09-16T17:14:02.434691] [DEBUG] [LOCAL ] AQN: Trimmed 56582 records to 38517 (keeping 150 days of data) [2025-09-16T17:14:02.434864] [DEBUG] [VALIDATOR ] ARDC: All expected trading days are present. [2025-09-16T17:14:02.434968] [DEBUG] [VALIDATOR ] ARDC: Primary data validation passed. [2025-09-16T17:14:02.435390] [INFO ] [MAIN ] ARDC: Fetched primary data is valid. [2025-09-16T17:14:02.435462] [DEBUG] [LOCAL ] ARDC: Saving primary data. [2025-09-16T17:14:02.436231] [DEBUG] [LOCAL ] ARDC: Trimmed 16030 records to 10395 (keeping 150 days of data) [2025-09-16T17:14:02.607437] [DEBUG] [LOCAL ] ARDC: Primary data saved (10395 records). [2025-09-16T17:14:02.608259] [INFO ] [MAIN ] ARDC: Full history saved. [2025-09-16T17:14:02.608842] [INFO ] [MAIN ] Update Progress: 138/2369 symbols completed. ARDC took 23.8s. Estimated remaining: 3h 46m 12s. Rate: 9.9 symbols/min [2025-09-16T17:14:02.790044] [INFO ] [MAIN ] ARE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:02.923196] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T17:14:02.941200] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-09. [2025-09-16T17:14:02.954922] [DEBUG] [LOCAL ] ARE: Loaded 37553 existing records from primary storage. [2025-09-16T17:14:02.955517] [INFO ] [MAIN ] ARE: Loaded 37553 existing records. [2025-09-16T17:14:03.144268] [WARN ] [MAIN ] ARE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:03.145658] [INFO ] [MAIN ] ARE: Wiping existing primary data before full refresh. [2025-09-16T17:14:03.146819] [INFO ] [LOCAL ] ARE: Starting deletion of primary data. [2025-09-16T17:14:03.151407] [DEBUG] [LOCAL ] ARE: Deleted primary data file. [2025-09-16T17:14:03.153602] [DEBUG] [LOCAL ] AQN: Primary data saved (38517 records). [2025-09-16T17:14:03.153844] [INFO ] [MAIN ] AQN: Full history saved. [2025-09-16T17:14:03.153990] [INFO ] [MAIN ] Update Progress: 139/2369 symbols completed. AQN took 35.7s. Estimated remaining: 3h 44m 37s. Rate: 9.9 symbols/min [2025-09-16T17:14:03.159342] [INFO ] [MAIN ] ARE: Starting full primary refresh. [2025-09-16T17:14:03.398531] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T17:14:03.556365] [INFO ] [MAIN ] ARES: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:03.582602] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-04. [2025-09-16T17:14:03.742700] [INFO ] [REMOTE ] ARE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:03.767781] [DEBUG] [LOCAL ] ARES: Loaded 36190 existing records from primary storage. [2025-09-16T17:14:03.767915] [INFO ] [MAIN ] ARES: Loaded 36190 existing records. [2025-09-16T17:14:03.945588] [WARN ] [MAIN ] ARES: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:03.945841] [INFO ] [MAIN ] ARES: Wiping existing primary data before full refresh. [2025-09-16T17:14:03.945889] [INFO ] [LOCAL ] ARES: Starting deletion of primary data. [2025-09-16T17:14:03.947368] [DEBUG] [LOCAL ] ARES: Deleted primary data file. [2025-09-16T17:14:03.947422] [INFO ] [MAIN ] ARES: Starting full primary refresh. [2025-09-16T17:14:04.307993] [INFO ] [REMOTE ] ARES: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:04.374872] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-02. [2025-09-16T17:14:04.548183] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T17:14:04.555211] [INFO ] [REMOTE ] AR: Full fetch complete (58263 records). [2025-09-16T17:14:04.555312] [DEBUG] [VALIDATOR ] AR: Validating primary data. [2025-09-16T17:14:04.950090] [DEBUG] [VALIDATOR ] AR: All expected trading days are present. [2025-09-16T17:14:04.950672] [DEBUG] [VALIDATOR ] AR: Primary data validation passed. [2025-09-16T17:14:04.951264] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T17:14:04.952117] [INFO ] [MAIN ] AR: Fetched primary data is valid. [2025-09-16T17:14:04.952206] [DEBUG] [LOCAL ] AR: Saving primary data. [2025-09-16T17:14:04.960766] [DEBUG] [LOCAL ] AR: Trimmed 58263 records to 39952 (keeping 150 days of data) [2025-09-16T17:14:05.176118] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-02. [2025-09-16T17:14:05.198931] [DEBUG] [LOCAL ] AR: Primary data saved (39952 records). [2025-09-16T17:14:05.199035] [INFO ] [MAIN ] AR: Full history saved. [2025-09-16T17:14:05.201924] [INFO ] [MAIN ] Update Progress: 140/2369 symbols completed. AR took 35.1s. Estimated remaining: 3h 43m 28s. Rate: 10.0 symbols/min [2025-09-16T17:14:05.374623] [INFO ] [MAIN ] ARI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:05.409475] [DEBUG] [LOCAL ] ARI: Loaded 31741 existing records from primary storage. [2025-09-16T17:14:05.409687] [INFO ] [MAIN ] ARI: Loaded 31741 existing records. [2025-09-16T17:14:05.602459] [WARN ] [MAIN ] ARI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:05.602700] [INFO ] [MAIN ] ARI: Wiping existing primary data before full refresh. [2025-09-16T17:14:05.602750] [INFO ] [LOCAL ] ARI: Starting deletion of primary data. [2025-09-16T17:14:05.603765] [DEBUG] [LOCAL ] ARI: Deleted primary data file. [2025-09-16T17:14:05.603821] [INFO ] [MAIN ] ARI: Starting full primary refresh. [2025-09-16T17:14:05.960234] [INFO ] [REMOTE ] ARI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:05.974849] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-02. [2025-09-16T17:14:06.320510] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-09-16T17:14:06.759396] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T17:14:06.774845] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-03. [2025-09-16T17:14:07.387558] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-09-16T17:14:07.574915] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-03. [2025-09-16T17:14:07.990687] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-09-16T17:14:08.374947] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-03. [2025-09-16T17:14:09.180253] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-04. [2025-09-16T17:14:09.201599] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T17:14:09.895000] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T17:14:09.974940] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-04. [2025-09-16T17:14:10.496509] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T17:14:10.774954] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-05. [2025-09-16T17:14:11.574948] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-04. [2025-09-16T17:14:11.817540] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T17:14:11.954564] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-09-16T17:14:12.374941] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-05. [2025-09-16T17:14:13.071312] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T17:14:13.174924] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-06. [2025-09-16T17:14:13.818629] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T17:14:13.942827] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T17:14:13.974867] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-07. [2025-09-16T17:14:14.774906] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-05. [2025-09-16T17:14:15.400967] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-09-16T17:14:15.574895] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-08. [2025-09-16T17:14:16.374910] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-05. [2025-09-16T17:14:16.415211] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T17:14:16.684083] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-09-16T17:14:17.174922] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-09. [2025-09-16T17:14:17.974934] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-06. [2025-09-16T17:14:18.218067] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-09-16T17:14:18.221426] [INFO ] [REMOTE ] ARDT: Full fetch complete (31416 records). [2025-09-16T17:14:18.221506] [DEBUG] [VALIDATOR ] ARDT: Validating primary data. [2025-09-16T17:14:18.548512] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-09-16T17:14:18.650984] [DEBUG] [VALIDATOR ] ARDT: All expected trading days are present. [2025-09-16T17:14:18.651061] [DEBUG] [VALIDATOR ] ARDT: Primary data validation passed. [2025-09-16T17:14:18.651436] [INFO ] [MAIN ] ARDT: Fetched primary data is valid. [2025-09-16T17:14:18.651543] [DEBUG] [LOCAL ] ARDT: Saving primary data. [2025-09-16T17:14:18.652432] [DEBUG] [LOCAL ] ARDT: Trimmed 31416 records to 20016 (keeping 150 days of data) [2025-09-16T17:14:18.771012] [DEBUG] [LOCAL ] ARDT: Primary data saved (20016 records). [2025-09-16T17:14:18.771138] [INFO ] [MAIN ] ARDT: Full history saved. [2025-09-16T17:14:18.771220] [INFO ] [MAIN ] Update Progress: 141/2369 symbols completed. ARDT took 21.2s. Estimated remaining: 3h 45m 21s. Rate: 9.9 symbols/min [2025-09-16T17:14:18.782592] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-06. [2025-09-16T17:14:18.945667] [INFO ] [MAIN ] ARIS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:18.977560] [DEBUG] [LOCAL ] ARIS: Loaded 30434 existing records from primary storage. [2025-09-16T17:14:18.977755] [INFO ] [MAIN ] ARIS: Loaded 30434 existing records. [2025-09-16T17:14:19.155332] [WARN ] [MAIN ] ARIS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:19.155611] [INFO ] [MAIN ] ARIS: Wiping existing primary data before full refresh. [2025-09-16T17:14:19.155681] [INFO ] [LOCAL ] ARIS: Starting deletion of primary data. [2025-09-16T17:14:19.156889] [DEBUG] [LOCAL ] ARIS: Deleted primary data file. [2025-09-16T17:14:19.156944] [INFO ] [MAIN ] ARIS: Starting full primary refresh. [2025-09-16T17:14:19.514940] [INFO ] [REMOTE ] ARIS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:19.540618] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T17:14:19.574890] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-07. [2025-09-16T17:14:20.374874] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-06. [2025-09-16T17:14:20.440739] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T17:14:20.944150] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T17:14:21.174900] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-07. [2025-09-16T17:14:21.976961] [INFO ] [REMOTE ] ARIS: Fetching primary data for 2025-02. [2025-09-16T17:14:21.980266] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-09-16T17:14:22.778299] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-07. [2025-09-16T17:14:22.780821] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-09-16T17:14:23.509487] [INFO ] [REMOTE ] ARIS: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T17:14:23.574862] [INFO ] [REMOTE ] ARIS: Fetching primary data for 2025-03. [2025-09-16T17:14:24.143767] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T17:14:24.374913] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-08. [2025-09-16T17:14:24.720417] [INFO ] [REMOTE ] ARIS: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T17:14:25.174917] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-08. [2025-09-16T17:14:25.934696] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T17:14:25.974916] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-09. [2025-09-16T17:14:26.710201] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T17:14:26.774920] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-09. [2025-09-16T17:14:27.581566] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-08. [2025-09-16T17:14:27.588884] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T17:14:27.597259] [INFO ] [REMOTE ] ARE: Full fetch complete (54815 records). [2025-09-16T17:14:27.597508] [DEBUG] [VALIDATOR ] ARE: Validating primary data. [2025-09-16T17:14:28.014271] [DEBUG] [VALIDATOR ] ARE: All expected trading days are present. [2025-09-16T17:14:28.014378] [DEBUG] [VALIDATOR ] ARE: Primary data validation passed. [2025-09-16T17:14:28.015064] [INFO ] [MAIN ] ARE: Fetched primary data is valid. [2025-09-16T17:14:28.015119] [DEBUG] [LOCAL ] ARE: Saving primary data. [2025-09-16T17:14:28.016531] [DEBUG] [LOCAL ] ARE: Trimmed 54815 records to 37881 (keeping 150 days of data) [2025-09-16T17:14:28.250158] [DEBUG] [LOCAL ] ARE: Primary data saved (37881 records). [2025-09-16T17:14:28.250320] [INFO ] [MAIN ] ARE: Full history saved. [2025-09-16T17:14:28.250410] [INFO ] [MAIN ] Update Progress: 142/2369 symbols completed. ARE took 25.6s. Estimated remaining: 3h 46m 8s. Rate: 9.8 symbols/min [2025-09-16T17:14:28.377449] [INFO ] [REMOTE ] ARIS: Fetching primary data for 2025-04. [2025-09-16T17:14:28.430884] [INFO ] [MAIN ] ARLO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:28.449252] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-09-16T17:14:28.455171] [INFO ] [REMOTE ] ARES: Full fetch complete (53920 records). [2025-09-16T17:14:28.455270] [DEBUG] [VALIDATOR ] ARES: Validating primary data. [2025-09-16T17:14:28.755743] [DEBUG] [LOCAL ] ARLO: Loaded 32320 existing records from primary storage. [2025-09-16T17:14:28.774068] [INFO ] [MAIN ] ARLO: Loaded 32320 existing records. [2025-09-16T17:14:28.975223] [DEBUG] [VALIDATOR ] ARES: All expected trading days are present. [2025-09-16T17:14:28.975365] [DEBUG] [VALIDATOR ] ARES: Primary data validation passed. [2025-09-16T17:14:28.976285] [INFO ] [MAIN ] ARES: Fetched primary data is valid. [2025-09-16T17:14:28.981704] [DEBUG] [LOCAL ] ARES: Saving primary data. [2025-09-16T17:14:29.055029] [WARN ] [MAIN ] ARLO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:29.055198] [DEBUG] [LOCAL ] ARES: Trimmed 53920 records to 36531 (keeping 150 days of data) [2025-09-16T17:14:29.078477] [INFO ] [MAIN ] ARLO: Wiping existing primary data before full refresh. [2025-09-16T17:14:29.080860] [INFO ] [LOCAL ] ARLO: Starting deletion of primary data. [2025-09-16T17:14:29.082215] [DEBUG] [LOCAL ] ARLO: Deleted primary data file. [2025-09-16T17:14:29.084730] [INFO ] [MAIN ] ARLO: Starting full primary refresh. [2025-09-16T17:14:29.084620] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T17:14:29.236592] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-09. [2025-09-16T17:14:29.567151] [INFO ] [REMOTE ] ARLO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:29.700332] [INFO ] [REMOTE ] ARIS: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T17:14:29.735478] [DEBUG] [LOCAL ] ARES: Primary data saved (36531 records). [2025-09-16T17:14:29.735697] [INFO ] [MAIN ] ARES: Full history saved. [2025-09-16T17:14:29.735809] [INFO ] [MAIN ] Update Progress: 143/2369 symbols completed. ARES took 26.6s. Estimated remaining: 3h 44m 51s. Rate: 9.9 symbols/min [2025-09-16T17:14:29.907350] [INFO ] [MAIN ] ARMK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:29.946763] [DEBUG] [LOCAL ] ARMK: Loaded 37030 existing records from primary storage. [2025-09-16T17:14:29.946910] [INFO ] [MAIN ] ARMK: Loaded 37030 existing records. [2025-09-16T17:14:29.980129] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-02. [2025-09-16T17:14:30.134070] [WARN ] [MAIN ] ARMK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:30.134290] [INFO ] [MAIN ] ARMK: Wiping existing primary data before full refresh. [2025-09-16T17:14:30.134341] [INFO ] [LOCAL ] ARMK: Starting deletion of primary data. [2025-09-16T17:14:30.135945] [DEBUG] [LOCAL ] ARMK: Deleted primary data file. [2025-09-16T17:14:30.136001] [INFO ] [MAIN ] ARMK: Starting full primary refresh. [2025-09-16T17:14:30.494921] [INFO ] [REMOTE ] ARMK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:30.774937] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-02. [2025-09-16T17:14:30.950768] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T17:14:30.956029] [INFO ] [REMOTE ] ARI: Full fetch complete (46884 records). [2025-09-16T17:14:30.956179] [DEBUG] [VALIDATOR ] ARI: Validating primary data. [2025-09-16T17:14:31.366523] [DEBUG] [VALIDATOR ] ARI: All expected trading days are present. [2025-09-16T17:14:31.366711] [DEBUG] [VALIDATOR ] ARI: Primary data validation passed. [2025-09-16T17:14:31.367475] [INFO ] [MAIN ] ARI: Fetched primary data is valid. [2025-09-16T17:14:31.367564] [DEBUG] [LOCAL ] ARI: Saving primary data. [2025-09-16T17:14:31.369361] [DEBUG] [LOCAL ] ARI: Trimmed 46884 records to 32069 (keeping 150 days of data) [2025-09-16T17:14:31.503168] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T17:14:31.576162] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-03. [2025-09-16T17:14:31.598062] [DEBUG] [LOCAL ] ARI: Primary data saved (32069 records). [2025-09-16T17:14:31.598225] [INFO ] [MAIN ] ARI: Full history saved. [2025-09-16T17:14:31.600619] [INFO ] [MAIN ] Update Progress: 144/2369 symbols completed. ARI took 26.4s. Estimated remaining: 3h 43m 40s. Rate: 9.9 symbols/min [2025-09-16T17:14:31.779041] [INFO ] [MAIN ] AROC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:31.825802] [DEBUG] [LOCAL ] AROC: Loaded 35804 existing records from primary storage. [2025-09-16T17:14:31.825976] [INFO ] [MAIN ] AROC: Loaded 35804 existing records. [2025-09-16T17:14:32.002611] [WARN ] [MAIN ] AROC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:32.002881] [INFO ] [MAIN ] AROC: Wiping existing primary data before full refresh. [2025-09-16T17:14:32.002928] [INFO ] [LOCAL ] AROC: Starting deletion of primary data. [2025-09-16T17:14:32.004204] [DEBUG] [LOCAL ] AROC: Deleted primary data file. [2025-09-16T17:14:32.004257] [INFO ] [MAIN ] AROC: Starting full primary refresh. [2025-09-16T17:14:32.367379] [INFO ] [REMOTE ] AROC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:32.379936] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-02. [2025-09-16T17:14:32.395663] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T17:14:33.018077] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T17:14:33.175014] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-03. [2025-09-16T17:14:33.761200] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T17:14:33.974954] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-03. [2025-09-16T17:14:34.774986] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-04. [2025-09-16T17:14:34.942791] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T17:14:35.574875] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-04. [2025-09-16T17:14:35.721555] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T17:14:35.863354] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T17:14:36.374944] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-05. [2025-09-16T17:14:36.827721] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-09-16T17:14:37.174981] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-04. [2025-09-16T17:14:37.824796] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T17:14:37.974907] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-06. [2025-09-16T17:14:38.612851] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-09-16T17:14:38.774948] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-05. [2025-09-16T17:14:39.151520] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-09-16T17:14:39.574925] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-07. [2025-09-16T17:14:40.345546] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T17:14:40.374880] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-05. [2025-09-16T17:14:40.908547] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-09-16T17:14:41.174936] [INFO ] [REMOTE ] ARIS: Fetching primary data for 2025-05. [2025-09-16T17:14:41.974945] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-08. [2025-09-16T17:14:42.137024] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T17:14:42.746721] [INFO ] [REMOTE ] ARIS: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T17:14:42.774870] [INFO ] [REMOTE ] ARIS: Fetching primary data for 2025-06. [2025-09-16T17:14:43.261639] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T17:14:43.574929] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-09. [2025-09-16T17:14:44.175107] [INFO ] [REMOTE ] ARIS: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-09-16T17:14:44.374970] [INFO ] [REMOTE ] ARIS: Fetching primary data for 2025-07. [2025-09-16T17:14:44.801872] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-09-16T17:14:44.808046] [INFO ] [REMOTE ] ARLO: Full fetch complete (47672 records). [2025-09-16T17:14:44.808214] [DEBUG] [VALIDATOR ] ARLO: Validating primary data. [2025-09-16T17:14:45.180166] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-06. [2025-09-16T17:14:45.368488] [DEBUG] [VALIDATOR ] ARLO: All expected trading days are present. [2025-09-16T17:14:45.368619] [DEBUG] [VALIDATOR ] ARLO: Primary data validation passed. [2025-09-16T17:14:45.369555] [INFO ] [MAIN ] ARLO: Fetched primary data is valid. [2025-09-16T17:14:45.369620] [DEBUG] [LOCAL ] ARLO: Saving primary data. [2025-09-16T17:14:45.371300] [DEBUG] [LOCAL ] ARLO: Trimmed 47672 records to 32647 (keeping 150 days of data) [2025-09-16T17:14:45.674315] [DEBUG] [LOCAL ] ARLO: Primary data saved (32647 records). [2025-09-16T17:14:45.674566] [INFO ] [MAIN ] ARLO: Full history saved. [2025-09-16T17:14:45.674764] [INFO ] [MAIN ] Update Progress: 145/2369 symbols completed. ARLO took 17.4s. Estimated remaining: 3h 45m 37s. Rate: 9.9 symbols/min [2025-09-16T17:14:45.926402] [INFO ] [MAIN ] ARR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:45.968410] [INFO ] [REMOTE ] ARIS: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T17:14:45.978984] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-06. [2025-09-16T17:14:46.025030] [DEBUG] [LOCAL ] ARR: Loaded 38305 existing records from primary storage. [2025-09-16T17:14:46.025428] [INFO ] [MAIN ] ARR: Loaded 38305 existing records. [2025-09-16T17:14:46.216743] [WARN ] [MAIN ] ARR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:46.217025] [INFO ] [MAIN ] ARR: Wiping existing primary data before full refresh. [2025-09-16T17:14:46.217088] [INFO ] [LOCAL ] ARR: Starting deletion of primary data. [2025-09-16T17:14:46.218752] [DEBUG] [LOCAL ] ARR: Deleted primary data file. [2025-09-16T17:14:46.218806] [INFO ] [MAIN ] ARR: Starting full primary refresh. [2025-09-16T17:14:46.575041] [INFO ] [REMOTE ] ARR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:46.774907] [INFO ] [REMOTE ] ARIS: Fetching primary data for 2025-08. [2025-09-16T17:14:46.941076] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T17:14:47.369253] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T17:14:47.574891] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-07. [2025-09-16T17:14:48.290400] [INFO ] [REMOTE ] ARIS: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-09-16T17:14:48.374894] [INFO ] [REMOTE ] ARIS: Fetching primary data for 2025-09. [2025-09-16T17:14:49.174940] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-02. [2025-09-16T17:14:49.335489] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T17:14:49.751978] [INFO ] [REMOTE ] ARIS: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T17:14:49.756882] [INFO ] [REMOTE ] ARIS: Full fetch complete (45271 records). [2025-09-16T17:14:49.756991] [DEBUG] [VALIDATOR ] ARIS: Validating primary data. [2025-09-16T17:14:49.757628] [FATAL] [VALIDATOR ] ARIS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:14:49.757731] [FATAL] [MAIN ] ARIS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:14:49.757771] [WARN ] [MAIN ] ARIS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:14:49.757816] [INFO ] [LOCAL ] ARIS: Starting deletion of primary data. [2025-09-16T17:14:49.757876] [DEBUG] [LOCAL ] ARIS: No primary data file found to delete. [2025-09-16T17:14:49.757914] [INFO ] [LOCAL ] ARIS: Starting deletion of derivative data. [2025-09-16T17:14:49.758268] [DEBUG] [LOCAL ] ARIS: Deleted derivative data file. [2025-09-16T17:14:49.759240] [INFO ] [MAIN ] Update Progress: 146/2369 symbols completed. ARIS took 31.0s. Estimated remaining: 3h 45m 0s. Rate: 9.9 symbols/min [2025-09-16T17:14:49.935995] [INFO ] [MAIN ] ARW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:49.961047] [DEBUG] [LOCAL ] ARW: Loaded 23693 existing records from primary storage. [2025-09-16T17:14:49.961188] [INFO ] [MAIN ] ARW: Loaded 23693 existing records. [2025-09-16T17:14:49.982367] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-07. [2025-09-16T17:14:50.148116] [WARN ] [MAIN ] ARW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:50.148400] [INFO ] [MAIN ] ARW: Wiping existing primary data before full refresh. [2025-09-16T17:14:50.148449] [INFO ] [LOCAL ] ARW: Starting deletion of primary data. [2025-09-16T17:14:50.149592] [DEBUG] [LOCAL ] ARW: Deleted primary data file. [2025-09-16T17:14:50.149672] [INFO ] [MAIN ] ARW: Starting full primary refresh. [2025-09-16T17:14:50.523430] [INFO ] [REMOTE ] ARW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:50.774977] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-08. [2025-09-16T17:14:50.888763] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-09-16T17:14:51.574959] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-02. [2025-09-16T17:14:51.763717] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-09-16T17:14:51.992061] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-09-16T17:14:52.374994] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-08. [2025-09-16T17:14:52.723734] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-09-16T17:14:53.174948] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-03. [2025-09-16T17:14:53.974949] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-09. [2025-09-16T17:14:54.074845] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-09-16T17:14:54.774969] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-09. [2025-09-16T17:14:55.003868] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T17:14:55.298974] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T17:14:55.304798] [INFO ] [REMOTE ] AROC: Full fetch complete (53272 records). [2025-09-16T17:14:55.304893] [DEBUG] [VALIDATOR ] AROC: Validating primary data. [2025-09-16T17:14:55.580055] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-03. [2025-09-16T17:14:55.691225] [DEBUG] [VALIDATOR ] AROC: All expected trading days are present. [2025-09-16T17:14:55.691315] [DEBUG] [VALIDATOR ] AROC: Primary data validation passed. [2025-09-16T17:14:55.691912] [INFO ] [MAIN ] AROC: Fetched primary data is valid. [2025-09-16T17:14:55.691957] [DEBUG] [LOCAL ] AROC: Saving primary data. [2025-09-16T17:14:55.693102] [DEBUG] [LOCAL ] AROC: Trimmed 53272 records to 36147 (keeping 150 days of data) [2025-09-16T17:14:55.918552] [DEBUG] [LOCAL ] AROC: Primary data saved (36147 records). [2025-09-16T17:14:55.918724] [INFO ] [MAIN ] AROC: Full history saved. [2025-09-16T17:14:55.920228] [INFO ] [MAIN ] Update Progress: 147/2369 symbols completed. AROC took 24.3s. Estimated remaining: 3h 44m 56s. Rate: 9.9 symbols/min [2025-09-16T17:14:56.093158] [INFO ] [MAIN ] ARX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:56.093406] [DEBUG] [LOCAL ] ARX: No existing data file found in primary storage. [2025-09-16T17:14:56.093445] [WARN ] [MAIN ] ARX: No existing data found, performing full refresh instead. [2025-09-16T17:14:56.093587] [INFO ] [MAIN ] ARX: Wiping existing primary data before full refresh. [2025-09-16T17:14:56.093660] [INFO ] [LOCAL ] ARX: Starting deletion of primary data. [2025-09-16T17:14:56.093709] [DEBUG] [LOCAL ] ARX: No primary data file found to delete. [2025-09-16T17:14:56.093770] [INFO ] [MAIN ] ARX: Starting full primary refresh. [2025-09-16T17:14:56.379735] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-04. [2025-09-16T17:14:56.461835] [INFO ] [REMOTE ] ARX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:56.464585] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-09-16T17:14:56.472198] [INFO ] [REMOTE ] ARMK: Full fetch complete (54683 records). [2025-09-16T17:14:56.472358] [DEBUG] [VALIDATOR ] ARMK: Validating primary data. [2025-09-16T17:14:56.855522] [DEBUG] [VALIDATOR ] ARMK: All expected trading days are present. [2025-09-16T17:14:56.855623] [DEBUG] [VALIDATOR ] ARMK: Primary data validation passed. [2025-09-16T17:14:56.856233] [INFO ] [MAIN ] ARMK: Fetched primary data is valid. [2025-09-16T17:14:56.856278] [DEBUG] [LOCAL ] ARMK: Saving primary data. [2025-09-16T17:14:56.857498] [DEBUG] [LOCAL ] ARMK: Trimmed 54683 records to 37405 (keeping 150 days of data) [2025-09-16T17:14:57.099219] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T17:14:57.128245] [DEBUG] [LOCAL ] ARMK: Primary data saved (37405 records). [2025-09-16T17:14:57.128439] [INFO ] [MAIN ] ARMK: Full history saved. [2025-09-16T17:14:57.128548] [INFO ] [MAIN ] Update Progress: 148/2369 symbols completed. ARMK took 27.4s. Estimated remaining: 3h 43m 37s. Rate: 9.9 symbols/min [2025-09-16T17:14:57.176672] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-04. [2025-09-16T17:14:57.302865] [INFO ] [MAIN ] AS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:14:57.340555] [DEBUG] [LOCAL ] AS: Loaded 39205 existing records from primary storage. [2025-09-16T17:14:57.340737] [INFO ] [MAIN ] AS: Loaded 39205 existing records. [2025-09-16T17:14:57.514864] [WARN ] [MAIN ] AS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:14:57.515149] [INFO ] [MAIN ] AS: Wiping existing primary data before full refresh. [2025-09-16T17:14:57.515202] [INFO ] [LOCAL ] AS: Starting deletion of primary data. [2025-09-16T17:14:57.516806] [DEBUG] [LOCAL ] AS: Deleted primary data file. [2025-09-16T17:14:57.516863] [INFO ] [MAIN ] AS: Starting full primary refresh. [2025-09-16T17:14:57.874971] [INFO ] [REMOTE ] AS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:14:57.974856] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-02. [2025-09-16T17:14:58.527183] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-09-16T17:14:58.566883] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T17:14:58.774898] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-05. [2025-09-16T17:14:59.337715] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T17:14:59.574897] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-02. [2025-09-16T17:14:59.814611] [DEBUG] [REMOTE ] ARX: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:14:59.814900] [WARN ] [REMOTE ] ARX: No data returned for month 2025-02 [2025-09-16T17:15:00.374934] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-05. [2025-09-16T17:15:00.588850] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T17:15:01.174913] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-03. [2025-09-16T17:15:01.980166] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-06. [2025-09-16T17:15:01.996028] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T17:15:02.774864] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-06. [2025-09-16T17:15:03.352322] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-03 (2.18 seconds). [2025-09-16T17:15:03.574892] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-03. [2025-09-16T17:15:03.785529] [DEBUG] [REMOTE ] ARX: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:15:03.785912] [WARN ] [REMOTE ] ARX: No data returned for month 2025-03 [2025-09-16T17:15:04.298338] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T17:15:04.374900] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-04. [2025-09-16T17:15:04.491273] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-06 (2.51 seconds). [2025-09-16T17:15:04.629279] [DEBUG] [REMOTE ] ARX: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:15:04.629583] [WARN ] [REMOTE ] ARX: No data returned for month 2025-04 [2025-09-16T17:15:05.174907] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-07. [2025-09-16T17:15:05.974896] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-04. [2025-09-16T17:15:06.774908] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-05. [2025-09-16T17:15:07.012790] [DEBUG] [REMOTE ] ARX: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:15:07.013108] [WARN ] [REMOTE ] ARX: No data returned for month 2025-05 [2025-09-16T17:15:07.150735] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-09-16T17:15:07.316215] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-09-16T17:15:07.574918] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-08. [2025-09-16T17:15:08.374935] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-07. [2025-09-16T17:15:09.174909] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-05. [2025-09-16T17:15:09.382884] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-09-16T17:15:09.476367] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-09-16T17:15:09.974953] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-09. [2025-09-16T17:15:10.752780] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T17:15:10.774848] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-06. [2025-09-16T17:15:11.517169] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T17:15:11.523326] [INFO ] [REMOTE ] ARR: Full fetch complete (56647 records). [2025-09-16T17:15:11.523422] [DEBUG] [VALIDATOR ] ARR: Validating primary data. [2025-09-16T17:15:11.583073] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-06. [2025-09-16T17:15:11.915524] [DEBUG] [REMOTE ] ARX: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:15:11.915710] [DEBUG] [VALIDATOR ] ARR: All expected trading days are present. [2025-09-16T17:15:11.915827] [DEBUG] [VALIDATOR ] ARR: Primary data validation passed. [2025-09-16T17:15:11.916590] [INFO ] [MAIN ] ARR: Fetched primary data is valid. [2025-09-16T17:15:11.916681] [WARN ] [REMOTE ] ARX: No data returned for month 2025-06 [2025-09-16T17:15:11.916823] [DEBUG] [LOCAL ] ARR: Saving primary data. [2025-09-16T17:15:11.918357] [DEBUG] [LOCAL ] ARR: Trimmed 56647 records to 38691 (keeping 150 days of data) [2025-09-16T17:15:12.158444] [DEBUG] [LOCAL ] ARR: Primary data saved (38691 records). [2025-09-16T17:15:12.158591] [INFO ] [MAIN ] ARR: Full history saved. [2025-09-16T17:15:12.158698] [INFO ] [MAIN ] Update Progress: 149/2369 symbols completed. ARR took 26.5s. Estimated remaining: 3h 45m 45s. Rate: 9.8 symbols/min [2025-09-16T17:15:12.350118] [INFO ] [MAIN ] ASA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:15:12.362443] [DEBUG] [LOCAL ] ASA: Loaded 10287 existing records from primary storage. [2025-09-16T17:15:12.362578] [INFO ] [MAIN ] ASA: Loaded 10287 existing records. [2025-09-16T17:15:12.383596] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-08. [2025-09-16T17:15:12.568700] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-09-16T17:15:12.598887] [WARN ] [MAIN ] ASA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:15:12.599553] [INFO ] [MAIN ] ASA: Wiping existing primary data before full refresh. [2025-09-16T17:15:12.599621] [INFO ] [LOCAL ] ASA: Starting deletion of primary data. [2025-09-16T17:15:12.600571] [DEBUG] [LOCAL ] ASA: Deleted primary data file. [2025-09-16T17:15:12.600631] [INFO ] [MAIN ] ASA: Starting full primary refresh. [2025-09-16T17:15:12.965870] [INFO ] [REMOTE ] ASA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:15:13.174936] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-07. [2025-09-16T17:15:13.614955] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-09-16T17:15:13.974960] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-09. [2025-09-16T17:15:14.774936] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-02. [2025-09-16T17:15:15.011276] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-09-16T17:15:15.281083] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T17:15:15.286521] [INFO ] [REMOTE ] ARW: Full fetch complete (36010 records). [2025-09-16T17:15:15.286628] [DEBUG] [VALIDATOR ] ARW: Validating primary data. [2025-09-16T17:15:15.575736] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-08. [2025-09-16T17:15:15.660674] [DEBUG] [VALIDATOR ] ARW: All expected trading days are present. [2025-09-16T17:15:15.660760] [DEBUG] [VALIDATOR ] ARW: Primary data validation passed. [2025-09-16T17:15:15.661188] [INFO ] [MAIN ] ARW: Fetched primary data is valid. [2025-09-16T17:15:15.661230] [DEBUG] [LOCAL ] ARW: Saving primary data. [2025-09-16T17:15:15.662255] [DEBUG] [LOCAL ] ARW: Trimmed 36010 records to 23944 (keeping 150 days of data) [2025-09-16T17:15:15.758877] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-09-16T17:15:15.818863] [DEBUG] [LOCAL ] ARW: Primary data saved (23944 records). [2025-09-16T17:15:15.818998] [INFO ] [MAIN ] ARW: Full history saved. [2025-09-16T17:15:15.819096] [INFO ] [MAIN ] Update Progress: 150/2369 symbols completed. ARW took 26.1s. Estimated remaining: 3h 45m 2s. Rate: 9.9 symbols/min [2025-09-16T17:15:15.992542] [INFO ] [MAIN ] ASAN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:15:16.029488] [DEBUG] [LOCAL ] ASAN: Loaded 38828 existing records from primary storage. [2025-09-16T17:15:16.029675] [INFO ] [MAIN ] ASAN: Loaded 38828 existing records. [2025-09-16T17:15:16.219095] [WARN ] [MAIN ] ASAN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:15:16.219386] [INFO ] [MAIN ] ASAN: Wiping existing primary data before full refresh. [2025-09-16T17:15:16.219433] [INFO ] [LOCAL ] ASAN: Starting deletion of primary data. [2025-09-16T17:15:16.220687] [DEBUG] [LOCAL ] ASAN: Deleted primary data file. [2025-09-16T17:15:16.220738] [INFO ] [MAIN ] ASAN: Starting full primary refresh. [2025-09-16T17:15:16.385107] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-03. [2025-09-16T17:15:16.572799] [INFO ] [REMOTE ] ASAN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:15:17.174911] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-02. [2025-09-16T17:15:17.183550] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T17:15:17.426964] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-09-16T17:15:17.974937] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-04. [2025-09-16T17:15:18.774953] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-07. [2025-09-16T17:15:18.965064] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-09-16T17:15:19.024733] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-09-16T17:15:19.574936] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-09. [2025-09-16T17:15:19.861763] [INFO ] [REMOTE ] ARX: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T17:15:20.374951] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-05. [2025-09-16T17:15:21.011958] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T17:15:21.020874] [INFO ] [REMOTE ] AS: Full fetch complete (57290 records). [2025-09-16T17:15:21.021064] [DEBUG] [VALIDATOR ] AS: Validating primary data. [2025-09-16T17:15:21.021932] [FATAL] [VALIDATOR ] AS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:15:21.022039] [FATAL] [MAIN ] AS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:15:21.022097] [WARN ] [MAIN ] AS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:15:21.022360] [INFO ] [LOCAL ] AS: Starting deletion of primary data. [2025-09-16T17:15:21.022438] [DEBUG] [LOCAL ] AS: No primary data file found to delete. [2025-09-16T17:15:21.022490] [INFO ] [LOCAL ] AS: Starting deletion of derivative data. [2025-09-16T17:15:21.022852] [DEBUG] [LOCAL ] AS: Deleted derivative data file. [2025-09-16T17:15:21.022980] [INFO ] [MAIN ] Update Progress: 151/2369 symbols completed. AS took 23.9s. Estimated remaining: 3h 44m 43s. Rate: 9.9 symbols/min [2025-09-16T17:15:21.175587] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-03. [2025-09-16T17:15:21.208210] [INFO ] [MAIN ] ASB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:15:21.217338] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T17:15:21.253477] [DEBUG] [LOCAL ] ASB: Loaded 36181 existing records from primary storage. [2025-09-16T17:15:21.253623] [INFO ] [MAIN ] ASB: Loaded 36181 existing records. [2025-09-16T17:15:21.439292] [WARN ] [MAIN ] ASB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:15:21.439612] [INFO ] [MAIN ] ASB: Wiping existing primary data before full refresh. [2025-09-16T17:15:21.439679] [INFO ] [LOCAL ] ASB: Starting deletion of primary data. [2025-09-16T17:15:21.441021] [DEBUG] [LOCAL ] ASB: Deleted primary data file. [2025-09-16T17:15:21.441077] [INFO ] [MAIN ] ASB: Starting full primary refresh. [2025-09-16T17:15:21.803666] [INFO ] [REMOTE ] ASB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:15:21.974893] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-06. [2025-09-16T17:15:22.770445] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-09-16T17:15:22.774845] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-07. [2025-09-16T17:15:23.079445] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-09-16T17:15:23.410216] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-09-16T17:15:23.574955] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-08. [2025-09-16T17:15:24.374966] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-08. [2025-09-16T17:15:24.959795] [INFO ] [REMOTE ] ARX: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T17:15:24.999771] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-09-16T17:15:25.174911] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-02. [2025-09-16T17:15:25.974937] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-04. [2025-09-16T17:15:26.688151] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-09-16T17:15:26.774951] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-09. [2025-09-16T17:15:27.451571] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T17:15:27.574866] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-03. [2025-09-16T17:15:27.712993] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-09-16T17:15:27.716868] [INFO ] [REMOTE ] ASA: Full fetch complete (14351 records). [2025-09-16T17:15:27.717148] [DEBUG] [VALIDATOR ] ASA: Validating primary data. [2025-09-16T17:15:28.085005] [DEBUG] [VALIDATOR ] ASA: All expected trading days are present. [2025-09-16T17:15:28.085088] [DEBUG] [VALIDATOR ] ASA: Primary data validation passed. [2025-09-16T17:15:28.085438] [INFO ] [MAIN ] ASA: Fetched primary data is valid. [2025-09-16T17:15:28.085482] [DEBUG] [LOCAL ] ASA: Saving primary data. [2025-09-16T17:15:28.086324] [DEBUG] [LOCAL ] ASA: Trimmed 14351 records to 10444 (keeping 150 days of data) [2025-09-16T17:15:28.147953] [DEBUG] [LOCAL ] ASA: Primary data saved (10444 records). [2025-09-16T17:15:28.148097] [INFO ] [MAIN ] ASA: Full history saved. [2025-09-16T17:15:28.148186] [INFO ] [MAIN ] Update Progress: 152/2369 symbols completed. ASA took 16.0s. Estimated remaining: 3h 44m 53s. Rate: 9.9 symbols/min [2025-09-16T17:15:28.315437] [INFO ] [MAIN ] ASBA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:15:28.319533] [DEBUG] [LOCAL ] ASBA: Loaded 2512 existing records from primary storage. [2025-09-16T17:15:28.319609] [INFO ] [MAIN ] ASBA: Loaded 2512 existing records. [2025-09-16T17:15:28.381565] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-05. [2025-09-16T17:15:28.495281] [WARN ] [MAIN ] ASBA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:15:28.495497] [INFO ] [MAIN ] ASBA: Wiping existing primary data before full refresh. [2025-09-16T17:15:28.495544] [INFO ] [LOCAL ] ASBA: Starting deletion of primary data. [2025-09-16T17:15:28.495769] [DEBUG] [LOCAL ] ASBA: Deleted primary data file. [2025-09-16T17:15:28.495817] [INFO ] [MAIN ] ASBA: Starting full primary refresh. [2025-09-16T17:15:28.849081] [INFO ] [REMOTE ] ASBA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:15:29.186712] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-02. [2025-09-16T17:15:29.193575] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T17:15:29.888057] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-09-16T17:15:29.891258] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-09-16T17:15:29.974928] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-04. [2025-09-16T17:15:30.774912] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-09. [2025-09-16T17:15:31.415537] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-09-16T17:15:31.574939] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-06. [2025-09-16T17:15:32.363395] [INFO ] [REMOTE ] ARX: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T17:15:32.368445] [INFO ] [REMOTE ] ARX: Full fetch complete (12016 records). [2025-09-16T17:15:32.368515] [DEBUG] [VALIDATOR ] ARX: Validating primary data. [2025-09-16T17:15:32.379955] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-03. [2025-09-16T17:15:32.583815] [WARN ] [VALIDATOR ] ARX: Validation failed - insufficient historical data. Have 38 unique days, need 150 days [2025-09-16T17:15:32.584093] [FATAL] [MAIN ] ARX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:15:32.584152] [WARN ] [MAIN ] ARX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:15:32.584213] [INFO ] [LOCAL ] ARX: Starting deletion of primary data. [2025-09-16T17:15:32.584275] [DEBUG] [LOCAL ] ARX: No primary data file found to delete. [2025-09-16T17:15:32.584311] [INFO ] [LOCAL ] ARX: Starting deletion of derivative data. [2025-09-16T17:15:32.584356] [DEBUG] [LOCAL ] ARX: No derivative data file found to delete. [2025-09-16T17:15:32.584413] [INFO ] [MAIN ] Update Progress: 153/2369 symbols completed. ARX took 36.7s. Estimated remaining: 3h 44m 23s. Rate: 9.9 symbols/min [2025-09-16T17:15:32.757293] [INFO ] [MAIN ] ASC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:15:32.787794] [DEBUG] [LOCAL ] ASC: Loaded 27506 existing records from primary storage. [2025-09-16T17:15:32.787953] [INFO ] [MAIN ] ASC: Loaded 27506 existing records. [2025-09-16T17:15:32.963926] [WARN ] [MAIN ] ASC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:15:32.966935] [INFO ] [MAIN ] ASC: Wiping existing primary data before full refresh. [2025-09-16T17:15:32.967414] [INFO ] [LOCAL ] ASC: Starting deletion of primary data. [2025-09-16T17:15:32.967566] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-09-16T17:15:32.969037] [DEBUG] [LOCAL ] ASC: Deleted primary data file. [2025-09-16T17:15:32.969099] [INFO ] [MAIN ] ASC: Starting full primary refresh. [2025-09-16T17:15:33.179959] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-04. [2025-09-16T17:15:33.322182] [INFO ] [REMOTE ] ASC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:15:33.579150] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-09-16T17:15:33.967058] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-09-16T17:15:33.974869] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-05. [2025-09-16T17:15:34.515784] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-09-16T17:15:34.774957] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-05. [2025-09-16T17:15:35.574909] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-02. [2025-09-16T17:15:36.048760] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T17:15:36.374911] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-06. [2025-09-16T17:15:37.174944] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-07. [2025-09-16T17:15:37.236859] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T17:15:37.879466] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T17:15:37.974862] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-07. [2025-09-16T17:15:38.774913] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-06. [2025-09-16T17:15:38.949235] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-09-16T17:15:39.201867] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-09-16T17:15:39.388267] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-09-16T17:15:39.574979] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-07. [2025-09-16T17:15:40.201518] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-09-16T17:15:40.374973] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-08. [2025-09-16T17:15:40.825747] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-09-16T17:15:41.174909] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-08. [2025-09-16T17:15:41.974898] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-09. [2025-09-16T17:15:42.490028] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-09-16T17:15:42.587216] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-09-16T17:15:42.588177] [INFO ] [REMOTE ] ASBA: Full fetch complete (3723 records). [2025-09-16T17:15:42.588247] [DEBUG] [VALIDATOR ] ASBA: Validating primary data. [2025-09-16T17:15:42.780071] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-08. [2025-09-16T17:15:42.944913] [DEBUG] [VALIDATOR ] ASBA: All expected trading days are present. [2025-09-16T17:15:42.944992] [DEBUG] [VALIDATOR ] ASBA: Primary data validation passed. [2025-09-16T17:15:42.945149] [INFO ] [MAIN ] ASBA: Fetched primary data is valid. [2025-09-16T17:15:42.945197] [DEBUG] [LOCAL ] ASBA: Saving primary data. [2025-09-16T17:15:42.945556] [DEBUG] [LOCAL ] ASBA: Trimmed 3723 records to 2522 (keeping 150 days of data) [2025-09-16T17:15:42.961868] [DEBUG] [LOCAL ] ASBA: Primary data saved (2522 records). [2025-09-16T17:15:42.961949] [INFO ] [MAIN ] ASBA: Full history saved. [2025-09-16T17:15:42.962039] [INFO ] [MAIN ] Update Progress: 154/2369 symbols completed. ASBA took 14.8s. Estimated remaining: 3h 45m 18s. Rate: 9.8 symbols/min [2025-09-16T17:15:43.137049] [INFO ] [MAIN ] ASG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:15:43.155535] [DEBUG] [LOCAL ] ASG: Loaded 14252 existing records from primary storage. [2025-09-16T17:15:43.155667] [INFO ] [MAIN ] ASG: Loaded 14252 existing records. [2025-09-16T17:15:43.331096] [WARN ] [MAIN ] ASG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:15:43.331330] [INFO ] [MAIN ] ASG: Wiping existing primary data before full refresh. [2025-09-16T17:15:43.331395] [INFO ] [LOCAL ] ASG: Starting deletion of primary data. [2025-09-16T17:15:43.332132] [DEBUG] [LOCAL ] ASG: Deleted primary data file. [2025-09-16T17:15:43.332192] [INFO ] [MAIN ] ASG: Starting full primary refresh. [2025-09-16T17:15:43.580233] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-03. [2025-09-16T17:15:43.687022] [INFO ] [REMOTE ] ASG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:15:44.374885] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-09. [2025-09-16T17:15:44.877452] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-08 (2.10 seconds). [2025-09-16T17:15:45.174899] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-09. [2025-09-16T17:15:45.322326] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T17:15:45.759208] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T17:15:45.765246] [INFO ] [REMOTE ] ASB: Full fetch complete (53981 records). [2025-09-16T17:15:45.765409] [DEBUG] [VALIDATOR ] ASB: Validating primary data. [2025-09-16T17:15:45.980130] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-02. [2025-09-16T17:15:46.147059] [DEBUG] [VALIDATOR ] ASB: All expected trading days are present. [2025-09-16T17:15:46.147131] [DEBUG] [VALIDATOR ] ASB: Primary data validation passed. [2025-09-16T17:15:46.147700] [INFO ] [MAIN ] ASB: Fetched primary data is valid. [2025-09-16T17:15:46.147745] [DEBUG] [LOCAL ] ASB: Saving primary data. [2025-09-16T17:15:46.148813] [DEBUG] [LOCAL ] ASB: Trimmed 53981 records to 36535 (keeping 150 days of data) [2025-09-16T17:15:46.379301] [DEBUG] [LOCAL ] ASB: Primary data saved (36535 records). [2025-09-16T17:15:46.379480] [INFO ] [MAIN ] ASB: Full history saved. [2025-09-16T17:15:46.379656] [INFO ] [MAIN ] Update Progress: 155/2369 symbols completed. ASB took 25.4s. Estimated remaining: 3h 44m 34s. Rate: 9.9 symbols/min [2025-09-16T17:15:46.549111] [INFO ] [MAIN ] ASGI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:15:46.549273] [DEBUG] [LOCAL ] ASGI: No existing data file found in primary storage. [2025-09-16T17:15:46.549312] [WARN ] [MAIN ] ASGI: No existing data found, performing full refresh instead. [2025-09-16T17:15:46.549432] [INFO ] [MAIN ] ASGI: Wiping existing primary data before full refresh. [2025-09-16T17:15:46.549471] [INFO ] [LOCAL ] ASGI: Starting deletion of primary data. [2025-09-16T17:15:46.549512] [DEBUG] [LOCAL ] ASGI: No primary data file found to delete. [2025-09-16T17:15:46.549558] [INFO ] [MAIN ] ASGI: Starting full primary refresh. [2025-09-16T17:15:46.786321] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-04. [2025-09-16T17:15:46.901770] [INFO ] [REMOTE ] ASGI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:15:47.159707] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-09-16T17:15:47.168617] [INFO ] [REMOTE ] ASAN: Full fetch complete (56978 records). [2025-09-16T17:15:47.168824] [DEBUG] [VALIDATOR ] ASAN: Validating primary data. [2025-09-16T17:15:47.169697] [FATAL] [VALIDATOR ] ASAN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:15:47.169796] [FATAL] [MAIN ] ASAN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:15:47.169853] [WARN ] [MAIN ] ASAN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:15:47.169914] [INFO ] [LOCAL ] ASAN: Starting deletion of primary data. [2025-09-16T17:15:47.170001] [DEBUG] [LOCAL ] ASAN: No primary data file found to delete. [2025-09-16T17:15:47.170055] [INFO ] [LOCAL ] ASAN: Starting deletion of derivative data. [2025-09-16T17:15:47.170346] [DEBUG] [LOCAL ] ASAN: Deleted derivative data file. [2025-09-16T17:15:47.171197] [INFO ] [MAIN ] Update Progress: 156/2369 symbols completed. ASAN took 31.4s. Estimated remaining: 3h 43m 13s. Rate: 9.9 symbols/min [2025-09-16T17:15:47.346508] [INFO ] [MAIN ] ASGN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:15:47.361963] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T17:15:47.391089] [DEBUG] [LOCAL ] ASGN: Loaded 26147 existing records from primary storage. [2025-09-16T17:15:47.391240] [INFO ] [MAIN ] ASGN: Loaded 26147 existing records. [2025-09-16T17:15:47.568621] [WARN ] [MAIN ] ASGN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:15:47.568935] [INFO ] [MAIN ] ASGN: Wiping existing primary data before full refresh. [2025-09-16T17:15:47.568998] [INFO ] [LOCAL ] ASGN: Starting deletion of primary data. [2025-09-16T17:15:47.569936] [DEBUG] [LOCAL ] ASGN: Deleted primary data file. [2025-09-16T17:15:47.569999] [INFO ] [MAIN ] ASGN: Starting full primary refresh. [2025-09-16T17:15:47.581251] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-02. [2025-09-16T17:15:47.947474] [INFO ] [REMOTE ] ASGN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:15:48.386695] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-03. [2025-09-16T17:15:48.403813] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T17:15:48.673071] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T17:15:49.174974] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-03. [2025-09-16T17:15:49.628940] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-09-16T17:15:49.974919] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-05. [2025-09-16T17:15:50.238203] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-09-16T17:15:50.774912] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-02. [2025-09-16T17:15:51.212189] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-09-16T17:15:51.574897] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-06. [2025-09-16T17:15:52.218245] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T17:15:52.374914] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-04. [2025-09-16T17:15:52.718716] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T17:15:53.087795] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-09-16T17:15:53.174882] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-05. [2025-09-16T17:15:53.974922] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-04. [2025-09-16T17:15:54.135256] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-09-16T17:15:54.774896] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-03. [2025-09-16T17:15:55.039058] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-09-16T17:15:55.574914] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-07. [2025-09-16T17:15:56.089852] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T17:15:56.374906] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-04. [2025-09-16T17:15:56.798001] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-09-16T17:15:57.174905] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-08. [2025-09-16T17:15:57.938821] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T17:15:57.974934] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-06. [2025-09-16T17:15:58.750993] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T17:15:58.774923] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-07. [2025-09-16T17:15:58.814371] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T17:15:59.575883] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-05. [2025-09-16T17:15:59.578082] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T17:16:00.374933] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-08. [2025-09-16T17:16:01.174912] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-05. [2025-09-16T17:16:01.193692] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T17:16:01.690459] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-09-16T17:16:01.974926] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-06. [2025-09-16T17:16:02.493017] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T17:16:02.774910] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-06. [2025-09-16T17:16:03.574883] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-09. [2025-09-16T17:16:03.682322] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T17:16:03.971106] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T17:16:04.374870] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-07. [2025-09-16T17:16:04.581076] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T17:16:04.583730] [INFO ] [REMOTE ] ASG: Full fetch complete (23124 records). [2025-09-16T17:16:04.583803] [DEBUG] [VALIDATOR ] ASG: Validating primary data. [2025-09-16T17:16:04.953210] [DEBUG] [VALIDATOR ] ASG: All expected trading days are present. [2025-09-16T17:16:04.953285] [DEBUG] [VALIDATOR ] ASG: Primary data validation passed. [2025-09-16T17:16:04.953609] [INFO ] [MAIN ] ASG: Fetched primary data is valid. [2025-09-16T17:16:04.953687] [DEBUG] [LOCAL ] ASG: Saving primary data. [2025-09-16T17:16:04.954461] [DEBUG] [LOCAL ] ASG: Trimmed 23124 records to 14378 (keeping 150 days of data) [2025-09-16T17:16:05.040105] [DEBUG] [LOCAL ] ASG: Primary data saved (14378 records). [2025-09-16T17:16:05.040230] [INFO ] [MAIN ] ASG: Full history saved. [2025-09-16T17:16:05.040312] [INFO ] [MAIN ] Update Progress: 157/2369 symbols completed. ASG took 22.1s. Estimated remaining: 3h 45m 53s. Rate: 9.8 symbols/min [2025-09-16T17:16:05.080166] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-09-16T17:16:05.174890] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-08. [2025-09-16T17:16:05.974855] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-07. [2025-09-16T17:16:06.232302] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-09-16T17:16:06.774896] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-09. [2025-09-16T17:16:07.115612] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T17:16:07.574960] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-09. [2025-09-16T17:16:07.798420] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T17:16:07.802736] [INFO ] [REMOTE ] ASC: Full fetch complete (42056 records). [2025-09-16T17:16:07.802821] [DEBUG] [VALIDATOR ] ASC: Validating primary data. [2025-09-16T17:16:08.174399] [DEBUG] [VALIDATOR ] ASC: All expected trading days are present. [2025-09-16T17:16:08.174480] [DEBUG] [VALIDATOR ] ASC: Primary data validation passed. [2025-09-16T17:16:08.175184] [INFO ] [MAIN ] ASC: Fetched primary data is valid. [2025-09-16T17:16:08.175233] [DEBUG] [LOCAL ] ASC: Saving primary data. [2025-09-16T17:16:08.176226] [DEBUG] [LOCAL ] ASC: Trimmed 42056 records to 27759 (keeping 150 days of data) [2025-09-16T17:16:08.360885] [DEBUG] [LOCAL ] ASC: Primary data saved (27759 records). [2025-09-16T17:16:08.361021] [INFO ] [MAIN ] ASC: Full history saved. [2025-09-16T17:16:08.361120] [INFO ] [MAIN ] Update Progress: 158/2369 symbols completed. ASC took 35.8s. Estimated remaining: 3h 45m 8s. Rate: 9.8 symbols/min [2025-09-16T17:16:08.374903] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-08. [2025-09-16T17:16:08.576608] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T17:16:08.579976] [INFO ] [REMOTE ] ASGI: Full fetch complete (23999 records). [2025-09-16T17:16:08.580048] [DEBUG] [VALIDATOR ] ASGI: Validating primary data. [2025-09-16T17:16:08.940065] [DEBUG] [VALIDATOR ] ASGI: All expected trading days are present. [2025-09-16T17:16:08.940141] [DEBUG] [VALIDATOR ] ASGI: Primary data validation passed. [2025-09-16T17:16:08.940468] [INFO ] [MAIN ] ASGI: Fetched primary data is valid. [2025-09-16T17:16:08.940509] [DEBUG] [LOCAL ] ASGI: Saving primary data. [2025-09-16T17:16:08.941117] [DEBUG] [LOCAL ] ASGI: Trimmed 23999 records to 16802 (keeping 150 days of data) [2025-09-16T17:16:09.046615] [DEBUG] [LOCAL ] ASGI: Primary data saved (16802 records). [2025-09-16T17:16:09.046766] [INFO ] [MAIN ] ASGI: Full history saved. [2025-09-16T17:16:09.046842] [INFO ] [MAIN ] Update Progress: 159/2369 symbols completed. ASGI took 22.7s. Estimated remaining: 3h 43m 46s. Rate: 9.9 symbols/min [2025-09-16T17:16:09.499379] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T17:16:09.507624] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-09. [2025-09-16T17:16:10.846989] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T17:16:10.851092] [INFO ] [REMOTE ] ASGN: Full fetch complete (39422 records). [2025-09-16T17:16:10.851203] [DEBUG] [VALIDATOR ] ASGN: Validating primary data. [2025-09-16T17:16:10.851848] [FATAL] [VALIDATOR ] ASGN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:16:10.851931] [FATAL] [MAIN ] ASGN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:16:10.851968] [WARN ] [MAIN ] ASGN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:16:10.852017] [INFO ] [LOCAL ] ASGN: Starting deletion of primary data. [2025-09-16T17:16:10.852082] [DEBUG] [LOCAL ] ASGN: No primary data file found to delete. [2025-09-16T17:16:10.852134] [INFO ] [LOCAL ] ASGN: Starting deletion of derivative data. [2025-09-16T17:16:10.852436] [DEBUG] [LOCAL ] ASGN: Deleted derivative data file. [2025-09-16T17:16:10.852568] [INFO ] [MAIN ] Update Progress: 160/2369 symbols completed. ASGN took 23.7s. Estimated remaining: 3h 42m 41s. Rate: 9.9 symbols/min [2025-09-16T17:16:11.507514] [INFO ] [MAIN ] ASH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:11.503321] [INFO ] [MAIN ] ASIX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:11.546357] [INFO ] [MAIN ] ASPN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:11.548283] [DEBUG] [LOCAL ] ASPN: No existing data file found in primary storage. [2025-09-16T17:16:11.548228] [DEBUG] [LOCAL ] ASH: No existing data file found in primary storage. [2025-09-16T17:16:11.547880] [INFO ] [MAIN ] ASR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:11.548441] [WARN ] [MAIN ] ASH: No existing data found, performing full refresh instead. [2025-09-16T17:16:11.548520] [WARN ] [MAIN ] ASPN: No existing data found, performing full refresh instead. [2025-09-16T17:16:11.548852] [INFO ] [MAIN ] ASPN: Wiping existing primary data before full refresh. [2025-09-16T17:16:11.548766] [DEBUG] [LOCAL ] ASR: No existing data file found in primary storage. [2025-09-16T17:16:11.548974] [WARN ] [MAIN ] ASR: No existing data found, performing full refresh instead. [2025-09-16T17:16:11.549333] [INFO ] [MAIN ] ASH: Wiping existing primary data before full refresh. [2025-09-16T17:16:11.549469] [INFO ] [MAIN ] ASR: Wiping existing primary data before full refresh. [2025-09-16T17:16:11.549615] [INFO ] [LOCAL ] ASR: Starting deletion of primary data. [2025-09-16T17:16:11.549708] [DEBUG] [LOCAL ] ASR: No primary data file found to delete. [2025-09-16T17:16:11.549765] [INFO ] [MAIN ] ASR: Starting full primary refresh. [2025-09-16T17:16:11.549526] [INFO ] [LOCAL ] ASH: Starting deletion of primary data. [2025-09-16T17:16:11.563101] [INFO ] [LOCAL ] ASPN: Starting deletion of primary data. [2025-09-16T17:16:11.599375] [DEBUG] [LOCAL ] ASH: No primary data file found to delete. [2025-09-16T17:16:11.608634] [DEBUG] [LOCAL ] ASPN: No primary data file found to delete. [2025-09-16T17:16:11.633627] [INFO ] [MAIN ] ASH: Starting full primary refresh. [2025-09-16T17:16:11.633541] [INFO ] [MAIN ] ASPN: Starting full primary refresh. [2025-09-16T17:16:11.852867] [DEBUG] [LOCAL ] ASIX: Loaded 16877 existing records from primary storage. [2025-09-16T17:16:11.887295] [INFO ] [MAIN ] ASIX: Loaded 16877 existing records. [2025-09-16T17:16:12.697211] [INFO ] [REMOTE ] ASPN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:12.711251] [WARN ] [MAIN ] ASIX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:16:12.727891] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-02. [2025-09-16T17:16:12.765113] [INFO ] [MAIN ] ASIX: Wiping existing primary data before full refresh. [2025-09-16T17:16:12.748093] [INFO ] [REMOTE ] ASR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:12.765503] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-02. [2025-09-16T17:16:12.765584] [INFO ] [LOCAL ] ASIX: Starting deletion of primary data. [2025-09-16T17:16:12.806851] [DEBUG] [LOCAL ] ASIX: Deleted primary data file. [2025-09-16T17:16:12.826487] [INFO ] [REMOTE ] ASH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:12.823315] [INFO ] [MAIN ] ASIX: Starting full primary refresh. [2025-09-16T17:16:12.857513] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-02. [2025-09-16T17:16:13.192310] [INFO ] [REMOTE ] ASIX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:13.192672] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-02. [2025-09-16T17:16:13.526924] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T17:16:13.528883] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-03. [2025-09-16T17:16:14.140222] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-09-16T17:16:14.145581] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-03. [2025-09-16T17:16:14.223957] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T17:16:14.374203] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-09-16T17:16:14.567362] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-09-16T17:16:14.774901] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-04. [2025-09-16T17:16:14.805758] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-09-16T17:16:15.574912] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-03. [2025-09-16T17:16:15.700437] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T17:16:16.374948] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-04. [2025-09-16T17:16:16.678550] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T17:16:17.174957] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-04. [2025-09-16T17:16:17.245155] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-09-16T17:16:17.974955] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-05. [2025-09-16T17:16:18.368075] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:16:18.774944] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-05. [2025-09-16T17:16:18.835366] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T17:16:19.574926] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-05. [2025-09-16T17:16:19.907136] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-09-16T17:16:20.374927] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-06. [2025-09-16T17:16:21.059446] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T17:16:21.174945] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-03. [2025-09-16T17:16:21.295613] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-09-16T17:16:21.974924] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-07. [2025-09-16T17:16:22.774901] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-06. [2025-09-16T17:16:22.920013] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T17:16:22.945720] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T17:16:23.533389] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-09-16T17:16:23.574885] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-07. [2025-09-16T17:16:24.374940] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-08. [2025-09-16T17:16:24.487703] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-09-16T17:16:25.174914] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-06. [2025-09-16T17:16:25.548721] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T17:16:25.974943] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-04. [2025-09-16T17:16:26.459211] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T17:16:26.774912] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-09. [2025-09-16T17:16:27.574925] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-07. [2025-09-16T17:16:27.770310] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T17:16:27.773312] [INFO ] [REMOTE ] ASIX: Full fetch complete (23888 records). [2025-09-16T17:16:27.773395] [DEBUG] [VALIDATOR ] ASIX: Validating primary data. [2025-09-16T17:16:28.189672] [DEBUG] [VALIDATOR ] ASIX: All expected trading days are present. [2025-09-16T17:16:28.189765] [DEBUG] [VALIDATOR ] ASIX: Primary data validation passed. [2025-09-16T17:16:28.190177] [INFO ] [MAIN ] ASIX: Fetched primary data is valid. [2025-09-16T17:16:28.190249] [DEBUG] [LOCAL ] ASIX: Saving primary data. [2025-09-16T17:16:28.191156] [DEBUG] [LOCAL ] ASIX: Trimmed 23888 records to 17000 (keeping 150 days of data) [2025-09-16T17:16:28.219866] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-04 (2.24 seconds). [2025-09-16T17:16:28.334813] [DEBUG] [LOCAL ] ASIX: Primary data saved (17000 records). [2025-09-16T17:16:28.334959] [INFO ] [MAIN ] ASIX: Full history saved. [2025-09-16T17:16:28.335045] [INFO ] [MAIN ] Update Progress: 161/2369 symbols completed. ASIX took 17.5s. Estimated remaining: 3h 45m 12s. Rate: 9.8 symbols/min [2025-09-16T17:16:28.380036] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-08. [2025-09-16T17:16:28.508120] [INFO ] [MAIN ] ASX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:28.508287] [DEBUG] [LOCAL ] ASX: No existing data file found in primary storage. [2025-09-16T17:16:28.508325] [WARN ] [MAIN ] ASX: No existing data found, performing full refresh instead. [2025-09-16T17:16:28.508445] [INFO ] [MAIN ] ASX: Wiping existing primary data before full refresh. [2025-09-16T17:16:28.508484] [INFO ] [LOCAL ] ASX: Starting deletion of primary data. [2025-09-16T17:16:28.508528] [DEBUG] [LOCAL ] ASX: No primary data file found to delete. [2025-09-16T17:16:28.508561] [INFO ] [MAIN ] ASX: Starting full primary refresh. [2025-09-16T17:16:28.868962] [INFO ] [REMOTE ] ASX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:28.916374] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-09-16T17:16:29.010947] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-09-16T17:16:29.174975] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-05. [2025-09-16T17:16:29.974942] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-02. [2025-09-16T17:16:30.774909] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-09. [2025-09-16T17:16:30.974433] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-09-16T17:16:31.512732] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-09-16T17:16:31.514630] [INFO ] [REMOTE ] ASR: Full fetch complete (9692 records). [2025-09-16T17:16:31.514742] [DEBUG] [VALIDATOR ] ASR: Validating primary data. [2025-09-16T17:16:31.580017] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-06. [2025-09-16T17:16:31.898224] [DEBUG] [VALIDATOR ] ASR: All expected trading days are present. [2025-09-16T17:16:31.898374] [DEBUG] [VALIDATOR ] ASR: Primary data validation passed. [2025-09-16T17:16:31.898717] [INFO ] [MAIN ] ASR: Fetched primary data is valid. [2025-09-16T17:16:31.898781] [DEBUG] [LOCAL ] ASR: Saving primary data. [2025-09-16T17:16:31.899425] [DEBUG] [LOCAL ] ASR: Trimmed 9692 records to 7108 (keeping 150 days of data) [2025-09-16T17:16:31.966265] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-09-16T17:16:31.983926] [DEBUG] [LOCAL ] ASR: Primary data saved (7108 records). [2025-09-16T17:16:31.984069] [INFO ] [MAIN ] ASR: Full history saved. [2025-09-16T17:16:31.984146] [INFO ] [MAIN ] Update Progress: 162/2369 symbols completed. ASR took 21.1s. Estimated remaining: 3h 44m 32s. Rate: 9.8 symbols/min [2025-09-16T17:16:32.160169] [INFO ] [MAIN ] ATEN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:32.196262] [DEBUG] [LOCAL ] ATEN: Loaded 28427 existing records from primary storage. [2025-09-16T17:16:32.196440] [INFO ] [MAIN ] ATEN: Loaded 28427 existing records. [2025-09-16T17:16:32.384183] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-08. [2025-09-16T17:16:32.392781] [WARN ] [MAIN ] ATEN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:16:32.394431] [INFO ] [MAIN ] ATEN: Wiping existing primary data before full refresh. [2025-09-16T17:16:32.394516] [INFO ] [LOCAL ] ATEN: Starting deletion of primary data. [2025-09-16T17:16:32.396300] [DEBUG] [LOCAL ] ATEN: Deleted primary data file. [2025-09-16T17:16:32.396405] [INFO ] [MAIN ] ATEN: Starting full primary refresh. [2025-09-16T17:16:32.943331] [INFO ] [REMOTE ] ATEN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:33.174986] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-03. [2025-09-16T17:16:33.260581] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T17:16:33.952289] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T17:16:33.974887] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-09. [2025-09-16T17:16:34.597893] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T17:16:34.774938] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-07. [2025-09-16T17:16:34.953962] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T17:16:34.958059] [INFO ] [REMOTE ] ASH: Full fetch complete (41243 records). [2025-09-16T17:16:34.958167] [DEBUG] [VALIDATOR ] ASH: Validating primary data. [2025-09-16T17:16:34.958840] [FATAL] [VALIDATOR ] ASH: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:16:34.958920] [FATAL] [MAIN ] ASH: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:16:34.958958] [WARN ] [MAIN ] ASH: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:16:34.958997] [INFO ] [LOCAL ] ASH: Starting deletion of primary data. [2025-09-16T17:16:34.959057] [DEBUG] [LOCAL ] ASH: No primary data file found to delete. [2025-09-16T17:16:34.959096] [INFO ] [LOCAL ] ASH: Starting deletion of derivative data. [2025-09-16T17:16:34.959159] [DEBUG] [LOCAL ] ASH: No derivative data file found to delete. [2025-09-16T17:16:34.959222] [INFO ] [MAIN ] Update Progress: 163/2369 symbols completed. ASH took 24.1s. Estimated remaining: 3h 43m 44s. Rate: 9.9 symbols/min [2025-09-16T17:16:35.134833] [INFO ] [MAIN ] ATGE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:35.160749] [DEBUG] [LOCAL ] ATGE: Loaded 22888 existing records from primary storage. [2025-09-16T17:16:35.160897] [INFO ] [MAIN ] ATGE: Loaded 22888 existing records. [2025-09-16T17:16:35.343786] [WARN ] [MAIN ] ATGE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:16:35.344067] [INFO ] [MAIN ] ATGE: Wiping existing primary data before full refresh. [2025-09-16T17:16:35.344132] [INFO ] [LOCAL ] ATGE: Starting deletion of primary data. [2025-09-16T17:16:35.345395] [DEBUG] [LOCAL ] ATGE: Deleted primary data file. [2025-09-16T17:16:35.345470] [INFO ] [MAIN ] ATGE: Starting full primary refresh. [2025-09-16T17:16:35.585536] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-04. [2025-09-16T17:16:35.702485] [INFO ] [REMOTE ] ATGE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:36.254825] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T17:16:36.374942] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-02. [2025-09-16T17:16:37.174951] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-08. [2025-09-16T17:16:37.261729] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-09-16T17:16:37.628038] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T17:16:37.974914] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-05. [2025-09-16T17:16:38.774925] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-02. [2025-09-16T17:16:39.199860] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-09-16T17:16:39.504048] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T17:16:39.574884] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-03. [2025-09-16T17:16:40.161916] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T17:16:40.374913] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-06. [2025-09-16T17:16:40.883540] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T17:16:41.174909] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-04. [2025-09-16T17:16:41.974947] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-03. [2025-09-16T17:16:42.217543] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-09-16T17:16:42.734047] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T17:16:42.774882] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-05. [2025-09-16T17:16:43.270138] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T17:16:43.574903] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-07. [2025-09-16T17:16:44.105494] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T17:16:44.374936] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-06. [2025-09-16T17:16:45.174914] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-04. [2025-09-16T17:16:45.289177] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T17:16:45.439212] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-09-16T17:16:45.974924] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-09. [2025-09-16T17:16:46.264412] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T17:16:46.774897] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-07. [2025-09-16T17:16:47.574857] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-05. [2025-09-16T17:16:47.679603] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T17:16:47.686600] [INFO ] [REMOTE ] ASPN: Full fetch complete (54248 records). [2025-09-16T17:16:47.686887] [DEBUG] [VALIDATOR ] ASPN: Validating primary data. [2025-09-16T17:16:48.077670] [DEBUG] [VALIDATOR ] ASPN: All expected trading days are present. [2025-09-16T17:16:48.077752] [DEBUG] [VALIDATOR ] ASPN: Primary data validation passed. [2025-09-16T17:16:48.078354] [INFO ] [MAIN ] ASPN: Fetched primary data is valid. [2025-09-16T17:16:48.078398] [DEBUG] [LOCAL ] ASPN: Saving primary data. [2025-09-16T17:16:48.079486] [DEBUG] [LOCAL ] ASPN: Trimmed 54248 records to 36948 (keeping 150 days of data) [2025-09-16T17:16:48.289699] [DEBUG] [LOCAL ] ASPN: Primary data saved (36948 records). [2025-09-16T17:16:48.289840] [INFO ] [MAIN ] ASPN: Full history saved. [2025-09-16T17:16:48.289916] [INFO ] [MAIN ] Update Progress: 164/2369 symbols completed. ASPN took 37.4s. Estimated remaining: 3h 45m 15s. Rate: 9.8 symbols/min [2025-09-16T17:16:48.379952] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-08. [2025-09-16T17:16:48.482213] [INFO ] [MAIN ] ATHM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:48.518442] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T17:16:48.535763] [DEBUG] [LOCAL ] ATHM: Loaded 22614 existing records from primary storage. [2025-09-16T17:16:48.536256] [INFO ] [MAIN ] ATHM: Loaded 22614 existing records. [2025-09-16T17:16:48.711766] [WARN ] [MAIN ] ATHM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:16:48.712016] [INFO ] [MAIN ] ATHM: Wiping existing primary data before full refresh. [2025-09-16T17:16:48.712060] [INFO ] [LOCAL ] ATHM: Starting deletion of primary data. [2025-09-16T17:16:48.713066] [DEBUG] [LOCAL ] ATHM: Deleted primary data file. [2025-09-16T17:16:48.713118] [INFO ] [MAIN ] ATHM: Starting full primary refresh. [2025-09-16T17:16:49.071326] [INFO ] [REMOTE ] ATHM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:49.101500] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T17:16:49.174875] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-06. [2025-09-16T17:16:49.974886] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-02. [2025-09-16T17:16:50.343047] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-09-16T17:16:50.471685] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-09-16T17:16:50.774924] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-07. [2025-09-16T17:16:51.515196] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-09-16T17:16:51.574923] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-03. [2025-09-16T17:16:51.888057] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-09-16T17:16:52.374910] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-08. [2025-09-16T17:16:53.006736] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-09-16T17:16:53.174955] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-09. [2025-09-16T17:16:53.519816] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T17:16:53.974949] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-09. [2025-09-16T17:16:54.774946] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-08. [2025-09-16T17:16:54.931899] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T17:16:54.938114] [INFO ] [REMOTE ] ASX: Full fetch complete (57562 records). [2025-09-16T17:16:54.938227] [DEBUG] [VALIDATOR ] ASX: Validating primary data. [2025-09-16T17:16:55.152755] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T17:16:55.248152] [INFO ] [REMOTE ] ATGE: Full fetch complete (34997 records). [2025-09-16T17:16:55.258674] [DEBUG] [VALIDATOR ] ATGE: Validating primary data. [2025-09-16T17:16:55.286621] [FATAL] [VALIDATOR ] ATGE: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:16:55.317904] [FATAL] [MAIN ] ATGE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:16:55.334852] [WARN ] [MAIN ] ATGE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:16:55.351769] [INFO ] [LOCAL ] ATGE: Starting deletion of primary data. [2025-09-16T17:16:55.352008] [DEBUG] [VALIDATOR ] ASX: All expected trading days are present. [2025-09-16T17:16:55.352204] [DEBUG] [LOCAL ] ATGE: No primary data file found to delete. [2025-09-16T17:16:55.352133] [DEBUG] [VALIDATOR ] ASX: Primary data validation passed. [2025-09-16T17:16:55.352994] [INFO ] [MAIN ] ASX: Fetched primary data is valid. [2025-09-16T17:16:55.352328] [INFO ] [LOCAL ] ATGE: Starting deletion of derivative data. [2025-09-16T17:16:55.353097] [DEBUG] [LOCAL ] ASX: Saving primary data. [2025-09-16T17:16:55.353418] [DEBUG] [LOCAL ] ATGE: Deleted derivative data file. [2025-09-16T17:16:55.353575] [INFO ] [MAIN ] Update Progress: 165/2369 symbols completed. ATGE took 20.4s. Estimated remaining: 3h 45m 21s. Rate: 9.8 symbols/min [2025-09-16T17:16:55.448374] [DEBUG] [LOCAL ] ASX: Trimmed 57562 records to 39389 (keeping 150 days of data) [2025-09-16T17:16:55.589704] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-04. [2025-09-16T17:16:55.620774] [INFO ] [MAIN ] ATHS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:55.622232] [DEBUG] [LOCAL ] ATHS: No existing data file found in primary storage. [2025-09-16T17:16:55.622861] [WARN ] [MAIN ] ATHS: No existing data found, performing full refresh instead. [2025-09-16T17:16:55.624147] [INFO ] [MAIN ] ATHS: Wiping existing primary data before full refresh. [2025-09-16T17:16:55.625348] [INFO ] [LOCAL ] ATHS: Starting deletion of primary data. [2025-09-16T17:16:55.626069] [DEBUG] [LOCAL ] ATHS: No primary data file found to delete. [2025-09-16T17:16:55.626841] [INFO ] [MAIN ] ATHS: Starting full primary refresh. [2025-09-16T17:16:56.005008] [INFO ] [REMOTE ] ATHS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:56.135620] [DEBUG] [LOCAL ] ASX: Primary data saved (39389 records). [2025-09-16T17:16:56.135835] [INFO ] [MAIN ] ASX: Full history saved. [2025-09-16T17:16:56.135945] [INFO ] [MAIN ] Update Progress: 166/2369 symbols completed. ASX took 27.8s. Estimated remaining: 3h 44m 4s. Rate: 9.8 symbols/min [2025-09-16T17:16:56.306961] [INFO ] [MAIN ] ATI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:56.329265] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T17:16:56.369810] [DEBUG] [LOCAL ] ATI: Loaded 38269 existing records from primary storage. [2025-09-16T17:16:56.369946] [INFO ] [MAIN ] ATI: Loaded 38269 existing records. [2025-09-16T17:16:56.381093] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-09. [2025-09-16T17:16:56.554327] [WARN ] [MAIN ] ATI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:16:56.554692] [INFO ] [MAIN ] ATI: Wiping existing primary data before full refresh. [2025-09-16T17:16:56.554739] [INFO ] [LOCAL ] ATI: Starting deletion of primary data. [2025-09-16T17:16:56.555936] [DEBUG] [LOCAL ] ATI: Deleted primary data file. [2025-09-16T17:16:56.555985] [INFO ] [MAIN ] ATI: Starting full primary refresh. [2025-09-16T17:16:56.907089] [INFO ] [REMOTE ] ATI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:57.174902] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-02. [2025-09-16T17:16:57.263753] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T17:16:57.865189] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T17:16:57.869730] [INFO ] [REMOTE ] ATEN: Full fetch complete (43673 records). [2025-09-16T17:16:57.869814] [DEBUG] [VALIDATOR ] ATEN: Validating primary data. [2025-09-16T17:16:57.981559] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-05. [2025-09-16T17:16:58.262618] [DEBUG] [VALIDATOR ] ATEN: All expected trading days are present. [2025-09-16T17:16:58.262712] [DEBUG] [VALIDATOR ] ATEN: Primary data validation passed. [2025-09-16T17:16:58.263184] [INFO ] [MAIN ] ATEN: Fetched primary data is valid. [2025-09-16T17:16:58.263225] [DEBUG] [LOCAL ] ATEN: Saving primary data. [2025-09-16T17:16:58.264191] [DEBUG] [LOCAL ] ATEN: Trimmed 43673 records to 28678 (keeping 150 days of data) [2025-09-16T17:16:58.451097] [DEBUG] [LOCAL ] ATEN: Primary data saved (28678 records). [2025-09-16T17:16:58.451233] [INFO ] [MAIN ] ATEN: Full history saved. [2025-09-16T17:16:58.453794] [INFO ] [MAIN ] Update Progress: 167/2369 symbols completed. ATEN took 26.5s. Estimated remaining: 3h 43m 8s. Rate: 9.9 symbols/min [2025-09-16T17:16:58.628183] [INFO ] [MAIN ] ATKR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:16:58.654874] [DEBUG] [LOCAL ] ATKR: Loaded 25276 existing records from primary storage. [2025-09-16T17:16:58.655011] [INFO ] [MAIN ] ATKR: Loaded 25276 existing records. [2025-09-16T17:16:58.775419] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-02. [2025-09-16T17:16:58.841090] [WARN ] [MAIN ] ATKR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:16:58.841800] [INFO ] [MAIN ] ATKR: Wiping existing primary data before full refresh. [2025-09-16T17:16:58.841880] [INFO ] [LOCAL ] ATKR: Starting deletion of primary data. [2025-09-16T17:16:58.843206] [DEBUG] [LOCAL ] ATKR: Deleted primary data file. [2025-09-16T17:16:58.843263] [INFO ] [MAIN ] ATKR: Starting full primary refresh. [2025-09-16T17:16:59.142939] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-09-16T17:16:59.250556] [INFO ] [REMOTE ] ATKR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:16:59.378447] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-09-16T17:16:59.564938] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T17:16:59.574866] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-03. [2025-09-16T17:17:00.092779] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-09-16T17:17:00.374913] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-03. [2025-09-16T17:17:01.175100] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-04. [2025-09-16T17:17:01.974929] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-02. [2025-09-16T17:17:02.368000] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:17:02.768461] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-03 (2.39 seconds). [2025-09-16T17:17:02.778782] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-05. [2025-09-16T17:17:03.546185] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-09-16T17:17:03.574931] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-06. [2025-09-16T17:17:03.663302] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T17:17:04.276242] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-09-16T17:17:04.374868] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-03. [2025-09-16T17:17:05.174890] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-06. [2025-09-16T17:17:05.602025] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-09-16T17:17:05.974916] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-07. [2025-09-16T17:17:06.228980] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T17:17:06.365053] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-09-16T17:17:06.774964] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-04. [2025-09-16T17:17:07.574949] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-04. [2025-09-16T17:17:08.366525] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-09-16T17:17:08.376908] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-08. [2025-09-16T17:17:08.893851] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-09-16T17:17:09.066323] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T17:17:09.174919] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-05. [2025-09-16T17:17:09.974917] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-07. [2025-09-16T17:17:10.774873] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-05. [2025-09-16T17:17:10.826095] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T17:17:11.161458] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-09-16T17:17:11.574904] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-09. [2025-09-16T17:17:11.783205] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T17:17:12.289917] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-09-16T17:17:12.291204] [INFO ] [REMOTE ] ATHS: Full fetch complete (7693 records). [2025-09-16T17:17:12.291275] [DEBUG] [VALIDATOR ] ATHS: Validating primary data. [2025-09-16T17:17:12.380052] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-06. [2025-09-16T17:17:12.672216] [DEBUG] [VALIDATOR ] ATHS: All expected trading days are present. [2025-09-16T17:17:12.672295] [DEBUG] [VALIDATOR ] ATHS: Primary data validation passed. [2025-09-16T17:17:12.672543] [INFO ] [MAIN ] ATHS: Fetched primary data is valid. [2025-09-16T17:17:12.672603] [DEBUG] [LOCAL ] ATHS: Saving primary data. [2025-09-16T17:17:12.673142] [DEBUG] [LOCAL ] ATHS: Trimmed 7693 records to 4952 (keeping 150 days of data) [2025-09-16T17:17:12.704905] [DEBUG] [LOCAL ] ATHS: Primary data saved (4952 records). [2025-09-16T17:17:12.704983] [INFO ] [MAIN ] ATHS: Full history saved. [2025-09-16T17:17:12.705052] [INFO ] [MAIN ] Update Progress: 168/2369 symbols completed. ATHS took 17.4s. Estimated remaining: 3h 44m 49s. Rate: 9.8 symbols/min [2025-09-16T17:17:12.885871] [INFO ] [MAIN ] ATMU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:17:12.886102] [DEBUG] [LOCAL ] ATMU: No existing data file found in primary storage. [2025-09-16T17:17:12.886143] [WARN ] [MAIN ] ATMU: No existing data found, performing full refresh instead. [2025-09-16T17:17:12.886282] [INFO ] [MAIN ] ATMU: Wiping existing primary data before full refresh. [2025-09-16T17:17:12.886324] [INFO ] [LOCAL ] ATMU: Starting deletion of primary data. [2025-09-16T17:17:12.886370] [DEBUG] [LOCAL ] ATMU: No primary data file found to delete. [2025-09-16T17:17:12.886414] [INFO ] [MAIN ] ATMU: Starting full primary refresh. [2025-09-16T17:17:13.179976] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-06. [2025-09-16T17:17:13.240813] [INFO ] [REMOTE ] ATMU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:17:13.889800] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-09-16T17:17:13.974851] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-07. [2025-09-16T17:17:14.597012] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T17:17:14.774931] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-07. [2025-09-16T17:17:15.517173] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-09-16T17:17:15.574888] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-08. [2025-09-16T17:17:16.241734] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T17:17:16.374948] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-02. [2025-09-16T17:17:17.174929] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-08. [2025-09-16T17:17:17.329199] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T17:17:17.752950] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T17:17:17.974951] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-03. [2025-09-16T17:17:18.529965] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-09-16T17:17:18.774921] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-08. [2025-09-16T17:17:19.498457] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T17:17:19.574931] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-09. [2025-09-16T17:17:20.042814] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-09-16T17:17:20.374891] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-09. [2025-09-16T17:17:20.600712] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-09-16T17:17:20.603846] [INFO ] [REMOTE ] ATHM: Full fetch complete (36762 records). [2025-09-16T17:17:20.603968] [DEBUG] [VALIDATOR ] ATHM: Validating primary data. [2025-09-16T17:17:20.604698] [FATAL] [VALIDATOR ] ATHM: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:17:20.604817] [FATAL] [MAIN ] ATHM: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:17:20.604869] [WARN ] [MAIN ] ATHM: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:17:20.604935] [INFO ] [LOCAL ] ATHM: Starting deletion of primary data. [2025-09-16T17:17:20.605012] [DEBUG] [LOCAL ] ATHM: No primary data file found to delete. [2025-09-16T17:17:20.605051] [INFO ] [LOCAL ] ATHM: Starting deletion of derivative data. [2025-09-16T17:17:20.605593] [DEBUG] [LOCAL ] ATHM: Deleted derivative data file. [2025-09-16T17:17:20.605706] [INFO ] [MAIN ] Update Progress: 169/2369 symbols completed. ATHM took 32.3s. Estimated remaining: 3h 45m 6s. Rate: 9.8 symbols/min [2025-09-16T17:17:20.784913] [INFO ] [MAIN ] ATO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:17:20.822575] [DEBUG] [LOCAL ] ATO: Loaded 30687 existing records from primary storage. [2025-09-16T17:17:20.822746] [INFO ] [MAIN ] ATO: Loaded 30687 existing records. [2025-09-16T17:17:21.007782] [WARN ] [MAIN ] ATO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:17:21.008077] [INFO ] [MAIN ] ATO: Wiping existing primary data before full refresh. [2025-09-16T17:17:21.008141] [INFO ] [LOCAL ] ATO: Starting deletion of primary data. [2025-09-16T17:17:21.009785] [DEBUG] [LOCAL ] ATO: Deleted primary data file. [2025-09-16T17:17:21.009849] [INFO ] [MAIN ] ATO: Starting full primary refresh. [2025-09-16T17:17:21.187173] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-09. [2025-09-16T17:17:21.370296] [INFO ] [REMOTE ] ATO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:17:21.974924] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-04. [2025-09-16T17:17:22.130020] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T17:17:22.136994] [INFO ] [REMOTE ] ATI: Full fetch complete (56309 records). [2025-09-16T17:17:22.137116] [DEBUG] [VALIDATOR ] ATI: Validating primary data. [2025-09-16T17:17:22.558781] [DEBUG] [VALIDATOR ] ATI: All expected trading days are present. [2025-09-16T17:17:22.559431] [DEBUG] [VALIDATOR ] ATI: Primary data validation passed. [2025-09-16T17:17:22.562525] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T17:17:22.560740] [INFO ] [MAIN ] ATI: Fetched primary data is valid. [2025-09-16T17:17:22.567425] [DEBUG] [LOCAL ] ATI: Saving primary data. [2025-09-16T17:17:22.568027] [INFO ] [REMOTE ] ATKR: Full fetch complete (39536 records). [2025-09-16T17:17:22.568144] [DEBUG] [VALIDATOR ] ATKR: Validating primary data. [2025-09-16T17:17:22.569413] [FATAL] [VALIDATOR ] ATKR: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:17:22.569585] [FATAL] [MAIN ] ATKR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:17:22.569733] [WARN ] [MAIN ] ATKR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:17:22.569957] [INFO ] [LOCAL ] ATKR: Starting deletion of primary data. [2025-09-16T17:17:22.570038] [DEBUG] [LOCAL ] ATKR: No primary data file found to delete. [2025-09-16T17:17:22.570081] [INFO ] [LOCAL ] ATKR: Starting deletion of derivative data. [2025-09-16T17:17:22.570264] [DEBUG] [LOCAL ] ATI: Trimmed 56309 records to 38648 (keeping 150 days of data) [2025-09-16T17:17:22.570765] [DEBUG] [LOCAL ] ATKR: Deleted derivative data file. [2025-09-16T17:17:22.570884] [INFO ] [MAIN ] Update Progress: 170/2369 symbols completed. ATKR took 24.1s. Estimated remaining: 3h 44m 6s. Rate: 9.8 symbols/min [2025-09-16T17:17:22.778706] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-02. [2025-09-16T17:17:22.857120] [INFO ] [MAIN ] ATR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:17:22.897813] [DEBUG] [LOCAL ] ATR: Loaded 22944 existing records from primary storage. [2025-09-16T17:17:22.898044] [INFO ] [MAIN ] ATR: Loaded 22944 existing records. [2025-09-16T17:17:23.162059] [WARN ] [MAIN ] ATR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:17:23.167151] [INFO ] [MAIN ] ATR: Wiping existing primary data before full refresh. [2025-09-16T17:17:23.167758] [INFO ] [LOCAL ] ATR: Starting deletion of primary data. [2025-09-16T17:17:23.169553] [DEBUG] [LOCAL ] ATR: Deleted primary data file. [2025-09-16T17:17:23.170111] [INFO ] [MAIN ] ATR: Starting full primary refresh. [2025-09-16T17:17:23.543363] [INFO ] [REMOTE ] ATR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:17:23.552083] [DEBUG] [LOCAL ] ATI: Primary data saved (38648 records). [2025-09-16T17:17:23.552209] [INFO ] [MAIN ] ATI: Full history saved. [2025-09-16T17:17:23.555623] [INFO ] [MAIN ] Update Progress: 171/2369 symbols completed. ATI took 27.4s. Estimated remaining: 3h 42m 54s. Rate: 9.9 symbols/min [2025-09-16T17:17:23.580043] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-02. [2025-09-16T17:17:23.700290] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T17:17:23.764620] [INFO ] [MAIN ] ATS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:17:23.781006] [DEBUG] [LOCAL ] ATS: Loaded 12990 existing records from primary storage. [2025-09-16T17:17:23.781173] [INFO ] [MAIN ] ATS: Loaded 12990 existing records. [2025-09-16T17:17:23.960163] [WARN ] [MAIN ] ATS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:17:23.960460] [INFO ] [MAIN ] ATS: Wiping existing primary data before full refresh. [2025-09-16T17:17:23.960507] [INFO ] [LOCAL ] ATS: Starting deletion of primary data. [2025-09-16T17:17:23.961296] [DEBUG] [LOCAL ] ATS: Deleted primary data file. [2025-09-16T17:17:23.961348] [INFO ] [MAIN ] ATS: Starting full primary refresh. [2025-09-16T17:17:24.326052] [INFO ] [REMOTE ] ATS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:17:24.372184] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T17:17:24.379800] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-02. [2025-09-16T17:17:24.693436] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-09-16T17:17:25.174891] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-03. [2025-09-16T17:17:25.257283] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-09-16T17:17:25.974962] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-03. [2025-09-16T17:17:26.411128] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-09-16T17:17:26.774874] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-04. [2025-09-16T17:17:27.076926] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T17:17:27.574966] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-03. [2025-09-16T17:17:28.037851] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-09-16T17:17:28.374870] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-05. [2025-09-16T17:17:29.002979] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-09-16T17:17:29.174878] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-04. [2025-09-16T17:17:29.670097] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T17:17:29.974931] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-05. [2025-09-16T17:17:30.100348] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T17:17:30.774899] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-04. [2025-09-16T17:17:31.110483] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T17:17:31.574960] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-05. [2025-09-16T17:17:32.124334] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T17:17:32.217305] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-09-16T17:17:32.374925] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-06. [2025-09-16T17:17:33.174916] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-06. [2025-09-16T17:17:33.272773] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-09-16T17:17:33.974931] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-06. [2025-09-16T17:17:34.284340] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-09-16T17:17:34.774906] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-07. [2025-09-16T17:17:35.517601] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-09-16T17:17:35.567931] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T17:17:35.574838] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-08. [2025-09-16T17:17:36.374937] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-07. [2025-09-16T17:17:36.511572] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-09-16T17:17:37.174897] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-09. [2025-09-16T17:17:37.847561] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T17:17:37.974887] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-08. [2025-09-16T17:17:38.149769] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T17:17:38.151884] [INFO ] [REMOTE ] ATS: Full fetch complete (19547 records). [2025-09-16T17:17:38.151953] [DEBUG] [VALIDATOR ] ATS: Validating primary data. [2025-09-16T17:17:38.513932] [DEBUG] [VALIDATOR ] ATS: All expected trading days are present. [2025-09-16T17:17:38.514009] [DEBUG] [VALIDATOR ] ATS: Primary data validation passed. [2025-09-16T17:17:38.514327] [INFO ] [MAIN ] ATS: Fetched primary data is valid. [2025-09-16T17:17:38.514381] [DEBUG] [LOCAL ] ATS: Saving primary data. [2025-09-16T17:17:38.515113] [DEBUG] [LOCAL ] ATS: Trimmed 19547 records to 13086 (keeping 150 days of data) [2025-09-16T17:17:38.590684] [DEBUG] [LOCAL ] ATS: Primary data saved (13086 records). [2025-09-16T17:17:38.590821] [INFO ] [MAIN ] ATS: Full history saved. [2025-09-16T17:17:38.590912] [INFO ] [MAIN ] Update Progress: 172/2369 symbols completed. ATS took 15.0s. Estimated remaining: 3h 44m 42s. Rate: 9.8 symbols/min [2025-09-16T17:17:38.758601] [INFO ] [MAIN ] ATUS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:17:38.779814] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-05. [2025-09-16T17:17:38.796256] [DEBUG] [LOCAL ] ATUS: Loaded 37048 existing records from primary storage. [2025-09-16T17:17:38.796389] [INFO ] [MAIN ] ATUS: Loaded 37048 existing records. [2025-09-16T17:17:38.971357] [WARN ] [MAIN ] ATUS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:17:38.971581] [INFO ] [MAIN ] ATUS: Wiping existing primary data before full refresh. [2025-09-16T17:17:38.971628] [INFO ] [LOCAL ] ATUS: Starting deletion of primary data. [2025-09-16T17:17:38.972873] [DEBUG] [LOCAL ] ATUS: Deleted primary data file. [2025-09-16T17:17:38.972925] [INFO ] [MAIN ] ATUS: Starting full primary refresh. [2025-09-16T17:17:39.324878] [INFO ] [REMOTE ] ATUS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:17:39.440886] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-09-16T17:17:39.574860] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-02. [2025-09-16T17:17:40.266589] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T17:17:40.374931] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-07. [2025-09-16T17:17:41.174916] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-06. [2025-09-16T17:17:41.213225] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T17:17:41.838783] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T17:17:41.974882] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-08. [2025-09-16T17:17:42.465665] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T17:17:42.774915] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-07. [2025-09-16T17:17:43.474578] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T17:17:43.574935] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-03. [2025-09-16T17:17:43.700191] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-09-16T17:17:44.374971] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-09. [2025-09-16T17:17:44.843321] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T17:17:45.174964] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-08. [2025-09-16T17:17:45.615218] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T17:17:45.620423] [INFO ] [REMOTE ] ATO: Full fetch complete (45461 records). [2025-09-16T17:17:45.620927] [DEBUG] [VALIDATOR ] ATO: Validating primary data. [2025-09-16T17:17:45.622429] [FATAL] [VALIDATOR ] ATO: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:17:45.622546] [FATAL] [MAIN ] ATO: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:17:45.622585] [WARN ] [MAIN ] ATO: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:17:45.622633] [INFO ] [LOCAL ] ATO: Starting deletion of primary data. [2025-09-16T17:17:45.622719] [DEBUG] [LOCAL ] ATO: No primary data file found to delete. [2025-09-16T17:17:45.622769] [INFO ] [LOCAL ] ATO: Starting deletion of derivative data. [2025-09-16T17:17:45.623127] [DEBUG] [LOCAL ] ATO: Deleted derivative data file. [2025-09-16T17:17:45.624993] [INFO ] [MAIN ] Update Progress: 173/2369 symbols completed. ATO took 25.0s. Estimated remaining: 3h 44m 47s. Rate: 9.8 symbols/min [2025-09-16T17:17:45.796660] [INFO ] [MAIN ] AU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:17:45.838672] [DEBUG] [LOCAL ] AU: Loaded 39087 existing records from primary storage. [2025-09-16T17:17:45.838828] [INFO ] [MAIN ] AU: Loaded 39087 existing records. [2025-09-16T17:17:45.980479] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-09. [2025-09-16T17:17:46.019360] [WARN ] [MAIN ] AU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:17:46.019601] [INFO ] [MAIN ] AU: Wiping existing primary data before full refresh. [2025-09-16T17:17:46.019666] [INFO ] [LOCAL ] AU: Starting deletion of primary data. [2025-09-16T17:17:46.021329] [DEBUG] [LOCAL ] AU: Deleted primary data file. [2025-09-16T17:17:46.021381] [INFO ] [MAIN ] AU: Starting full primary refresh. [2025-09-16T17:17:46.384788] [INFO ] [REMOTE ] AU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:17:46.436423] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-09-16T17:17:46.774970] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-02. [2025-09-16T17:17:47.241206] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T17:17:47.246681] [INFO ] [REMOTE ] ATMU: Full fetch complete (43354 records). [2025-09-16T17:17:47.246770] [DEBUG] [VALIDATOR ] ATMU: Validating primary data. [2025-09-16T17:17:47.580061] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-09. [2025-09-16T17:17:47.618345] [DEBUG] [VALIDATOR ] ATMU: All expected trading days are present. [2025-09-16T17:17:47.618429] [DEBUG] [VALIDATOR ] ATMU: Primary data validation passed. [2025-09-16T17:17:47.618964] [INFO ] [MAIN ] ATMU: Fetched primary data is valid. [2025-09-16T17:17:47.619012] [DEBUG] [LOCAL ] ATMU: Saving primary data. [2025-09-16T17:17:47.620147] [DEBUG] [LOCAL ] ATMU: Trimmed 43354 records to 28708 (keeping 150 days of data) [2025-09-16T17:17:47.798700] [DEBUG] [LOCAL ] ATMU: Primary data saved (28708 records). [2025-09-16T17:17:47.798849] [INFO ] [MAIN ] ATMU: Full history saved. [2025-09-16T17:17:47.798927] [INFO ] [MAIN ] Update Progress: 174/2369 symbols completed. ATMU took 35.1s. Estimated remaining: 3h 43m 51s. Rate: 9.8 symbols/min [2025-09-16T17:17:47.968533] [INFO ] [MAIN ] AUB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:17:48.002895] [DEBUG] [LOCAL ] AUB: Loaded 33612 existing records from primary storage. [2025-09-16T17:17:48.003036] [INFO ] [MAIN ] AUB: Loaded 33612 existing records. [2025-09-16T17:17:48.177432] [WARN ] [MAIN ] AUB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:17:48.177745] [INFO ] [MAIN ] AUB: Wiping existing primary data before full refresh. [2025-09-16T17:17:48.177793] [INFO ] [LOCAL ] AUB: Starting deletion of primary data. [2025-09-16T17:17:48.179129] [DEBUG] [LOCAL ] AUB: Deleted primary data file. [2025-09-16T17:17:48.179180] [INFO ] [MAIN ] AUB: Starting full primary refresh. [2025-09-16T17:17:48.383712] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-04. [2025-09-16T17:17:48.529919] [INFO ] [REMOTE ] AUB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:17:48.893849] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-02 (2.12 seconds). [2025-09-16T17:17:49.050150] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T17:17:49.054275] [INFO ] [REMOTE ] ATR: Full fetch complete (33389 records). [2025-09-16T17:17:49.054365] [DEBUG] [VALIDATOR ] ATR: Validating primary data. [2025-09-16T17:17:49.180039] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-02. [2025-09-16T17:17:49.434280] [DEBUG] [VALIDATOR ] ATR: All expected trading days are present. [2025-09-16T17:17:49.434365] [DEBUG] [VALIDATOR ] ATR: Primary data validation passed. [2025-09-16T17:17:49.434829] [INFO ] [MAIN ] ATR: Fetched primary data is valid. [2025-09-16T17:17:49.434876] [DEBUG] [LOCAL ] ATR: Saving primary data. [2025-09-16T17:17:49.435790] [DEBUG] [LOCAL ] ATR: Trimmed 33389 records to 23182 (keeping 150 days of data) [2025-09-16T17:17:49.588996] [DEBUG] [LOCAL ] ATR: Primary data saved (23182 records). [2025-09-16T17:17:49.589138] [INFO ] [MAIN ] ATR: Full history saved. [2025-09-16T17:17:49.589249] [INFO ] [MAIN ] Update Progress: 175/2369 symbols completed. ATR took 27.0s. Estimated remaining: 3h 42m 51s. Rate: 9.8 symbols/min [2025-09-16T17:17:49.760493] [INFO ] [MAIN ] AUNA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:17:49.765558] [DEBUG] [LOCAL ] AUNA: Loaded 3616 existing records from primary storage. [2025-09-16T17:17:49.765668] [INFO ] [MAIN ] AUNA: Loaded 3616 existing records. [2025-09-16T17:17:49.940147] [WARN ] [MAIN ] AUNA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:17:49.940401] [INFO ] [MAIN ] AUNA: Wiping existing primary data before full refresh. [2025-09-16T17:17:49.940450] [INFO ] [LOCAL ] AUNA: Starting deletion of primary data. [2025-09-16T17:17:49.940810] [DEBUG] [LOCAL ] AUNA: Deleted primary data file. [2025-09-16T17:17:49.940861] [INFO ] [MAIN ] AUNA: Starting full primary refresh. [2025-09-16T17:17:49.984577] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-03. [2025-09-16T17:17:50.313219] [INFO ] [REMOTE ] AUNA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:17:50.353304] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-09-16T17:17:50.725709] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T17:17:50.774852] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-03. [2025-09-16T17:17:51.574893] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-05. [2025-09-16T17:17:51.667925] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-09-16T17:17:52.374918] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-02. [2025-09-16T17:17:52.691383] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-09-16T17:17:53.028680] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T17:17:53.081093] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-09-16T17:17:53.174886] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-03. [2025-09-16T17:17:53.751033] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T17:17:53.974912] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-06. [2025-09-16T17:17:54.774905] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-04. [2025-09-16T17:17:55.580134] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-04. [2025-09-16T17:17:55.598905] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T17:17:56.374926] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-07. [2025-09-16T17:17:56.811431] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-09-16T17:17:57.174910] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-05. [2025-09-16T17:17:57.518679] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-09-16T17:17:57.974909] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-05. [2025-09-16T17:17:58.088137] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T17:17:58.709919] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T17:17:58.774852] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-06. [2025-09-16T17:17:59.574912] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-08. [2025-09-16T17:18:00.247398] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-05 (2.27 seconds). [2025-09-16T17:18:00.304207] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-09-16T17:18:00.374855] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-07. [2025-09-16T17:18:01.174925] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-06. [2025-09-16T17:18:01.529716] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-09-16T17:18:01.974903] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-09. [2025-09-16T17:18:02.407788] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-09-16T17:18:02.780209] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-08. [2025-09-16T17:18:02.810910] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T17:18:03.574934] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-04. [2025-09-16T17:18:03.735392] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T17:18:03.741355] [INFO ] [REMOTE ] ATUS: Full fetch complete (53134 records). [2025-09-16T17:18:03.741477] [DEBUG] [VALIDATOR ] ATUS: Validating primary data. [2025-09-16T17:18:03.742214] [FATAL] [VALIDATOR ] ATUS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:18:03.742306] [FATAL] [MAIN ] ATUS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:18:03.742348] [WARN ] [MAIN ] ATUS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:18:03.742541] [INFO ] [LOCAL ] ATUS: Starting deletion of primary data. [2025-09-16T17:18:03.742927] [DEBUG] [LOCAL ] ATUS: No primary data file found to delete. [2025-09-16T17:18:03.742994] [INFO ] [LOCAL ] ATUS: Starting deletion of derivative data. [2025-09-16T17:18:03.743310] [DEBUG] [LOCAL ] ATUS: Deleted derivative data file. [2025-09-16T17:18:03.744340] [INFO ] [MAIN ] Update Progress: 176/2369 symbols completed. ATUS took 25.2s. Estimated remaining: 3h 44m 25s. Rate: 9.8 symbols/min [2025-09-16T17:18:03.920253] [INFO ] [MAIN ] AVA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:03.954856] [DEBUG] [LOCAL ] AVA: Loaded 28643 existing records from primary storage. [2025-09-16T17:18:03.955011] [INFO ] [MAIN ] AVA: Loaded 28643 existing records. [2025-09-16T17:18:04.135280] [WARN ] [MAIN ] AVA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:18:04.135838] [INFO ] [MAIN ] AVA: Wiping existing primary data before full refresh. [2025-09-16T17:18:04.135938] [INFO ] [LOCAL ] AVA: Starting deletion of primary data. [2025-09-16T17:18:04.137230] [DEBUG] [LOCAL ] AVA: Deleted primary data file. [2025-09-16T17:18:04.137284] [INFO ] [MAIN ] AVA: Starting full primary refresh. [2025-09-16T17:18:04.331464] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-09-16T17:18:04.381413] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-07. [2025-09-16T17:18:04.423795] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T17:18:04.572108] [INFO ] [REMOTE ] AVA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:05.174924] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-02. [2025-09-16T17:18:05.974942] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-05. [2025-09-16T17:18:06.517713] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-09-16T17:18:06.589374] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-07 (2.21 seconds). [2025-09-16T17:18:06.655085] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T17:18:06.774935] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-08. [2025-09-16T17:18:07.574932] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-03. [2025-09-16T17:18:08.374938] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-06. [2025-09-16T17:18:08.811751] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-09-16T17:18:08.905713] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-09-16T17:18:09.136095] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T17:18:09.174855] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-04. [2025-09-16T17:18:09.974912] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-07. [2025-09-16T17:18:10.650878] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-09-16T17:18:10.688782] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T17:18:10.774911] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-05. [2025-09-16T17:18:11.574949] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-09. [2025-09-16T17:18:11.975452] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T17:18:12.374925] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-06. [2025-09-16T17:18:13.174936] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-09. [2025-09-16T17:18:13.620485] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T17:18:13.644251] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-09-16T17:18:13.650533] [INFO ] [REMOTE ] AU: Full fetch complete (57430 records). [2025-09-16T17:18:13.650664] [DEBUG] [VALIDATOR ] AU: Validating primary data. [2025-09-16T17:18:13.988486] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-08. [2025-09-16T17:18:14.041877] [DEBUG] [VALIDATOR ] AU: All expected trading days are present. [2025-09-16T17:18:14.041967] [DEBUG] [VALIDATOR ] AU: Primary data validation passed. [2025-09-16T17:18:14.042618] [INFO ] [MAIN ] AU: Fetched primary data is valid. [2025-09-16T17:18:14.042714] [DEBUG] [LOCAL ] AU: Saving primary data. [2025-09-16T17:18:14.044006] [DEBUG] [LOCAL ] AU: Trimmed 57430 records to 39477 (keeping 150 days of data) [2025-09-16T17:18:14.296627] [DEBUG] [LOCAL ] AU: Primary data saved (39477 records). [2025-09-16T17:18:14.296811] [INFO ] [MAIN ] AU: Full history saved. [2025-09-16T17:18:14.296918] [INFO ] [MAIN ] Update Progress: 177/2369 symbols completed. AU took 28.7s. Estimated remaining: 3h 45m 14s. Rate: 9.7 symbols/min [2025-09-16T17:18:14.472075] [INFO ] [MAIN ] AVAL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:14.479942] [DEBUG] [LOCAL ] AVAL: Loaded 5769 existing records from primary storage. [2025-09-16T17:18:14.480116] [INFO ] [MAIN ] AVAL: Loaded 5769 existing records. [2025-09-16T17:18:14.570319] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:18:14.603420] [INFO ] [REMOTE ] AUB: Full fetch complete (50225 records). [2025-09-16T17:18:14.609530] [DEBUG] [VALIDATOR ] AUB: Validating primary data. [2025-09-16T17:18:14.771297] [WARN ] [MAIN ] AVAL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:18:14.781845] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-07. [2025-09-16T17:18:14.792974] [INFO ] [MAIN ] AVAL: Wiping existing primary data before full refresh. [2025-09-16T17:18:14.796519] [INFO ] [LOCAL ] AVAL: Starting deletion of primary data. [2025-09-16T17:18:14.798381] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T17:18:14.809079] [DEBUG] [LOCAL ] AVAL: Deleted primary data file. [2025-09-16T17:18:14.814280] [INFO ] [MAIN ] AVAL: Starting full primary refresh. [2025-09-16T17:18:15.336089] [DEBUG] [VALIDATOR ] AUB: All expected trading days are present. [2025-09-16T17:18:15.347576] [DEBUG] [VALIDATOR ] AUB: Primary data validation passed. [2025-09-16T17:18:15.364785] [INFO ] [MAIN ] AUB: Fetched primary data is valid. [2025-09-16T17:18:15.372807] [DEBUG] [LOCAL ] AUB: Saving primary data. [2025-09-16T17:18:15.490438] [DEBUG] [LOCAL ] AUB: Trimmed 50225 records to 33888 (keeping 150 days of data) [2025-09-16T17:18:15.495111] [INFO ] [REMOTE ] AVAL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:15.591180] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-02. [2025-09-16T17:18:15.709760] [DEBUG] [LOCAL ] AUB: Primary data saved (33888 records). [2025-09-16T17:18:15.709919] [INFO ] [MAIN ] AUB: Full history saved. [2025-09-16T17:18:15.710018] [INFO ] [MAIN ] Update Progress: 178/2369 symbols completed. AUB took 27.9s. Estimated remaining: 3h 44m 9s. Rate: 9.8 symbols/min [2025-09-16T17:18:15.878678] [INFO ] [MAIN ] AVB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:15.911127] [DEBUG] [LOCAL ] AVB: Loaded 30819 existing records from primary storage. [2025-09-16T17:18:15.911275] [INFO ] [MAIN ] AVB: Loaded 30819 existing records. [2025-09-16T17:18:16.086604] [WARN ] [MAIN ] AVB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:18:16.086885] [INFO ] [MAIN ] AVB: Wiping existing primary data before full refresh. [2025-09-16T17:18:16.086932] [INFO ] [LOCAL ] AVB: Starting deletion of primary data. [2025-09-16T17:18:16.088406] [DEBUG] [LOCAL ] AVB: Deleted primary data file. [2025-09-16T17:18:16.088458] [INFO ] [MAIN ] AVB: Starting full primary refresh. [2025-09-16T17:18:16.379908] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-09. [2025-09-16T17:18:16.441279] [INFO ] [REMOTE ] AVB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:16.450714] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T17:18:16.477144] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T17:18:16.958070] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-09-16T17:18:16.959289] [INFO ] [REMOTE ] AUNA: Full fetch complete (5195 records). [2025-09-16T17:18:16.959357] [DEBUG] [VALIDATOR ] AUNA: Validating primary data. [2025-09-16T17:18:17.180051] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-02. [2025-09-16T17:18:17.321147] [DEBUG] [VALIDATOR ] AUNA: All expected trading days are present. [2025-09-16T17:18:17.321298] [DEBUG] [VALIDATOR ] AUNA: Primary data validation passed. [2025-09-16T17:18:17.321469] [INFO ] [MAIN ] AUNA: Fetched primary data is valid. [2025-09-16T17:18:17.321515] [DEBUG] [LOCAL ] AUNA: Saving primary data. [2025-09-16T17:18:17.321925] [DEBUG] [LOCAL ] AUNA: Trimmed 5195 records to 3645 (keeping 150 days of data) [2025-09-16T17:18:17.343037] [DEBUG] [LOCAL ] AUNA: Primary data saved (3645 records). [2025-09-16T17:18:17.343108] [INFO ] [MAIN ] AUNA: Full history saved. [2025-09-16T17:18:17.343211] [INFO ] [MAIN ] Update Progress: 179/2369 symbols completed. AUNA took 27.8s. Estimated remaining: 3h 43m 8s. Rate: 9.8 symbols/min [2025-09-16T17:18:17.512149] [INFO ] [MAIN ] AVBC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:17.512346] [DEBUG] [LOCAL ] AVBC: No existing data file found in primary storage. [2025-09-16T17:18:17.512385] [WARN ] [MAIN ] AVBC: No existing data found, performing full refresh instead. [2025-09-16T17:18:17.512510] [INFO ] [MAIN ] AVBC: Wiping existing primary data before full refresh. [2025-09-16T17:18:17.512550] [INFO ] [LOCAL ] AVBC: Starting deletion of primary data. [2025-09-16T17:18:17.512592] [DEBUG] [LOCAL ] AVBC: No primary data file found to delete. [2025-09-16T17:18:17.512624] [INFO ] [MAIN ] AVBC: Starting full primary refresh. [2025-09-16T17:18:17.865088] [INFO ] [REMOTE ] AVBC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:17.974946] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-08. [2025-09-16T17:18:18.579105] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-09-16T17:18:18.774910] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-03. [2025-09-16T17:18:19.502303] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T17:18:19.574856] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-09. [2025-09-16T17:18:20.379949] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-03. [2025-09-16T17:18:20.395302] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T17:18:20.795320] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-09-16T17:18:20.800042] [INFO ] [REMOTE ] AVA: Full fetch complete (42638 records). [2025-09-16T17:18:20.800136] [DEBUG] [VALIDATOR ] AVA: Validating primary data. [2025-09-16T17:18:20.800789] [FATAL] [VALIDATOR ] AVA: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:18:20.800866] [FATAL] [MAIN ] AVA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:18:20.800903] [WARN ] [MAIN ] AVA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:18:20.800941] [INFO ] [LOCAL ] AVA: Starting deletion of primary data. [2025-09-16T17:18:20.801007] [DEBUG] [LOCAL ] AVA: No primary data file found to delete. [2025-09-16T17:18:20.801052] [INFO ] [LOCAL ] AVA: Starting deletion of derivative data. [2025-09-16T17:18:20.801277] [DEBUG] [LOCAL ] AVA: Deleted derivative data file. [2025-09-16T17:18:20.802547] [INFO ] [MAIN ] Update Progress: 180/2369 symbols completed. AVA took 17.1s. Estimated remaining: 3h 42m 29s. Rate: 9.8 symbols/min [2025-09-16T17:18:20.979520] [INFO ] [MAIN ] AVD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:21.001982] [DEBUG] [LOCAL ] AVD: Loaded 16275 existing records from primary storage. [2025-09-16T17:18:21.002113] [INFO ] [MAIN ] AVD: Loaded 16275 existing records. [2025-09-16T17:18:21.176807] [WARN ] [MAIN ] AVD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:18:21.176632] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-04. [2025-09-16T17:18:21.177805] [INFO ] [MAIN ] AVD: Wiping existing primary data before full refresh. [2025-09-16T17:18:21.177864] [INFO ] [LOCAL ] AVD: Starting deletion of primary data. [2025-09-16T17:18:21.178576] [DEBUG] [LOCAL ] AVD: Deleted primary data file. [2025-09-16T17:18:21.178624] [INFO ] [MAIN ] AVD: Starting full primary refresh. [2025-09-16T17:18:21.556828] [INFO ] [REMOTE ] AVD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:21.553466] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T17:18:21.974918] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-02. [2025-09-16T17:18:22.195193] [DEBUG] [REMOTE ] AVBC: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:18:22.195502] [WARN ] [REMOTE ] AVBC: No data returned for month 2025-02 [2025-09-16T17:18:22.774903] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-04. [2025-09-16T17:18:22.922869] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-09-16T17:18:23.352274] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-09-16T17:18:23.574924] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-03. [2025-09-16T17:18:23.796383] [DEBUG] [REMOTE ] AVBC: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:18:23.796756] [WARN ] [REMOTE ] AVBC: No data returned for month 2025-03 [2025-09-16T17:18:24.374921] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-05. [2025-09-16T17:18:24.941042] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-09-16T17:18:25.174901] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-02. [2025-09-16T17:18:25.974857] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-05. [2025-09-16T17:18:26.277931] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T17:18:26.774959] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-03. [2025-09-16T17:18:27.112666] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T17:18:27.574908] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-06. [2025-09-16T17:18:28.071397] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T17:18:28.374869] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-04. [2025-09-16T17:18:28.574562] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T17:18:29.174900] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-04. [2025-09-16T17:18:29.361433] [DEBUG] [REMOTE ] AVBC: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:18:29.361717] [WARN ] [REMOTE ] AVBC: No data returned for month 2025-04 [2025-09-16T17:18:29.564108] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:18:29.974920] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-05. [2025-09-16T17:18:30.774921] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-07. [2025-09-16T17:18:31.099577] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T17:18:31.385968] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-09-16T17:18:31.574912] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-08. [2025-09-16T17:18:32.295430] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-09-16T17:18:32.374932] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-06. [2025-09-16T17:18:33.174937] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-06. [2025-09-16T17:18:33.279340] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-09-16T17:18:33.974924] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-09. [2025-09-16T17:18:34.601935] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T17:18:34.727224] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-09-16T17:18:34.728524] [INFO ] [REMOTE ] AVAL: Full fetch complete (10959 records). [2025-09-16T17:18:34.728597] [DEBUG] [VALIDATOR ] AVAL: Validating primary data. [2025-09-16T17:18:34.729181] [FATAL] [VALIDATOR ] AVAL: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:18:34.729264] [FATAL] [MAIN ] AVAL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:18:34.729303] [WARN ] [MAIN ] AVAL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:18:34.729355] [INFO ] [LOCAL ] AVAL: Starting deletion of primary data. [2025-09-16T17:18:34.729434] [DEBUG] [LOCAL ] AVAL: No primary data file found to delete. [2025-09-16T17:18:34.729475] [INFO ] [LOCAL ] AVAL: Starting deletion of derivative data. [2025-09-16T17:18:34.729739] [DEBUG] [LOCAL ] AVAL: Deleted derivative data file. [2025-09-16T17:18:34.729835] [INFO ] [MAIN ] Update Progress: 181/2369 symbols completed. AVAL took 20.4s. Estimated remaining: 3h 43m 58s. Rate: 9.8 symbols/min [2025-09-16T17:18:34.783927] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-07. [2025-09-16T17:18:34.904325] [INFO ] [MAIN ] AVK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:34.920139] [DEBUG] [LOCAL ] AVK: Loaded 13357 existing records from primary storage. [2025-09-16T17:18:34.920296] [INFO ] [MAIN ] AVK: Loaded 13357 existing records. [2025-09-16T17:18:35.107892] [WARN ] [MAIN ] AVK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:18:35.108263] [INFO ] [MAIN ] AVK: Wiping existing primary data before full refresh. [2025-09-16T17:18:35.108313] [INFO ] [LOCAL ] AVK: Starting deletion of primary data. [2025-09-16T17:18:35.109290] [DEBUG] [LOCAL ] AVK: Deleted primary data file. [2025-09-16T17:18:35.109369] [INFO ] [MAIN ] AVK: Starting full primary refresh. [2025-09-16T17:18:35.463623] [INFO ] [REMOTE ] AVK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:35.574897] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-05. [2025-09-16T17:18:35.714825] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-09-16T17:18:35.796933] [DEBUG] [REMOTE ] AVBC: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:18:35.797195] [WARN ] [REMOTE ] AVBC: No data returned for month 2025-05 [2025-09-16T17:18:36.374940] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-07. [2025-09-16T17:18:37.174969] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-08. [2025-09-16T17:18:37.725211] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T17:18:37.975040] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-06. [2025-09-16T17:18:38.167605] [DEBUG] [REMOTE ] AVBC: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:18:38.167956] [WARN ] [REMOTE ] AVBC: No data returned for month 2025-06 [2025-09-16T17:18:38.335116] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T17:18:38.774943] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-02. [2025-09-16T17:18:39.574945] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-09. [2025-09-16T17:18:39.618687] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T17:18:40.374891] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-08. [2025-09-16T17:18:40.601483] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-09-16T17:18:40.604117] [INFO ] [REMOTE ] AVD: Full fetch complete (24590 records). [2025-09-16T17:18:40.604208] [DEBUG] [VALIDATOR ] AVD: Validating primary data. [2025-09-16T17:18:40.604779] [FATAL] [VALIDATOR ] AVD: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:18:40.604866] [FATAL] [MAIN ] AVD: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:18:40.604904] [WARN ] [MAIN ] AVD: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:18:40.604950] [INFO ] [LOCAL ] AVD: Starting deletion of primary data. [2025-09-16T17:18:40.605008] [DEBUG] [LOCAL ] AVD: No primary data file found to delete. [2025-09-16T17:18:40.605044] [INFO ] [LOCAL ] AVD: Starting deletion of derivative data. [2025-09-16T17:18:40.605317] [DEBUG] [LOCAL ] AVD: Deleted derivative data file. [2025-09-16T17:18:40.605409] [INFO ] [MAIN ] Update Progress: 182/2369 symbols completed. AVD took 19.8s. Estimated remaining: 3h 43m 49s. Rate: 9.8 symbols/min [2025-09-16T17:18:40.775163] [INFO ] [MAIN ] AVNS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:40.797885] [DEBUG] [LOCAL ] AVNS: Loaded 24755 existing records from primary storage. [2025-09-16T17:18:40.798011] [INFO ] [MAIN ] AVNS: Loaded 24755 existing records. [2025-09-16T17:18:40.976624] [WARN ] [MAIN ] AVNS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:18:40.976877] [INFO ] [MAIN ] AVNS: Wiping existing primary data before full refresh. [2025-09-16T17:18:40.976923] [INFO ] [LOCAL ] AVNS: Starting deletion of primary data. [2025-09-16T17:18:40.977881] [DEBUG] [LOCAL ] AVNS: Deleted primary data file. [2025-09-16T17:18:40.977932] [INFO ] [MAIN ] AVNS: Starting full primary refresh. [2025-09-16T17:18:41.182355] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-03. [2025-09-16T17:18:41.329484] [INFO ] [REMOTE ] AVNS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:41.974869] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-07. [2025-09-16T17:18:42.072283] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-09-16T17:18:42.199906] [DEBUG] [REMOTE ] AVBC: Error fetching primary data for 2025-07: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:18:42.200213] [WARN ] [REMOTE ] AVBC: No data returned for month 2025-07 [2025-09-16T17:18:42.256474] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-09-16T17:18:42.774914] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-09. [2025-09-16T17:18:43.574904] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-08. [2025-09-16T17:18:44.380402] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-02. [2025-09-16T17:18:44.390029] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T17:18:44.396143] [INFO ] [REMOTE ] AVB: Full fetch complete (44681 records). [2025-09-16T17:18:44.396251] [DEBUG] [VALIDATOR ] AVB: Validating primary data. [2025-09-16T17:18:44.786442] [DEBUG] [VALIDATOR ] AVB: All expected trading days are present. [2025-09-16T17:18:44.786544] [DEBUG] [VALIDATOR ] AVB: Primary data validation passed. [2025-09-16T17:18:44.787158] [INFO ] [MAIN ] AVB: Fetched primary data is valid. [2025-09-16T17:18:44.787299] [DEBUG] [LOCAL ] AVB: Saving primary data. [2025-09-16T17:18:44.799483] [INFO ] [REMOTE ] AVBC: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-09-16T17:18:44.805459] [DEBUG] [LOCAL ] AVB: Trimmed 44681 records to 31165 (keeping 150 days of data) [2025-09-16T17:18:45.032179] [DEBUG] [LOCAL ] AVB: Primary data saved (31165 records). [2025-09-16T17:18:45.032402] [INFO ] [MAIN ] AVB: Full history saved. [2025-09-16T17:18:45.032515] [INFO ] [MAIN ] Update Progress: 183/2369 symbols completed. AVB took 29.3s. Estimated remaining: 3h 43m 22s. Rate: 9.8 symbols/min [2025-09-16T17:18:45.183792] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-04. [2025-09-16T17:18:45.221501] [INFO ] [MAIN ] AVNT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:45.260083] [DEBUG] [LOCAL ] AVNT: Loaded 28547 existing records from primary storage. [2025-09-16T17:18:45.260315] [INFO ] [MAIN ] AVNT: Loaded 28547 existing records. [2025-09-16T17:18:45.449517] [WARN ] [MAIN ] AVNT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:18:45.449840] [INFO ] [MAIN ] AVNT: Wiping existing primary data before full refresh. [2025-09-16T17:18:45.449892] [INFO ] [LOCAL ] AVNT: Starting deletion of primary data. [2025-09-16T17:18:45.451517] [DEBUG] [LOCAL ] AVNT: Deleted primary data file. [2025-09-16T17:18:45.451578] [INFO ] [MAIN ] AVNT: Starting full primary refresh. [2025-09-16T17:18:45.812533] [INFO ] [REMOTE ] AVNT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:45.850804] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T17:18:45.920095] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-09-16T17:18:45.974905] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-05. [2025-09-16T17:18:46.774911] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-02. [2025-09-16T17:18:46.949603] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T17:18:47.574903] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-06. [2025-09-16T17:18:47.965835] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T17:18:48.374917] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-03. [2025-09-16T17:18:48.626292] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T17:18:49.174909] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-03. [2025-09-16T17:18:49.483838] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-09-16T17:18:49.974921] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-04. [2025-09-16T17:18:50.762909] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-09-16T17:18:50.774844] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-04. [2025-09-16T17:18:51.116942] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-09-16T17:18:51.574909] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-07. [2025-09-16T17:18:52.270931] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T17:18:52.374909] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-05. [2025-09-16T17:18:52.540828] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T17:18:53.174933] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-09. [2025-09-16T17:18:53.913825] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T17:18:53.974919] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-05. [2025-09-16T17:18:54.097394] [INFO ] [REMOTE ] AVBC: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T17:18:54.099258] [INFO ] [REMOTE ] AVBC: Full fetch complete (3974 records). [2025-09-16T17:18:54.099326] [DEBUG] [VALIDATOR ] AVBC: Validating primary data. [2025-09-16T17:18:54.099699] [FATAL] [VALIDATOR ] AVBC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:18:54.099776] [FATAL] [MAIN ] AVBC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:18:54.099819] [WARN ] [MAIN ] AVBC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:18:54.099881] [INFO ] [LOCAL ] AVBC: Starting deletion of primary data. [2025-09-16T17:18:54.100017] [DEBUG] [LOCAL ] AVBC: No primary data file found to delete. [2025-09-16T17:18:54.100074] [INFO ] [LOCAL ] AVBC: Starting deletion of derivative data. [2025-09-16T17:18:54.100135] [DEBUG] [LOCAL ] AVBC: No derivative data file found to delete. [2025-09-16T17:18:54.100207] [INFO ] [MAIN ] Update Progress: 184/2369 symbols completed. AVBC took 36.8s. Estimated remaining: 3h 43m 51s. Rate: 9.8 symbols/min [2025-09-16T17:18:54.274610] [INFO ] [MAIN ] AVTR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:54.321964] [DEBUG] [LOCAL ] AVTR: Loaded 39554 existing records from primary storage. [2025-09-16T17:18:54.322134] [INFO ] [MAIN ] AVTR: Loaded 39554 existing records. [2025-09-16T17:18:54.497370] [WARN ] [MAIN ] AVTR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:18:54.497609] [INFO ] [MAIN ] AVTR: Wiping existing primary data before full refresh. [2025-09-16T17:18:54.497677] [INFO ] [LOCAL ] AVTR: Starting deletion of primary data. [2025-09-16T17:18:54.499299] [DEBUG] [LOCAL ] AVTR: Deleted primary data file. [2025-09-16T17:18:54.499352] [INFO ] [MAIN ] AVTR: Starting full primary refresh. [2025-09-16T17:18:54.782695] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-08. [2025-09-16T17:18:54.884931] [INFO ] [REMOTE ] AVTR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:54.940774] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T17:18:55.574918] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-06. [2025-09-16T17:18:55.665106] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-09-16T17:18:56.374910] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-09. [2025-09-16T17:18:56.820473] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T17:18:57.174905] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-02. [2025-09-16T17:18:57.534665] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-09-16T17:18:57.537327] [INFO ] [REMOTE ] AVK: Full fetch complete (19702 records). [2025-09-16T17:18:57.537407] [DEBUG] [VALIDATOR ] AVK: Validating primary data. [2025-09-16T17:18:57.897190] [DEBUG] [VALIDATOR ] AVK: All expected trading days are present. [2025-09-16T17:18:57.897281] [DEBUG] [VALIDATOR ] AVK: Primary data validation passed. [2025-09-16T17:18:57.897602] [INFO ] [MAIN ] AVK: Fetched primary data is valid. [2025-09-16T17:18:57.897662] [DEBUG] [LOCAL ] AVK: Saving primary data. [2025-09-16T17:18:57.898475] [DEBUG] [LOCAL ] AVK: Trimmed 19702 records to 13450 (keeping 150 days of data) [2025-09-16T17:18:57.975156] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-06. [2025-09-16T17:18:57.985403] [DEBUG] [LOCAL ] AVK: Primary data saved (13450 records). [2025-09-16T17:18:57.985496] [INFO ] [MAIN ] AVK: Full history saved. [2025-09-16T17:18:57.985572] [INFO ] [MAIN ] Update Progress: 185/2369 symbols completed. AVK took 23.3s. Estimated remaining: 3h 43m 18s. Rate: 9.8 symbols/min [2025-09-16T17:18:58.162464] [INFO ] [MAIN ] AVY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:18:58.162668] [DEBUG] [LOCAL ] AVY: No existing data file found in primary storage. [2025-09-16T17:18:58.162710] [WARN ] [MAIN ] AVY: No existing data found, performing full refresh instead. [2025-09-16T17:18:58.162838] [INFO ] [MAIN ] AVY: Wiping existing primary data before full refresh. [2025-09-16T17:18:58.162901] [INFO ] [LOCAL ] AVY: Starting deletion of primary data. [2025-09-16T17:18:58.162969] [DEBUG] [LOCAL ] AVY: No primary data file found to delete. [2025-09-16T17:18:58.163015] [INFO ] [MAIN ] AVY: Starting full primary refresh. [2025-09-16T17:18:58.514169] [INFO ] [REMOTE ] AVY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:18:58.776791] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-07. [2025-09-16T17:18:58.779609] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T17:18:59.398520] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T17:18:59.574913] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-07. [2025-09-16T17:18:59.756270] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-09-16T17:19:00.374901] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-02. [2025-09-16T17:19:01.174903] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-08. [2025-09-16T17:19:01.743737] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-09-16T17:19:01.974901] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-08. [2025-09-16T17:19:02.200737] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T17:19:02.386368] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-09-16T17:19:02.774949] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-03. [2025-09-16T17:19:03.252465] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-09-16T17:19:03.574949] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-03. [2025-09-16T17:19:04.374913] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-09. [2025-09-16T17:19:04.570467] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T17:19:05.174903] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-09. [2025-09-16T17:19:05.212363] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T17:19:05.326702] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T17:19:05.331219] [INFO ] [REMOTE ] AVNT: Full fetch complete (43059 records). [2025-09-16T17:19:05.331373] [DEBUG] [VALIDATOR ] AVNT: Validating primary data. [2025-09-16T17:19:05.700797] [DEBUG] [VALIDATOR ] AVNT: All expected trading days are present. [2025-09-16T17:19:05.700878] [DEBUG] [VALIDATOR ] AVNT: Primary data validation passed. [2025-09-16T17:19:05.701372] [INFO ] [MAIN ] AVNT: Fetched primary data is valid. [2025-09-16T17:19:05.701414] [DEBUG] [LOCAL ] AVNT: Saving primary data. [2025-09-16T17:19:05.702399] [DEBUG] [LOCAL ] AVNT: Trimmed 43059 records to 28832 (keeping 150 days of data) [2025-09-16T17:19:05.880827] [DEBUG] [LOCAL ] AVNT: Primary data saved (28832 records). [2025-09-16T17:19:05.880970] [INFO ] [MAIN ] AVNT: Full history saved. [2025-09-16T17:19:05.881054] [INFO ] [MAIN ] Update Progress: 186/2369 symbols completed. AVNT took 20.8s. Estimated remaining: 3h 43m 32s. Rate: 9.8 symbols/min [2025-09-16T17:19:05.980046] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-04. [2025-09-16T17:19:06.055733] [INFO ] [MAIN ] AWF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:19:06.055933] [DEBUG] [LOCAL ] AWF: No existing data file found in primary storage. [2025-09-16T17:19:06.055998] [WARN ] [MAIN ] AWF: No existing data found, performing full refresh instead. [2025-09-16T17:19:06.056123] [INFO ] [MAIN ] AWF: Wiping existing primary data before full refresh. [2025-09-16T17:19:06.056164] [INFO ] [LOCAL ] AWF: Starting deletion of primary data. [2025-09-16T17:19:06.056205] [DEBUG] [LOCAL ] AWF: No primary data file found to delete. [2025-09-16T17:19:06.056239] [INFO ] [MAIN ] AWF: Starting full primary refresh. [2025-09-16T17:19:06.426901] [INFO ] [REMOTE ] AWF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:19:06.576097] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:19:06.580357] [INFO ] [REMOTE ] AVNS: Full fetch complete (36344 records). [2025-09-16T17:19:06.580451] [DEBUG] [VALIDATOR ] AVNS: Validating primary data. [2025-09-16T17:19:06.784050] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-04. [2025-09-16T17:19:06.949406] [DEBUG] [VALIDATOR ] AVNS: All expected trading days are present. [2025-09-16T17:19:06.949494] [DEBUG] [VALIDATOR ] AVNS: Primary data validation passed. [2025-09-16T17:19:06.949982] [INFO ] [MAIN ] AVNS: Fetched primary data is valid. [2025-09-16T17:19:06.950027] [DEBUG] [LOCAL ] AVNS: Saving primary data. [2025-09-16T17:19:06.950915] [DEBUG] [LOCAL ] AVNS: Trimmed 36344 records to 25002 (keeping 150 days of data) [2025-09-16T17:19:07.102815] [DEBUG] [LOCAL ] AVNS: Primary data saved (25002 records). [2025-09-16T17:19:07.102953] [INFO ] [MAIN ] AVNS: Full history saved. [2025-09-16T17:19:07.103036] [INFO ] [MAIN ] Update Progress: 187/2369 symbols completed. AVNS took 26.5s. Estimated remaining: 3h 42m 29s. Rate: 9.8 symbols/min [2025-09-16T17:19:07.273265] [INFO ] [MAIN ] AWI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:19:07.292422] [DEBUG] [LOCAL ] AWI: Loaded 18971 existing records from primary storage. [2025-09-16T17:19:07.292537] [INFO ] [MAIN ] AWI: Loaded 18971 existing records. [2025-09-16T17:19:07.469561] [WARN ] [MAIN ] AWI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:19:07.469825] [INFO ] [MAIN ] AWI: Wiping existing primary data before full refresh. [2025-09-16T17:19:07.469871] [INFO ] [LOCAL ] AWI: Starting deletion of primary data. [2025-09-16T17:19:07.470814] [DEBUG] [LOCAL ] AWI: Deleted primary data file. [2025-09-16T17:19:07.470864] [INFO ] [MAIN ] AWI: Starting full primary refresh. [2025-09-16T17:19:07.575399] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-02. [2025-09-16T17:19:07.842782] [INFO ] [REMOTE ] AWI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:19:07.870132] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-09-16T17:19:08.345255] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T17:19:08.374896] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-02. [2025-09-16T17:19:08.656516] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-09-16T17:19:09.174959] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-03. [2025-09-16T17:19:09.318156] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-09-16T17:19:09.974874] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-05. [2025-09-16T17:19:10.200220] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-09-16T17:19:10.774924] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-03. [2025-09-16T17:19:11.455123] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T17:19:11.574898] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-06. [2025-09-16T17:19:11.713703] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T17:19:12.374908] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-04. [2025-09-16T17:19:12.628820] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T17:19:13.175787] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-09-16T17:19:13.174871] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-04. [2025-09-16T17:19:13.974938] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-05. [2025-09-16T17:19:14.487900] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T17:19:14.774867] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-07. [2025-09-16T17:19:14.881988] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T17:19:15.574908] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-06. [2025-09-16T17:19:15.896098] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T17:19:16.226787] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-09-16T17:19:16.374869] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-07. [2025-09-16T17:19:17.174897] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-08. [2025-09-16T17:19:17.496979] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T17:19:17.974942] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-05. [2025-09-16T17:19:18.518710] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T17:19:18.774892] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-08. [2025-09-16T17:19:18.791178] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T17:19:19.574899] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-05. [2025-09-16T17:19:19.846294] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T17:19:20.374898] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-09. [2025-09-16T17:19:21.174862] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-06. [2025-09-16T17:19:21.335838] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T17:19:21.339096] [INFO ] [REMOTE ] AWF: Full fetch complete (26544 records). [2025-09-16T17:19:21.339167] [DEBUG] [VALIDATOR ] AWF: Validating primary data. [2025-09-16T17:19:21.722036] [DEBUG] [VALIDATOR ] AWF: All expected trading days are present. [2025-09-16T17:19:21.722193] [DEBUG] [VALIDATOR ] AWF: Primary data validation passed. [2025-09-16T17:19:21.722590] [INFO ] [MAIN ] AWF: Fetched primary data is valid. [2025-09-16T17:19:21.722658] [DEBUG] [LOCAL ] AWF: Saving primary data. [2025-09-16T17:19:21.723606] [DEBUG] [LOCAL ] AWF: Trimmed 26544 records to 17033 (keeping 150 days of data) [2025-09-16T17:19:21.830477] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-05 (2.26 seconds). [2025-09-16T17:19:21.869028] [DEBUG] [LOCAL ] AWF: Primary data saved (17033 records). [2025-09-16T17:19:21.869155] [INFO ] [MAIN ] AWF: Full history saved. [2025-09-16T17:19:21.869235] [INFO ] [MAIN ] Update Progress: 188/2369 symbols completed. AWF took 16.0s. Estimated remaining: 3h 44m 3s. Rate: 9.7 symbols/min [2025-09-16T17:19:21.976286] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-06. [2025-09-16T17:19:22.041771] [INFO ] [MAIN ] AWK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:19:22.057853] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-09-16T17:19:22.096042] [DEBUG] [LOCAL ] AWK: Loaded 35633 existing records from primary storage. [2025-09-16T17:19:22.096175] [INFO ] [MAIN ] AWK: Loaded 35633 existing records. [2025-09-16T17:19:22.278766] [WARN ] [MAIN ] AWK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:19:22.279120] [INFO ] [MAIN ] AWK: Wiping existing primary data before full refresh. [2025-09-16T17:19:22.279174] [INFO ] [LOCAL ] AWK: Starting deletion of primary data. [2025-09-16T17:19:22.280559] [DEBUG] [LOCAL ] AWK: Deleted primary data file. [2025-09-16T17:19:22.280611] [INFO ] [MAIN ] AWK: Starting full primary refresh. [2025-09-16T17:19:22.631632] [INFO ] [REMOTE ] AWK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:19:22.774896] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-07. [2025-09-16T17:19:23.574879] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-09. [2025-09-16T17:19:23.665265] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T17:19:23.715013] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-09-16T17:19:24.374928] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-07. [2025-09-16T17:19:24.833206] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T17:19:24.837819] [INFO ] [REMOTE ] AVY: Full fetch complete (41228 records). [2025-09-16T17:19:24.837907] [DEBUG] [VALIDATOR ] AVY: Validating primary data. [2025-09-16T17:19:25.175556] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-02. [2025-09-16T17:19:25.206814] [DEBUG] [VALIDATOR ] AVY: All expected trading days are present. [2025-09-16T17:19:25.206889] [DEBUG] [VALIDATOR ] AVY: Primary data validation passed. [2025-09-16T17:19:25.207367] [INFO ] [MAIN ] AVY: Fetched primary data is valid. [2025-09-16T17:19:25.207410] [DEBUG] [LOCAL ] AVY: Saving primary data. [2025-09-16T17:19:25.208722] [DEBUG] [LOCAL ] AVY: Trimmed 41228 records to 27220 (keeping 150 days of data) [2025-09-16T17:19:25.387780] [DEBUG] [LOCAL ] AVY: Primary data saved (27220 records). [2025-09-16T17:19:25.388041] [INFO ] [MAIN ] AVY: Full history saved. [2025-09-16T17:19:25.388119] [INFO ] [MAIN ] Update Progress: 189/2369 symbols completed. AVY took 27.4s. Estimated remaining: 3h 43m 26s. Rate: 9.8 symbols/min [2025-09-16T17:19:25.556499] [INFO ] [MAIN ] AWP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:19:25.572818] [DEBUG] [LOCAL ] AWP: Loaded 16615 existing records from primary storage. [2025-09-16T17:19:25.572927] [INFO ] [MAIN ] AWP: Loaded 16615 existing records. [2025-09-16T17:19:25.752051] [WARN ] [MAIN ] AWP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:19:25.752372] [INFO ] [MAIN ] AWP: Wiping existing primary data before full refresh. [2025-09-16T17:19:25.752426] [INFO ] [LOCAL ] AWP: Starting deletion of primary data. [2025-09-16T17:19:25.753607] [DEBUG] [LOCAL ] AWP: Deleted primary data file. [2025-09-16T17:19:25.753699] [INFO ] [MAIN ] AWP: Starting full primary refresh. [2025-09-16T17:19:25.979827] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-08. [2025-09-16T17:19:26.121409] [INFO ] [REMOTE ] AWP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:19:26.177700] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-09-16T17:19:26.777785] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-08. [2025-09-16T17:19:26.794541] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T17:19:27.250706] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-09-16T17:19:27.574946] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-03. [2025-09-16T17:19:28.374882] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-09. [2025-09-16T17:19:28.458032] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T17:19:29.039581] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T17:19:29.174925] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-04. [2025-09-16T17:19:29.397360] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T17:19:29.400622] [INFO ] [REMOTE ] AWI: Full fetch complete (28462 records). [2025-09-16T17:19:29.400715] [DEBUG] [VALIDATOR ] AWI: Validating primary data. [2025-09-16T17:19:29.767956] [DEBUG] [VALIDATOR ] AWI: All expected trading days are present. [2025-09-16T17:19:29.768032] [DEBUG] [VALIDATOR ] AWI: Primary data validation passed. [2025-09-16T17:19:29.768409] [INFO ] [MAIN ] AWI: Fetched primary data is valid. [2025-09-16T17:19:29.768449] [DEBUG] [LOCAL ] AWI: Saving primary data. [2025-09-16T17:19:29.769144] [DEBUG] [LOCAL ] AWI: Trimmed 28462 records to 19097 (keeping 150 days of data) [2025-09-16T17:19:29.893531] [DEBUG] [LOCAL ] AWI: Primary data saved (19097 records). [2025-09-16T17:19:29.893732] [INFO ] [MAIN ] AWI: Full history saved. [2025-09-16T17:19:29.893845] [INFO ] [MAIN ] Update Progress: 190/2369 symbols completed. AWI took 22.8s. Estimated remaining: 3h 43m 1s. Rate: 9.8 symbols/min [2025-09-16T17:19:29.980081] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-02. [2025-09-16T17:19:30.072581] [INFO ] [MAIN ] AWR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:19:30.090081] [DEBUG] [LOCAL ] AWR: Loaded 16093 existing records from primary storage. [2025-09-16T17:19:30.090744] [INFO ] [MAIN ] AWR: Loaded 16093 existing records. [2025-09-16T17:19:30.275157] [WARN ] [MAIN ] AWR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:19:30.275449] [INFO ] [MAIN ] AWR: Wiping existing primary data before full refresh. [2025-09-16T17:19:30.275506] [INFO ] [LOCAL ] AWR: Starting deletion of primary data. [2025-09-16T17:19:30.276500] [DEBUG] [LOCAL ] AWR: Deleted primary data file. [2025-09-16T17:19:30.276566] [INFO ] [MAIN ] AWR: Starting full primary refresh. [2025-09-16T17:19:30.777384] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-09. [2025-09-16T17:19:30.840214] [INFO ] [REMOTE ] AWR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:19:30.845703] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T17:19:30.917074] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-09-16T17:19:31.574912] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-03. [2025-09-16T17:19:32.362949] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T17:19:32.369572] [INFO ] [REMOTE ] AVTR: Full fetch complete (58235 records). [2025-09-16T17:19:32.369716] [DEBUG] [VALIDATOR ] AVTR: Validating primary data. [2025-09-16T17:19:32.382066] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-05. [2025-09-16T17:19:32.768079] [DEBUG] [VALIDATOR ] AVTR: All expected trading days are present. [2025-09-16T17:19:32.768208] [DEBUG] [VALIDATOR ] AVTR: Primary data validation passed. [2025-09-16T17:19:32.768993] [INFO ] [MAIN ] AVTR: Fetched primary data is valid. [2025-09-16T17:19:32.769065] [DEBUG] [LOCAL ] AVTR: Saving primary data. [2025-09-16T17:19:32.770657] [DEBUG] [LOCAL ] AVTR: Trimmed 58235 records to 39944 (keeping 150 days of data) [2025-09-16T17:19:32.886446] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T17:19:33.040414] [DEBUG] [LOCAL ] AVTR: Primary data saved (39944 records). [2025-09-16T17:19:33.040578] [INFO ] [MAIN ] AVTR: Full history saved. [2025-09-16T17:19:33.042847] [INFO ] [MAIN ] Update Progress: 191/2369 symbols completed. AVTR took 38.9s. Estimated remaining: 3h 42m 21s. Rate: 9.8 symbols/min [2025-09-16T17:19:33.185444] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-04. [2025-09-16T17:19:33.217392] [INFO ] [MAIN ] AX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:19:33.238562] [DEBUG] [LOCAL ] AX: Loaded 20149 existing records from primary storage. [2025-09-16T17:19:33.238722] [INFO ] [MAIN ] AX: Loaded 20149 existing records. [2025-09-16T17:19:33.452299] [WARN ] [MAIN ] AX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:19:33.452561] [INFO ] [MAIN ] AX: Wiping existing primary data before full refresh. [2025-09-16T17:19:33.452611] [INFO ] [LOCAL ] AX: Starting deletion of primary data. [2025-09-16T17:19:33.453369] [DEBUG] [LOCAL ] AX: Deleted primary data file. [2025-09-16T17:19:33.453420] [INFO ] [MAIN ] AX: Starting full primary refresh. [2025-09-16T17:19:33.817506] [INFO ] [REMOTE ] AX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:19:33.929754] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-09-16T17:19:33.974902] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-05. [2025-09-16T17:19:34.178502] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-09-16T17:19:34.774973] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-02. [2025-09-16T17:19:34.824835] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-09-16T17:19:35.574905] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-06. [2025-09-16T17:19:36.124564] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-09-16T17:19:36.318160] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-09-16T17:19:36.374912] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-02. [2025-09-16T17:19:37.174880] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-06. [2025-09-16T17:19:37.533914] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-09-16T17:19:37.974921] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-07. [2025-09-16T17:19:38.774926] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-03. [2025-09-16T17:19:38.919376] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T17:19:39.351223] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-09-16T17:19:39.520691] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-09-16T17:19:39.574869] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-04. [2025-09-16T17:19:40.374911] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-03. [2025-09-16T17:19:40.620987] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-09-16T17:19:41.174936] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-07. [2025-09-16T17:19:41.438190] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-09-16T17:19:41.974901] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-04. [2025-09-16T17:19:42.685060] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T17:19:42.774866] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-08. [2025-09-16T17:19:43.370117] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T17:19:43.574905] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-08. [2025-09-16T17:19:44.374865] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-05. [2025-09-16T17:19:44.483813] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-09-16T17:19:44.712933] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T17:19:45.142352] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-09-16T17:19:45.174888] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-06. [2025-09-16T17:19:45.974882] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-09. [2025-09-16T17:19:46.206919] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-09-16T17:19:46.774896] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-05. [2025-09-16T17:19:47.341452] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T17:19:47.347135] [INFO ] [REMOTE ] AWK: Full fetch complete (53205 records). [2025-09-16T17:19:47.347249] [DEBUG] [VALIDATOR ] AWK: Validating primary data. [2025-09-16T17:19:47.582777] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-09. [2025-09-16T17:19:47.735629] [DEBUG] [VALIDATOR ] AWK: All expected trading days are present. [2025-09-16T17:19:47.735730] [DEBUG] [VALIDATOR ] AWK: Primary data validation passed. [2025-09-16T17:19:47.736416] [INFO ] [MAIN ] AWK: Fetched primary data is valid. [2025-09-16T17:19:47.736461] [DEBUG] [LOCAL ] AWK: Saving primary data. [2025-09-16T17:19:47.738302] [DEBUG] [LOCAL ] AWK: Trimmed 53205 records to 35976 (keeping 150 days of data) [2025-09-16T17:19:47.870167] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-09-16T17:19:47.989563] [DEBUG] [LOCAL ] AWK: Primary data saved (35976 records). [2025-09-16T17:19:47.989726] [INFO ] [MAIN ] AWK: Full history saved. [2025-09-16T17:19:47.989826] [INFO ] [MAIN ] Update Progress: 192/2369 symbols completed. AWK took 26.1s. Estimated remaining: 3h 43m 55s. Rate: 9.7 symbols/min [2025-09-16T17:19:48.161264] [INFO ] [MAIN ] AXL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:19:48.195159] [DEBUG] [LOCAL ] AXL: Loaded 37150 existing records from primary storage. [2025-09-16T17:19:48.195334] [INFO ] [MAIN ] AXL: Loaded 37150 existing records. [2025-09-16T17:19:48.370831] [WARN ] [MAIN ] AXL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:19:48.371069] [INFO ] [MAIN ] AXL: Wiping existing primary data before full refresh. [2025-09-16T17:19:48.371114] [INFO ] [LOCAL ] AXL: Starting deletion of primary data. [2025-09-16T17:19:48.372289] [DEBUG] [LOCAL ] AXL: Deleted primary data file. [2025-09-16T17:19:48.372344] [INFO ] [MAIN ] AXL: Starting full primary refresh. [2025-09-16T17:19:48.383564] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-06. [2025-09-16T17:19:48.726118] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T17:19:48.730126] [INFO ] [REMOTE ] AXL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:19:48.733381] [INFO ] [REMOTE ] AWP: Full fetch complete (23923 records). [2025-09-16T17:19:48.733472] [DEBUG] [VALIDATOR ] AWP: Validating primary data. [2025-09-16T17:19:49.111965] [DEBUG] [VALIDATOR ] AWP: All expected trading days are present. [2025-09-16T17:19:49.112059] [DEBUG] [VALIDATOR ] AWP: Primary data validation passed. [2025-09-16T17:19:49.112433] [INFO ] [MAIN ] AWP: Fetched primary data is valid. [2025-09-16T17:19:49.112476] [DEBUG] [LOCAL ] AWP: Saving primary data. [2025-09-16T17:19:49.113135] [DEBUG] [LOCAL ] AWP: Trimmed 23923 records to 16751 (keeping 150 days of data) [2025-09-16T17:19:49.183396] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-02. [2025-09-16T17:19:49.212346] [DEBUG] [LOCAL ] AWP: Primary data saved (16751 records). [2025-09-16T17:19:49.212481] [INFO ] [MAIN ] AWP: Full history saved. [2025-09-16T17:19:49.212580] [INFO ] [MAIN ] Update Progress: 193/2369 symbols completed. AWP took 23.8s. Estimated remaining: 3h 42m 53s. Rate: 9.8 symbols/min [2025-09-16T17:19:49.400345] [INFO ] [MAIN ] AXP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:19:49.400782] [DEBUG] [LOCAL ] AXP: No existing data file found in primary storage. [2025-09-16T17:19:49.400912] [WARN ] [MAIN ] AXP: No existing data found, performing full refresh instead. [2025-09-16T17:19:49.401163] [INFO ] [MAIN ] AXP: Wiping existing primary data before full refresh. [2025-09-16T17:19:49.401236] [INFO ] [LOCAL ] AXP: Starting deletion of primary data. [2025-09-16T17:19:49.401420] [DEBUG] [LOCAL ] AXP: No primary data file found to delete. [2025-09-16T17:19:49.401552] [INFO ] [MAIN ] AXP: Starting full primary refresh. [2025-09-16T17:19:49.678872] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:19:49.781093] [INFO ] [REMOTE ] AXP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:19:49.974898] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-02. [2025-09-16T17:19:50.774909] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-07. [2025-09-16T17:19:51.064577] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-09-16T17:19:51.574851] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-03. [2025-09-16T17:19:51.646112] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T17:19:51.721114] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T17:19:52.374905] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-03. [2025-09-16T17:19:53.174875] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-07. [2025-09-16T17:19:53.223081] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-09-16T17:19:53.974904] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-08. [2025-09-16T17:19:54.341097] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-09-16T17:19:54.495165] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-09-16T17:19:54.774950] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-04. [2025-09-16T17:19:54.813812] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-09-16T17:19:55.574898] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-09. [2025-09-16T17:19:56.374875] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-04. [2025-09-16T17:19:56.690293] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-09-16T17:19:56.726238] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T17:19:56.729934] [INFO ] [REMOTE ] AX: Full fetch complete (31165 records). [2025-09-16T17:19:56.730014] [DEBUG] [VALIDATOR ] AX: Validating primary data. [2025-09-16T17:19:57.106326] [DEBUG] [VALIDATOR ] AX: All expected trading days are present. [2025-09-16T17:19:57.106410] [DEBUG] [VALIDATOR ] AX: Primary data validation passed. [2025-09-16T17:19:57.106868] [INFO ] [MAIN ] AX: Fetched primary data is valid. [2025-09-16T17:19:57.106917] [DEBUG] [LOCAL ] AX: Saving primary data. [2025-09-16T17:19:57.107744] [DEBUG] [LOCAL ] AX: Trimmed 31165 records to 20338 (keeping 150 days of data) [2025-09-16T17:19:57.194513] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-08. [2025-09-16T17:19:57.229015] [DEBUG] [LOCAL ] AX: Primary data saved (20338 records). [2025-09-16T17:19:57.229121] [INFO ] [MAIN ] AX: Full history saved. [2025-09-16T17:19:57.229204] [INFO ] [MAIN ] Update Progress: 194/2369 symbols completed. AX took 24.2s. Estimated remaining: 3h 43m 8s. Rate: 9.7 symbols/min [2025-09-16T17:19:57.400897] [INFO ] [MAIN ] AXR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:19:57.404636] [DEBUG] [LOCAL ] AXR: Loaded 2129 existing records from primary storage. [2025-09-16T17:19:57.404733] [INFO ] [MAIN ] AXR: Loaded 2129 existing records. [2025-09-16T17:19:57.586320] [WARN ] [MAIN ] AXR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:19:57.586596] [INFO ] [MAIN ] AXR: Wiping existing primary data before full refresh. [2025-09-16T17:19:57.586662] [INFO ] [LOCAL ] AXR: Starting deletion of primary data. [2025-09-16T17:19:57.586949] [DEBUG] [LOCAL ] AXR: Deleted primary data file. [2025-09-16T17:19:57.586995] [INFO ] [MAIN ] AXR: Starting full primary refresh. [2025-09-16T17:19:57.943731] [INFO ] [REMOTE ] AXR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:19:57.974847] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-02. [2025-09-16T17:19:58.081890] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-09-16T17:19:58.226141] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T17:19:58.586302] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-09-16T17:19:58.774915] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-03. [2025-09-16T17:19:59.166856] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-09-16T17:19:59.574876] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-09. [2025-09-16T17:20:00.374906] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-05. [2025-09-16T17:20:01.014731] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T17:20:01.019435] [INFO ] [REMOTE ] AWR: Full fetch complete (24272 records). [2025-09-16T17:20:01.019560] [DEBUG] [VALIDATOR ] AWR: Validating primary data. [2025-09-16T17:20:01.179998] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-04. [2025-09-16T17:20:01.408077] [DEBUG] [VALIDATOR ] AWR: All expected trading days are present. [2025-09-16T17:20:01.408166] [DEBUG] [VALIDATOR ] AWR: Primary data validation passed. [2025-09-16T17:20:01.408541] [INFO ] [MAIN ] AWR: Fetched primary data is valid. [2025-09-16T17:20:01.408586] [DEBUG] [LOCAL ] AWR: Saving primary data. [2025-09-16T17:20:01.409518] [DEBUG] [LOCAL ] AWR: Trimmed 24272 records to 16266 (keeping 150 days of data) [2025-09-16T17:20:01.513963] [DEBUG] [LOCAL ] AWR: Primary data saved (16266 records). [2025-09-16T17:20:01.514118] [INFO ] [MAIN ] AWR: Full history saved. [2025-09-16T17:20:01.514206] [INFO ] [MAIN ] Update Progress: 195/2369 symbols completed. AWR took 31.6s. Estimated remaining: 3h 42m 41s. Rate: 9.8 symbols/min [2025-09-16T17:20:01.692851] [INFO ] [MAIN ] AXS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:01.724597] [DEBUG] [LOCAL ] AXS: Loaded 28095 existing records from primary storage. [2025-09-16T17:20:01.724792] [INFO ] [MAIN ] AXS: Loaded 28095 existing records. [2025-09-16T17:20:01.869954] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-09-16T17:20:01.967463] [WARN ] [MAIN ] AXS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:20:01.967744] [INFO ] [MAIN ] AXS: Wiping existing primary data before full refresh. [2025-09-16T17:20:01.967802] [INFO ] [LOCAL ] AXS: Starting deletion of primary data. [2025-09-16T17:20:01.968737] [DEBUG] [LOCAL ] AXS: Deleted primary data file. [2025-09-16T17:20:01.968800] [INFO ] [MAIN ] AXS: Starting full primary refresh. [2025-09-16T17:20:01.981196] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-05. [2025-09-16T17:20:02.374879] [INFO ] [REMOTE ] AXS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:02.437243] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-05 (2.06 seconds). [2025-09-16T17:20:02.774892] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-02. [2025-09-16T17:20:03.574905] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-05. [2025-09-16T17:20:04.118933] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-09-16T17:20:04.349340] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T17:20:04.374853] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-03. [2025-09-16T17:20:04.486673] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-05 (2.51 seconds). [2025-09-16T17:20:05.174925] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-06. [2025-09-16T17:20:05.606860] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-09-16T17:20:05.974923] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-04. [2025-09-16T17:20:06.754109] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T17:20:06.774869] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-07. [2025-09-16T17:20:07.471992] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T17:20:07.574940] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-06. [2025-09-16T17:20:07.977272] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-09-16T17:20:08.374914] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-06. [2025-09-16T17:20:08.560342] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-09-16T17:20:09.174919] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-07. [2025-09-16T17:20:09.986587] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-08. [2025-09-16T17:20:09.991702] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T17:20:10.774944] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-07. [2025-09-16T17:20:11.574958] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-05. [2025-09-16T17:20:11.635853] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-07 (2.46 seconds). [2025-09-16T17:20:11.970138] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-09-16T17:20:12.228789] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T17:20:12.374923] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-08. [2025-09-16T17:20:13.174909] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-09. [2025-09-16T17:20:13.205711] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-09-16T17:20:13.754603] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T17:20:13.974895] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-08. [2025-09-16T17:20:14.411536] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-09-16T17:20:14.774901] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-06. [2025-09-16T17:20:15.438569] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-09 (2.26 seconds). [2025-09-16T17:20:15.445139] [INFO ] [REMOTE ] AXP: Full fetch complete (57857 records). [2025-09-16T17:20:15.445290] [DEBUG] [VALIDATOR ] AXP: Validating primary data. [2025-09-16T17:20:15.580004] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-09. [2025-09-16T17:20:15.853393] [DEBUG] [VALIDATOR ] AXP: All expected trading days are present. [2025-09-16T17:20:15.853476] [DEBUG] [VALIDATOR ] AXP: Primary data validation passed. [2025-09-16T17:20:15.854101] [INFO ] [MAIN ] AXP: Fetched primary data is valid. [2025-09-16T17:20:15.854150] [DEBUG] [LOCAL ] AXP: Saving primary data. [2025-09-16T17:20:15.855362] [DEBUG] [LOCAL ] AXP: Trimmed 57857 records to 39661 (keeping 150 days of data) [2025-09-16T17:20:16.107761] [DEBUG] [LOCAL ] AXP: Primary data saved (39661 records). [2025-09-16T17:20:16.107993] [INFO ] [MAIN ] AXP: Full history saved. [2025-09-16T17:20:16.108080] [INFO ] [MAIN ] Update Progress: 196/2369 symbols completed. AXP took 26.9s. Estimated remaining: 3h 44m 8s. Rate: 9.7 symbols/min [2025-09-16T17:20:16.254888] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T17:20:16.321141] [INFO ] [MAIN ] AXTA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:16.364065] [DEBUG] [LOCAL ] AXTA: Loaded 38451 existing records from primary storage. [2025-09-16T17:20:16.364248] [INFO ] [MAIN ] AXTA: Loaded 38451 existing records. [2025-09-16T17:20:16.380025] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-07. [2025-09-16T17:20:16.555024] [WARN ] [MAIN ] AXTA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:20:16.555268] [INFO ] [MAIN ] AXTA: Wiping existing primary data before full refresh. [2025-09-16T17:20:16.555331] [INFO ] [LOCAL ] AXTA: Starting deletion of primary data. [2025-09-16T17:20:16.556363] [DEBUG] [LOCAL ] AXTA: Deleted primary data file. [2025-09-16T17:20:16.556416] [INFO ] [MAIN ] AXTA: Starting full primary refresh. [2025-09-16T17:20:16.928584] [INFO ] [REMOTE ] AXTA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:17.174932] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-09. [2025-09-16T17:20:17.658922] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-09-16T17:20:17.717674] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-09-16T17:20:17.718869] [INFO ] [REMOTE ] AXR: Full fetch complete (3351 records). [2025-09-16T17:20:17.718937] [DEBUG] [VALIDATOR ] AXR: Validating primary data. [2025-09-16T17:20:17.980050] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-02. [2025-09-16T17:20:18.082656] [DEBUG] [VALIDATOR ] AXR: All expected trading days are present. [2025-09-16T17:20:18.082742] [DEBUG] [VALIDATOR ] AXR: Primary data validation passed. [2025-09-16T17:20:18.082931] [INFO ] [MAIN ] AXR: Fetched primary data is valid. [2025-09-16T17:20:18.082989] [DEBUG] [LOCAL ] AXR: Saving primary data. [2025-09-16T17:20:18.083360] [DEBUG] [LOCAL ] AXR: Trimmed 3351 records to 2150 (keeping 150 days of data) [2025-09-16T17:20:18.096297] [DEBUG] [LOCAL ] AXR: Primary data saved (2150 records). [2025-09-16T17:20:18.096368] [INFO ] [MAIN ] AXR: Full history saved. [2025-09-16T17:20:18.096470] [INFO ] [MAIN ] Update Progress: 197/2369 symbols completed. AXR took 20.9s. Estimated remaining: 3h 43m 16s. Rate: 9.7 symbols/min [2025-09-16T17:20:18.774934] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-08. [2025-09-16T17:20:18.812546] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-09 (3.23 seconds). [2025-09-16T17:20:18.819223] [INFO ] [REMOTE ] AXL: Full fetch complete (55048 records). [2025-09-16T17:20:18.819343] [DEBUG] [VALIDATOR ] AXL: Validating primary data. [2025-09-16T17:20:19.208372] [DEBUG] [VALIDATOR ] AXL: All expected trading days are present. [2025-09-16T17:20:19.208472] [DEBUG] [VALIDATOR ] AXL: Primary data validation passed. [2025-09-16T17:20:19.209227] [INFO ] [MAIN ] AXL: Fetched primary data is valid. [2025-09-16T17:20:19.209288] [DEBUG] [LOCAL ] AXL: Saving primary data. [2025-09-16T17:20:19.211036] [DEBUG] [LOCAL ] AXL: Trimmed 55048 records to 37509 (keeping 150 days of data) [2025-09-16T17:20:19.449553] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T17:20:19.466188] [DEBUG] [LOCAL ] AXL: Primary data saved (37509 records). [2025-09-16T17:20:19.466288] [INFO ] [MAIN ] AXL: Full history saved. [2025-09-16T17:20:19.466398] [INFO ] [MAIN ] Update Progress: 198/2369 symbols completed. AXL took 31.5s. Estimated remaining: 3h 42m 17s. Rate: 9.8 symbols/min [2025-09-16T17:20:19.574877] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-03. [2025-09-16T17:20:19.939467] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T17:20:20.374933] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-09. [2025-09-16T17:20:20.874334] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T17:20:21.174910] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-04. [2025-09-16T17:20:21.429722] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T17:20:21.433999] [INFO ] [REMOTE ] AXS: Full fetch complete (41191 records). [2025-09-16T17:20:21.434088] [DEBUG] [VALIDATOR ] AXS: Validating primary data. [2025-09-16T17:20:21.434687] [FATAL] [VALIDATOR ] AXS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:20:21.434761] [FATAL] [MAIN ] AXS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:20:21.434808] [WARN ] [MAIN ] AXS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:20:21.434856] [INFO ] [LOCAL ] AXS: Starting deletion of primary data. [2025-09-16T17:20:21.434914] [DEBUG] [LOCAL ] AXS: No primary data file found to delete. [2025-09-16T17:20:21.434971] [INFO ] [LOCAL ] AXS: Starting deletion of derivative data. [2025-09-16T17:20:21.435281] [DEBUG] [LOCAL ] AXS: Deleted derivative data file. [2025-09-16T17:20:21.436492] [INFO ] [MAIN ] Update Progress: 199/2369 symbols completed. AXS took 19.9s. Estimated remaining: 3h 41m 26s. Rate: 9.8 symbols/min [2025-09-16T17:20:22.490816] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T17:20:22.501749] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-05. [2025-09-16T17:20:25.568239] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-05 (3.07 seconds). [2025-09-16T17:20:25.579774] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-06. [2025-09-16T17:20:26.820176] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T17:20:26.830745] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-07. [2025-09-16T17:20:26.938093] [DEBUG] [REMOTE ] AXTA: Error fetching primary data for 2025-07: HTTPSConnectionPool(host='www.alphavantage.co', port=443): Max retries exceeded with url: /query?function=TIME_SERIES_INTRADAY&symbol=AXTA&interval=1min&month=2025-07&outputsize=full&apikey=2N37ABM4OC9PIBKK&extended_hours=false&entitlement=delayed (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1032)'))) [2025-09-16T17:20:26.938310] [WARN ] [REMOTE ] AXTA: No data returned for month 2025-07 [2025-09-16T17:20:26.938786] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-08. [2025-09-16T17:20:28.190777] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-09-16T17:20:28.201923] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-09. [2025-09-16T17:20:29.626924] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T17:20:29.633159] [INFO ] [REMOTE ] AXTA: Full fetch complete (48365 records). [2025-09-16T17:20:29.633284] [DEBUG] [VALIDATOR ] AXTA: Validating primary data. [2025-09-16T17:20:29.824937] [WARN ] [VALIDATOR ] AXTA: Validation failed - insufficient historical data. Have 128 unique days, need 150 days [2025-09-16T17:20:29.825463] [FATAL] [MAIN ] AXTA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:20:29.825514] [WARN ] [MAIN ] AXTA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:20:29.825560] [INFO ] [LOCAL ] AXTA: Starting deletion of primary data. [2025-09-16T17:20:29.825631] [DEBUG] [LOCAL ] AXTA: No primary data file found to delete. [2025-09-16T17:20:29.825692] [INFO ] [LOCAL ] AXTA: Starting deletion of derivative data. [2025-09-16T17:20:29.825921] [DEBUG] [LOCAL ] AXTA: Deleted derivative data file. [2025-09-16T17:20:29.825999] [INFO ] [MAIN ] Update Progress: 200/2369 symbols completed. AXTA took 13.7s. Estimated remaining: 3h 41m 44s. Rate: 9.8 symbols/min [2025-09-16T17:20:30.429589] [INFO ] [MAIN ] AZZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:30.499085] [INFO ] [MAIN ] AYI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:30.554777] [INFO ] [MAIN ] B: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:30.576999] [DEBUG] [LOCAL ] B: No existing data file found in primary storage. [2025-09-16T17:20:30.579008] [WARN ] [MAIN ] B: No existing data found, performing full refresh instead. [2025-09-16T17:20:30.578950] [INFO ] [MAIN ] AZO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:30.579199] [INFO ] [MAIN ] B: Wiping existing primary data before full refresh. [2025-09-16T17:20:30.579604] [INFO ] [LOCAL ] B: Starting deletion of primary data. [2025-09-16T17:20:30.579671] [DEBUG] [LOCAL ] B: No primary data file found to delete. [2025-09-16T17:20:30.579708] [INFO ] [MAIN ] B: Starting full primary refresh. [2025-09-16T17:20:30.724497] [DEBUG] [LOCAL ] AYI: Loaded 17494 existing records from primary storage. [2025-09-16T17:20:30.724996] [INFO ] [MAIN ] AYI: Loaded 17494 existing records. [2025-09-16T17:20:30.725227] [DEBUG] [LOCAL ] AZO: Loaded 7801 existing records from primary storage. [2025-09-16T17:20:30.724856] [DEBUG] [LOCAL ] AZZ: Loaded 16542 existing records from primary storage. [2025-09-16T17:20:30.789671] [INFO ] [MAIN ] AZZ: Loaded 16542 existing records. [2025-09-16T17:20:30.789596] [INFO ] [MAIN ] AZO: Loaded 7801 existing records. [2025-09-16T17:20:31.229787] [WARN ] [MAIN ] AYI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:20:31.280701] [INFO ] [MAIN ] AYI: Wiping existing primary data before full refresh. [2025-09-16T17:20:31.286089] [INFO ] [LOCAL ] AYI: Starting deletion of primary data. [2025-09-16T17:20:31.297029] [DEBUG] [LOCAL ] AYI: Deleted primary data file. [2025-09-16T17:20:31.377137] [INFO ] [MAIN ] AYI: Starting full primary refresh. [2025-09-16T17:20:31.515731] [WARN ] [MAIN ] AZZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:20:31.500749] [INFO ] [REMOTE ] B: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:31.520049] [WARN ] [MAIN ] AZO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:20:31.526211] [INFO ] [MAIN ] AZZ: Wiping existing primary data before full refresh. [2025-09-16T17:20:31.526129] [INFO ] [REMOTE ] B: Fetching primary data for 2025-02. [2025-09-16T17:20:31.541911] [INFO ] [LOCAL ] AZZ: Starting deletion of primary data. [2025-09-16T17:20:31.532233] [INFO ] [MAIN ] AZO: Wiping existing primary data before full refresh. [2025-09-16T17:20:31.566919] [INFO ] [LOCAL ] AZO: Starting deletion of primary data. [2025-09-16T17:20:31.576014] [DEBUG] [LOCAL ] AZO: Deleted primary data file. [2025-09-16T17:20:31.575963] [DEBUG] [LOCAL ] AZZ: Deleted primary data file. [2025-09-16T17:20:31.595418] [INFO ] [MAIN ] AZO: Starting full primary refresh. [2025-09-16T17:20:31.590229] [INFO ] [MAIN ] AZZ: Starting full primary refresh. [2025-09-16T17:20:32.317547] [DEBUG] [REMOTE ] B: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:20:32.376314] [WARN ] [REMOTE ] B: No data returned for month 2025-02 [2025-09-16T17:20:32.407507] [INFO ] [REMOTE ] B: Fetching primary data for 2025-03. [2025-09-16T17:20:32.519844] [INFO ] [REMOTE ] AYI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:32.561592] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-02. [2025-09-16T17:20:32.615240] [INFO ] [REMOTE ] AZO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:32.652071] [INFO ] [REMOTE ] AZZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:32.652436] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-02. [2025-09-16T17:20:32.653427] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-02. [2025-09-16T17:20:32.768196] [DEBUG] [REMOTE ] B: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:20:32.768526] [WARN ] [REMOTE ] B: No data returned for month 2025-03 [2025-09-16T17:20:32.769035] [INFO ] [REMOTE ] B: Fetching primary data for 2025-04. [2025-09-16T17:20:32.974833] [DEBUG] [REMOTE ] B: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:20:32.975140] [WARN ] [REMOTE ] B: No data returned for month 2025-04 [2025-09-16T17:20:32.975630] [INFO ] [REMOTE ] B: Fetching primary data for 2025-05. [2025-09-16T17:20:33.472080] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T17:20:33.474284] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-03. [2025-09-16T17:20:33.596182] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-09-16T17:20:33.600811] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-03. [2025-09-16T17:20:33.651553] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-09-16T17:20:33.657132] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-03. [2025-09-16T17:20:34.261726] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T17:20:34.265570] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-04. [2025-09-16T17:20:34.480981] [INFO ] [REMOTE ] B: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-09-16T17:20:34.498262] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-09-16T17:20:34.667564] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-09-16T17:20:34.774954] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-04. [2025-09-16T17:20:35.473179] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-09-16T17:20:35.574951] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-04. [2025-09-16T17:20:35.676375] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-09-16T17:20:36.374926] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-05. [2025-09-16T17:20:36.603393] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T17:20:36.984146] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-09-16T17:20:37.174922] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-05. [2025-09-16T17:20:37.974894] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-05. [2025-09-16T17:20:38.015044] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T17:20:38.651242] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-09-16T17:20:38.774926] [INFO ] [REMOTE ] B: Fetching primary data for 2025-06. [2025-09-16T17:20:39.574920] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-06. [2025-09-16T17:20:40.210159] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-09-16T17:20:40.374911] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-07. [2025-09-16T17:20:40.600303] [INFO ] [REMOTE ] B: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T17:20:41.174920] [INFO ] [REMOTE ] B: Fetching primary data for 2025-07. [2025-09-16T17:20:41.340793] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T17:20:41.974911] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-06. [2025-09-16T17:20:42.775096] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-06. [2025-09-16T17:20:42.888450] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-09-16T17:20:42.963201] [INFO ] [REMOTE ] B: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-09-16T17:20:43.574957] [INFO ] [REMOTE ] B: Fetching primary data for 2025-08. [2025-09-16T17:20:43.787317] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T17:20:44.374920] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-08. [2025-09-16T17:20:44.897447] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-09-16T17:20:45.174926] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-07. [2025-09-16T17:20:45.912228] [INFO ] [REMOTE ] B: Incremental fetch complete for 2025-08 (2.34 seconds). [2025-09-16T17:20:45.974905] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-07. [2025-09-16T17:20:46.421507] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-09-16T17:20:46.774900] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-08. [2025-09-16T17:20:47.574929] [INFO ] [REMOTE ] B: Fetching primary data for 2025-09. [2025-09-16T17:20:47.685811] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-09-16T17:20:47.772231] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-09-16T17:20:48.374931] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-09. [2025-09-16T17:20:49.088439] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-09-16T17:20:49.090094] [INFO ] [REMOTE ] AZO: Full fetch complete (12142 records). [2025-09-16T17:20:49.090166] [DEBUG] [VALIDATOR ] AZO: Validating primary data. [2025-09-16T17:20:49.090585] [FATAL] [VALIDATOR ] AZO: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:20:49.090678] [FATAL] [MAIN ] AZO: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:20:49.090718] [WARN ] [MAIN ] AZO: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:20:49.090755] [INFO ] [LOCAL ] AZO: Starting deletion of primary data. [2025-09-16T17:20:49.090811] [DEBUG] [LOCAL ] AZO: No primary data file found to delete. [2025-09-16T17:20:49.090847] [INFO ] [LOCAL ] AZO: Starting deletion of derivative data. [2025-09-16T17:20:49.091179] [DEBUG] [LOCAL ] AZO: Deleted derivative data file. [2025-09-16T17:20:49.091290] [INFO ] [MAIN ] Update Progress: 201/2369 symbols completed. AZO took 19.3s. Estimated remaining: 3h 43m 59s. Rate: 9.7 symbols/min [2025-09-16T17:20:49.179981] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-08. [2025-09-16T17:20:49.293942] [INFO ] [MAIN ] BA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:49.295884] [INFO ] [REMOTE ] B: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-09-16T17:20:49.302236] [INFO ] [REMOTE ] B: Full fetch complete (34240 records). [2025-09-16T17:20:49.302407] [DEBUG] [VALIDATOR ] B: Validating primary data. [2025-09-16T17:20:49.303109] [FATAL] [VALIDATOR ] B: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:20:49.303199] [FATAL] [MAIN ] B: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:20:49.303238] [WARN ] [MAIN ] B: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:20:49.303285] [INFO ] [LOCAL ] B: Starting deletion of primary data. [2025-09-16T17:20:49.303391] [DEBUG] [LOCAL ] B: No primary data file found to delete. [2025-09-16T17:20:49.303432] [INFO ] [LOCAL ] B: Starting deletion of derivative data. [2025-09-16T17:20:49.303484] [DEBUG] [LOCAL ] B: No derivative data file found to delete. [2025-09-16T17:20:49.303550] [INFO ] [MAIN ] Update Progress: 202/2369 symbols completed. B took 19.5s. Estimated remaining: 3h 42m 49s. Rate: 9.7 symbols/min [2025-09-16T17:20:49.501992] [INFO ] [MAIN ] BABA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:49.517180] [DEBUG] [LOCAL ] BA: Loaded 39751 existing records from primary storage. [2025-09-16T17:20:49.517404] [INFO ] [MAIN ] BA: Loaded 39751 existing records. [2025-09-16T17:20:49.730181] [WARN ] [MAIN ] BA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:20:49.730463] [INFO ] [MAIN ] BA: Wiping existing primary data before full refresh. [2025-09-16T17:20:49.730518] [INFO ] [LOCAL ] BA: Starting deletion of primary data. [2025-09-16T17:20:49.732068] [DEBUG] [LOCAL ] BA: Deleted primary data file. [2025-09-16T17:20:49.732138] [INFO ] [MAIN ] BA: Starting full primary refresh. [2025-09-16T17:20:49.879857] [DEBUG] [LOCAL ] BABA: Loaded 39780 existing records from primary storage. [2025-09-16T17:20:49.911199] [INFO ] [MAIN ] BABA: Loaded 39780 existing records. [2025-09-16T17:20:50.000279] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-09. [2025-09-16T17:20:50.244355] [WARN ] [MAIN ] BABA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:20:50.276536] [INFO ] [MAIN ] BABA: Wiping existing primary data before full refresh. [2025-09-16T17:20:50.317415] [INFO ] [REMOTE ] BA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:50.312655] [INFO ] [LOCAL ] BABA: Starting deletion of primary data. [2025-09-16T17:20:50.324321] [DEBUG] [LOCAL ] BABA: Deleted primary data file. [2025-09-16T17:20:50.329127] [INFO ] [MAIN ] BABA: Starting full primary refresh. [2025-09-16T17:20:50.367428] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-09-16T17:20:50.705058] [INFO ] [REMOTE ] BABA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:50.774873] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-02. [2025-09-16T17:20:50.843273] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T17:20:50.845701] [INFO ] [REMOTE ] AZZ: Full fetch complete (23792 records). [2025-09-16T17:20:50.845773] [DEBUG] [VALIDATOR ] AZZ: Validating primary data. [2025-09-16T17:20:50.846235] [FATAL] [VALIDATOR ] AZZ: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:20:50.846312] [FATAL] [MAIN ] AZZ: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:20:50.846349] [WARN ] [MAIN ] AZZ: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:20:50.846387] [INFO ] [LOCAL ] AZZ: Starting deletion of primary data. [2025-09-16T17:20:50.846445] [DEBUG] [LOCAL ] AZZ: No primary data file found to delete. [2025-09-16T17:20:50.846480] [INFO ] [LOCAL ] AZZ: Starting deletion of derivative data. [2025-09-16T17:20:50.846700] [DEBUG] [LOCAL ] AZZ: Deleted derivative data file. [2025-09-16T17:20:50.846781] [INFO ] [MAIN ] Update Progress: 203/2369 symbols completed. AZZ took 21.0s. Estimated remaining: 3h 41m 54s. Rate: 9.8 symbols/min [2025-09-16T17:20:51.016537] [INFO ] [MAIN ] BAC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:51.060458] [DEBUG] [LOCAL ] BAC: Loaded 39768 existing records from primary storage. [2025-09-16T17:20:51.060601] [INFO ] [MAIN ] BAC: Loaded 39768 existing records. [2025-09-16T17:20:51.237623] [WARN ] [MAIN ] BAC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:20:51.237885] [INFO ] [MAIN ] BAC: Wiping existing primary data before full refresh. [2025-09-16T17:20:51.237932] [INFO ] [LOCAL ] BAC: Starting deletion of primary data. [2025-09-16T17:20:51.239142] [DEBUG] [LOCAL ] BAC: Deleted primary data file. [2025-09-16T17:20:51.239199] [INFO ] [MAIN ] BAC: Starting full primary refresh. [2025-09-16T17:20:51.596093] [INFO ] [REMOTE ] BAC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:51.575346] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-09. [2025-09-16T17:20:52.374862] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-02. [2025-09-16T17:20:52.609609] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-09-16T17:20:52.612390] [INFO ] [REMOTE ] AYI: Full fetch complete (27586 records). [2025-09-16T17:20:52.612469] [DEBUG] [VALIDATOR ] AYI: Validating primary data. [2025-09-16T17:20:52.975582] [DEBUG] [VALIDATOR ] AYI: All expected trading days are present. [2025-09-16T17:20:52.975681] [DEBUG] [VALIDATOR ] AYI: Primary data validation passed. [2025-09-16T17:20:52.976096] [INFO ] [MAIN ] AYI: Fetched primary data is valid. [2025-09-16T17:20:52.976138] [DEBUG] [LOCAL ] AYI: Saving primary data. [2025-09-16T17:20:52.976922] [DEBUG] [LOCAL ] AYI: Trimmed 27586 records to 17656 (keeping 150 days of data) [2025-09-16T17:20:53.091498] [DEBUG] [LOCAL ] AYI: Primary data saved (17656 records). [2025-09-16T17:20:53.091658] [INFO ] [MAIN ] AYI: Full history saved. [2025-09-16T17:20:53.091746] [INFO ] [MAIN ] Update Progress: 204/2369 symbols completed. AYI took 23.3s. Estimated remaining: 3h 41m 6s. Rate: 9.8 symbols/min [2025-09-16T17:20:53.184845] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-02. [2025-09-16T17:20:53.268174] [INFO ] [MAIN ] BAH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:20:53.314878] [DEBUG] [LOCAL ] BAH: Loaded 38077 existing records from primary storage. [2025-09-16T17:20:53.315015] [INFO ] [MAIN ] BAH: Loaded 38077 existing records. [2025-09-16T17:20:53.522582] [WARN ] [MAIN ] BAH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:20:53.522947] [INFO ] [MAIN ] BAH: Wiping existing primary data before full refresh. [2025-09-16T17:20:53.523375] [INFO ] [LOCAL ] BAH: Starting deletion of primary data. [2025-09-16T17:20:53.524443] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-02 (2.75 seconds). [2025-09-16T17:20:53.533902] [DEBUG] [LOCAL ] BAH: Deleted primary data file. [2025-09-16T17:20:53.534474] [INFO ] [MAIN ] BAH: Starting full primary refresh. [2025-09-16T17:20:53.884933] [INFO ] [REMOTE ] BAH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:20:53.974902] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-03. [2025-09-16T17:20:54.722800] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-09-16T17:20:54.774858] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-03. [2025-09-16T17:20:55.535147] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-09-16T17:20:55.574880] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-02. [2025-09-16T17:20:56.374914] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-03. [2025-09-16T17:20:56.679507] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-03 (2.70 seconds). [2025-09-16T17:20:56.860445] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-09-16T17:20:57.174886] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-04. [2025-09-16T17:20:57.239421] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T17:20:57.974907] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-03. [2025-09-16T17:20:58.774966] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-04. [2025-09-16T17:20:58.814681] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-03 (2.44 seconds). [2025-09-16T17:20:59.574932] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-04. [2025-09-16T17:20:59.757803] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-04 (2.58 seconds). [2025-09-16T17:20:59.965692] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-09-16T17:21:00.374902] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-04. [2025-09-16T17:21:01.174901] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-05. [2025-09-16T17:21:01.784143] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-04 (3.01 seconds). [2025-09-16T17:21:01.974897] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-05. [2025-09-16T17:21:02.497940] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-04 (2.92 seconds). [2025-09-16T17:21:02.591921] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-04 (2.22 seconds). [2025-09-16T17:21:02.774892] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-05. [2025-09-16T17:21:03.574894] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-05. [2025-09-16T17:21:04.172536] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-05 (3.00 seconds). [2025-09-16T17:21:04.374911] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-06. [2025-09-16T17:21:04.729471] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-05 (2.75 seconds). [2025-09-16T17:21:05.174871] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-06. [2025-09-16T17:21:05.313609] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-05 (2.54 seconds). [2025-09-16T17:21:05.418772] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T17:21:05.974898] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-06. [2025-09-16T17:21:06.655727] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-06 (2.28 seconds). [2025-09-16T17:21:06.774925] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-06. [2025-09-16T17:21:07.217208] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-06 (2.04 seconds). [2025-09-16T17:21:07.574906] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-07. [2025-09-16T17:21:07.909295] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-09-16T17:21:08.374916] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-07. [2025-09-16T17:21:08.754818] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-09-16T17:21:09.174938] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-07. [2025-09-16T17:21:09.974952] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-07. [2025-09-16T17:21:10.182339] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-07 (2.61 seconds). [2025-09-16T17:21:10.774913] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-08. [2025-09-16T17:21:10.847423] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-09-16T17:21:10.919885] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-07 (2.54 seconds). [2025-09-16T17:21:11.574924] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-08. [2025-09-16T17:21:12.374978] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-08. [2025-09-16T17:21:12.979833] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-08 (2.20 seconds). [2025-09-16T17:21:13.074844] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-07 (3.10 seconds). [2025-09-16T17:21:13.174926] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-09. [2025-09-16T17:21:13.230806] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T17:21:13.974904] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-08. [2025-09-16T17:21:14.556401] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-09-16T17:21:14.774917] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-09. [2025-09-16T17:21:15.106548] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-09-16T17:21:15.113037] [INFO ] [REMOTE ] BA: Full fetch complete (58471 records). [2025-09-16T17:21:15.113154] [DEBUG] [VALIDATOR ] BA: Validating primary data. [2025-09-16T17:21:15.527861] [DEBUG] [VALIDATOR ] BA: All expected trading days are present. [2025-09-16T17:21:15.527965] [DEBUG] [VALIDATOR ] BA: Primary data validation passed. [2025-09-16T17:21:15.528697] [INFO ] [MAIN ] BA: Fetched primary data is valid. [2025-09-16T17:21:15.528756] [DEBUG] [LOCAL ] BA: Saving primary data. [2025-09-16T17:21:15.530172] [DEBUG] [LOCAL ] BA: Trimmed 58471 records to 40141 (keeping 150 days of data) [2025-09-16T17:21:15.589198] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-09. [2025-09-16T17:21:15.791905] [DEBUG] [LOCAL ] BA: Primary data saved (40141 records). [2025-09-16T17:21:15.792040] [INFO ] [MAIN ] BA: Full history saved. [2025-09-16T17:21:15.792123] [INFO ] [MAIN ] Update Progress: 205/2369 symbols completed. BA took 26.7s. Estimated remaining: 3h 43m 55s. Rate: 9.7 symbols/min [2025-09-16T17:21:15.969979] [INFO ] [MAIN ] BAK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:21:15.998774] [DEBUG] [LOCAL ] BAK: Loaded 28792 existing records from primary storage. [2025-09-16T17:21:15.998906] [INFO ] [MAIN ] BAK: Loaded 28792 existing records. [2025-09-16T17:21:16.177916] [WARN ] [MAIN ] BAK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:21:16.178160] [INFO ] [MAIN ] BAK: Wiping existing primary data before full refresh. [2025-09-16T17:21:16.178207] [INFO ] [LOCAL ] BAK: Starting deletion of primary data. [2025-09-16T17:21:16.179197] [DEBUG] [LOCAL ] BAK: Deleted primary data file. [2025-09-16T17:21:16.179251] [INFO ] [MAIN ] BAK: Starting full primary refresh. [2025-09-16T17:21:16.538602] [INFO ] [REMOTE ] BAK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:21:16.538928] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-02. [2025-09-16T17:21:16.612708] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-08 (2.64 seconds). [2025-09-16T17:21:16.989109] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:21:16.995244] [INFO ] [REMOTE ] BAH: Full fetch complete (56050 records). [2025-09-16T17:21:16.995431] [DEBUG] [VALIDATOR ] BAH: Validating primary data. [2025-09-16T17:21:16.996497] [FATAL] [VALIDATOR ] BAH: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:21:16.996579] [FATAL] [MAIN ] BAH: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:21:16.996619] [WARN ] [MAIN ] BAH: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:21:16.996688] [INFO ] [LOCAL ] BAH: Starting deletion of primary data. [2025-09-16T17:21:16.996750] [DEBUG] [LOCAL ] BAH: No primary data file found to delete. [2025-09-16T17:21:16.996786] [INFO ] [LOCAL ] BAH: Starting deletion of derivative data. [2025-09-16T17:21:16.997127] [DEBUG] [LOCAL ] BAH: Deleted derivative data file. [2025-09-16T17:21:16.999376] [INFO ] [MAIN ] Update Progress: 206/2369 symbols completed. BAH took 23.9s. Estimated remaining: 3h 42m 56s. Rate: 9.7 symbols/min [2025-09-16T17:21:17.191997] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-09. [2025-09-16T17:21:17.194776] [INFO ] [MAIN ] BALL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:21:17.197301] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-09 (2.42 seconds). [2025-09-16T17:21:17.204392] [INFO ] [REMOTE ] BAC: Full fetch complete (58488 records). [2025-09-16T17:21:17.204600] [DEBUG] [VALIDATOR ] BAC: Validating primary data. [2025-09-16T17:21:17.539190] [DEBUG] [LOCAL ] BALL: Loaded 38150 existing records from primary storage. [2025-09-16T17:21:17.544820] [INFO ] [MAIN ] BALL: Loaded 38150 existing records. [2025-09-16T17:21:17.732703] [DEBUG] [VALIDATOR ] BAC: All expected trading days are present. [2025-09-16T17:21:17.733053] [DEBUG] [VALIDATOR ] BAC: Primary data validation passed. [2025-09-16T17:21:17.744333] [INFO ] [MAIN ] BAC: Fetched primary data is valid. [2025-09-16T17:21:17.754565] [DEBUG] [LOCAL ] BAC: Saving primary data. [2025-09-16T17:21:17.812166] [WARN ] [MAIN ] BALL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:21:17.812459] [INFO ] [MAIN ] BALL: Wiping existing primary data before full refresh. [2025-09-16T17:21:17.812627] [INFO ] [LOCAL ] BALL: Starting deletion of primary data. [2025-09-16T17:21:17.812861] [DEBUG] [LOCAL ] BAC: Trimmed 58488 records to 40158 (keeping 150 days of data) [2025-09-16T17:21:17.813959] [DEBUG] [LOCAL ] BALL: Deleted primary data file. [2025-09-16T17:21:17.814017] [INFO ] [MAIN ] BALL: Starting full primary refresh. [2025-09-16T17:21:18.310108] [INFO ] [REMOTE ] BALL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:21:18.311407] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-02. [2025-09-16T17:21:18.345341] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-09-16T17:21:18.509803] [DEBUG] [LOCAL ] BAC: Primary data saved (40158 records). [2025-09-16T17:21:18.509945] [INFO ] [MAIN ] BAC: Full history saved. [2025-09-16T17:21:18.511314] [INFO ] [MAIN ] Update Progress: 207/2369 symbols completed. BAC took 27.7s. Estimated remaining: 3h 42m 1s. Rate: 9.7 symbols/min [2025-09-16T17:21:18.686992] [INFO ] [MAIN ] BALY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:21:18.697010] [DEBUG] [LOCAL ] BALY: Loaded 7145 existing records from primary storage. [2025-09-16T17:21:18.697137] [INFO ] [MAIN ] BALY: Loaded 7145 existing records. [2025-09-16T17:21:18.788713] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-03. [2025-09-16T17:21:18.878919] [WARN ] [MAIN ] BALY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:21:18.879163] [INFO ] [MAIN ] BALY: Wiping existing primary data before full refresh. [2025-09-16T17:21:18.879214] [INFO ] [LOCAL ] BALY: Starting deletion of primary data. [2025-09-16T17:21:18.879527] [DEBUG] [LOCAL ] BALY: Deleted primary data file. [2025-09-16T17:21:18.879574] [INFO ] [MAIN ] BALY: Starting full primary refresh. [2025-09-16T17:21:19.228309] [INFO ] [REMOTE ] BALY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:21:19.574943] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-02. [2025-09-16T17:21:19.790174] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T17:21:19.835042] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-09 (2.64 seconds). [2025-09-16T17:21:19.841348] [INFO ] [REMOTE ] BABA: Full fetch complete (58500 records). [2025-09-16T17:21:19.841476] [DEBUG] [VALIDATOR ] BABA: Validating primary data. [2025-09-16T17:21:20.148930] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-09-16T17:21:20.234085] [DEBUG] [VALIDATOR ] BABA: All expected trading days are present. [2025-09-16T17:21:20.234168] [DEBUG] [VALIDATOR ] BABA: Primary data validation passed. [2025-09-16T17:21:20.234826] [INFO ] [MAIN ] BABA: Fetched primary data is valid. [2025-09-16T17:21:20.234873] [DEBUG] [LOCAL ] BABA: Saving primary data. [2025-09-16T17:21:20.236775] [DEBUG] [LOCAL ] BABA: Trimmed 58500 records to 40170 (keeping 150 days of data) [2025-09-16T17:21:20.372446] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-09-16T17:21:20.381686] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-03. [2025-09-16T17:21:20.525624] [DEBUG] [LOCAL ] BABA: Primary data saved (40170 records). [2025-09-16T17:21:20.525774] [INFO ] [MAIN ] BABA: Full history saved. [2025-09-16T17:21:20.527869] [INFO ] [MAIN ] Update Progress: 208/2369 symbols completed. BABA took 31.2s. Estimated remaining: 3h 41m 12s. Rate: 9.8 symbols/min [2025-09-16T17:21:20.696624] [INFO ] [MAIN ] BAM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:21:20.735137] [DEBUG] [LOCAL ] BAM: Loaded 38250 existing records from primary storage. [2025-09-16T17:21:20.735283] [INFO ] [MAIN ] BAM: Loaded 38250 existing records. [2025-09-16T17:21:20.917989] [WARN ] [MAIN ] BAM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:21:20.918427] [INFO ] [MAIN ] BAM: Wiping existing primary data before full refresh. [2025-09-16T17:21:20.918478] [INFO ] [LOCAL ] BAM: Starting deletion of primary data. [2025-09-16T17:21:20.919828] [DEBUG] [LOCAL ] BAM: Deleted primary data file. [2025-09-16T17:21:20.919880] [INFO ] [MAIN ] BAM: Starting full primary refresh. [2025-09-16T17:21:21.182992] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-03. [2025-09-16T17:21:21.212809] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T17:21:21.295551] [INFO ] [REMOTE ] BAM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:21:21.974919] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-04. [2025-09-16T17:21:22.651714] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T17:21:22.774908] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-04. [2025-09-16T17:21:23.025916] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T17:21:23.574897] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-02. [2025-09-16T17:21:24.363814] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-09-16T17:21:24.374839] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-05. [2025-09-16T17:21:24.900878] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-09-16T17:21:25.174870] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-04. [2025-09-16T17:21:25.804789] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-09-16T17:21:25.974941] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-05. [2025-09-16T17:21:26.573156] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-09-16T17:21:26.608621] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:21:26.774911] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-03. [2025-09-16T17:21:27.574946] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-06. [2025-09-16T17:21:28.374951] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-06. [2025-09-16T17:21:28.551061] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T17:21:28.728185] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-09-16T17:21:29.174938] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-07. [2025-09-16T17:21:29.432778] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-09-16T17:21:29.974942] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-05. [2025-09-16T17:21:30.299831] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T17:21:30.774933] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-08. [2025-09-16T17:21:31.302256] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T17:21:31.574922] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-06. [2025-09-16T17:21:31.842454] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T17:21:32.374909] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-09. [2025-09-16T17:21:33.174897] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-04. [2025-09-16T17:21:33.276246] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T17:21:33.589534] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-09-16T17:21:33.594139] [INFO ] [REMOTE ] BAK: Full fetch complete (42228 records). [2025-09-16T17:21:33.594275] [DEBUG] [VALIDATOR ] BAK: Validating primary data. [2025-09-16T17:21:33.977905] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-07. [2025-09-16T17:21:33.978156] [DEBUG] [VALIDATOR ] BAK: All expected trading days are present. [2025-09-16T17:21:33.979145] [DEBUG] [VALIDATOR ] BAK: Primary data validation passed. [2025-09-16T17:21:33.979876] [INFO ] [MAIN ] BAK: Fetched primary data is valid. [2025-09-16T17:21:33.980030] [DEBUG] [LOCAL ] BAK: Saving primary data. [2025-09-16T17:21:33.981473] [DEBUG] [LOCAL ] BAK: Trimmed 42228 records to 29020 (keeping 150 days of data) [2025-09-16T17:21:34.161973] [DEBUG] [LOCAL ] BAK: Primary data saved (29020 records). [2025-09-16T17:21:34.162136] [INFO ] [MAIN ] BAK: Full history saved. [2025-09-16T17:21:34.162250] [INFO ] [MAIN ] Update Progress: 209/2369 symbols completed. BAK took 18.4s. Estimated remaining: 3h 42m 23s. Rate: 9.7 symbols/min [2025-09-16T17:21:34.339580] [INFO ] [MAIN ] BANC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:21:34.377525] [DEBUG] [LOCAL ] BANC: Loaded 37542 existing records from primary storage. [2025-09-16T17:21:34.377714] [INFO ] [MAIN ] BANC: Loaded 37542 existing records. [2025-09-16T17:21:34.559679] [WARN ] [MAIN ] BANC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:21:34.560034] [INFO ] [MAIN ] BANC: Wiping existing primary data before full refresh. [2025-09-16T17:21:34.560200] [INFO ] [LOCAL ] BANC: Starting deletion of primary data. [2025-09-16T17:21:34.561533] [DEBUG] [LOCAL ] BANC: Deleted primary data file. [2025-09-16T17:21:34.561590] [INFO ] [MAIN ] BANC: Starting full primary refresh. [2025-09-16T17:21:34.777429] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-07. [2025-09-16T17:21:34.973461] [INFO ] [REMOTE ] BANC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:21:34.979312] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-09-16T17:21:35.509063] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-09-16T17:21:35.574986] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-05. [2025-09-16T17:21:36.009408] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-09-16T17:21:36.374955] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-08. [2025-09-16T17:21:37.174871] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-02. [2025-09-16T17:21:37.267443] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-09-16T17:21:37.974952] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-06. [2025-09-16T17:21:38.118925] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-09-16T17:21:38.731758] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T17:21:38.774919] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-03. [2025-09-16T17:21:39.268074] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T17:21:39.574905] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-08. [2025-09-16T17:21:40.097314] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-09-16T17:21:40.374919] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-09. [2025-09-16T17:21:40.577973] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T17:21:41.174879] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-07. [2025-09-16T17:21:41.252181] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-09-16T17:21:41.254283] [INFO ] [REMOTE ] BALY: Full fetch complete (9127 records). [2025-09-16T17:21:41.254421] [DEBUG] [VALIDATOR ] BALY: Validating primary data. [2025-09-16T17:21:41.636805] [DEBUG] [VALIDATOR ] BALY: All expected trading days are present. [2025-09-16T17:21:41.636900] [DEBUG] [VALIDATOR ] BALY: Primary data validation passed. [2025-09-16T17:21:41.637168] [INFO ] [MAIN ] BALY: Fetched primary data is valid. [2025-09-16T17:21:41.637224] [DEBUG] [LOCAL ] BALY: Saving primary data. [2025-09-16T17:21:41.637800] [DEBUG] [LOCAL ] BALY: Trimmed 9127 records to 7203 (keeping 150 days of data) [2025-09-16T17:21:41.680922] [DEBUG] [LOCAL ] BALY: Primary data saved (7203 records). [2025-09-16T17:21:41.681043] [INFO ] [MAIN ] BALY: Full history saved. [2025-09-16T17:21:41.681140] [INFO ] [MAIN ] Update Progress: 210/2369 symbols completed. BALY took 23.2s. Estimated remaining: 3h 42m 31s. Rate: 9.7 symbols/min [2025-09-16T17:21:41.870515] [INFO ] [MAIN ] BAP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:21:41.889927] [DEBUG] [LOCAL ] BAP: Loaded 18874 existing records from primary storage. [2025-09-16T17:21:41.890096] [INFO ] [MAIN ] BAP: Loaded 18874 existing records. [2025-09-16T17:21:41.982171] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-04. [2025-09-16T17:21:42.095859] [WARN ] [MAIN ] BAP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:21:42.096115] [INFO ] [MAIN ] BAP: Wiping existing primary data before full refresh. [2025-09-16T17:21:42.096162] [INFO ] [LOCAL ] BAP: Starting deletion of primary data. [2025-09-16T17:21:42.096955] [DEBUG] [LOCAL ] BAP: Deleted primary data file. [2025-09-16T17:21:42.097007] [INFO ] [MAIN ] BAP: Starting full primary refresh. [2025-09-16T17:21:42.473758] [INFO ] [REMOTE ] BAP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:21:42.663733] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T17:21:42.774900] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-08. [2025-09-16T17:21:43.465959] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T17:21:43.574895] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-05. [2025-09-16T17:21:44.306575] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T17:21:44.374848] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-09. [2025-09-16T17:21:45.174949] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-09. [2025-09-16T17:21:45.247216] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-09-16T17:21:45.901187] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-09-16T17:21:45.907621] [INFO ] [REMOTE ] BAM: Full fetch complete (56556 records). [2025-09-16T17:21:45.907758] [DEBUG] [VALIDATOR ] BAM: Validating primary data. [2025-09-16T17:21:45.983549] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-02. [2025-09-16T17:21:46.292746] [DEBUG] [VALIDATOR ] BAM: All expected trading days are present. [2025-09-16T17:21:46.292839] [DEBUG] [VALIDATOR ] BAM: Primary data validation passed. [2025-09-16T17:21:46.293535] [INFO ] [MAIN ] BAM: Fetched primary data is valid. [2025-09-16T17:21:46.293611] [DEBUG] [LOCAL ] BAM: Saving primary data. [2025-09-16T17:21:46.294948] [DEBUG] [LOCAL ] BAM: Trimmed 56556 records to 38634 (keeping 150 days of data) [2025-09-16T17:21:46.543375] [DEBUG] [LOCAL ] BAM: Primary data saved (38634 records). [2025-09-16T17:21:46.543512] [INFO ] [MAIN ] BAM: Full history saved. [2025-09-16T17:21:46.543592] [INFO ] [MAIN ] Update Progress: 211/2369 symbols completed. BAM took 26.0s. Estimated remaining: 3h 42m 11s. Rate: 9.7 symbols/min [2025-09-16T17:21:46.714840] [INFO ] [MAIN ] BARK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:21:46.715029] [DEBUG] [LOCAL ] BARK: No existing data file found in primary storage. [2025-09-16T17:21:46.715068] [WARN ] [MAIN ] BARK: No existing data found, performing full refresh instead. [2025-09-16T17:21:46.715192] [INFO ] [MAIN ] BARK: Wiping existing primary data before full refresh. [2025-09-16T17:21:46.715230] [INFO ] [LOCAL ] BARK: Starting deletion of primary data. [2025-09-16T17:21:46.715272] [DEBUG] [LOCAL ] BARK: No primary data file found to delete. [2025-09-16T17:21:46.715304] [INFO ] [MAIN ] BARK: Starting full primary refresh. [2025-09-16T17:21:46.778767] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-06. [2025-09-16T17:21:47.077826] [INFO ] [REMOTE ] BARK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:21:47.095057] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-09-16T17:21:47.101207] [INFO ] [REMOTE ] BALL: Full fetch complete (56473 records). [2025-09-16T17:21:47.101335] [DEBUG] [VALIDATOR ] BALL: Validating primary data. [2025-09-16T17:21:47.498766] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T17:21:47.501626] [DEBUG] [VALIDATOR ] BALL: All expected trading days are present. [2025-09-16T17:21:47.507097] [DEBUG] [VALIDATOR ] BALL: Primary data validation passed. [2025-09-16T17:21:47.507748] [INFO ] [MAIN ] BALL: Fetched primary data is valid. [2025-09-16T17:21:47.507797] [DEBUG] [LOCAL ] BALL: Saving primary data. [2025-09-16T17:21:47.508933] [DEBUG] [LOCAL ] BALL: Trimmed 56473 records to 38534 (keeping 150 days of data) [2025-09-16T17:21:47.575604] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-02. [2025-09-16T17:21:47.748031] [DEBUG] [LOCAL ] BALL: Primary data saved (38534 records). [2025-09-16T17:21:47.748186] [INFO ] [MAIN ] BALL: Full history saved. [2025-09-16T17:21:47.749276] [INFO ] [MAIN ] Update Progress: 212/2369 symbols completed. BALL took 30.7s. Estimated remaining: 3h 41m 14s. Rate: 9.7 symbols/min [2025-09-16T17:21:47.924897] [INFO ] [MAIN ] BAX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:21:47.925100] [DEBUG] [LOCAL ] BAX: No existing data file found in primary storage. [2025-09-16T17:21:47.925139] [WARN ] [MAIN ] BAX: No existing data found, performing full refresh instead. [2025-09-16T17:21:47.925275] [INFO ] [MAIN ] BAX: Wiping existing primary data before full refresh. [2025-09-16T17:21:47.925315] [INFO ] [LOCAL ] BAX: Starting deletion of primary data. [2025-09-16T17:21:47.925356] [DEBUG] [LOCAL ] BAX: No primary data file found to delete. [2025-09-16T17:21:47.925389] [INFO ] [MAIN ] BAX: Starting full primary refresh. [2025-09-16T17:21:48.279792] [INFO ] [REMOTE ] BAX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:21:48.317159] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-09-16T17:21:48.374906] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-07. [2025-09-16T17:21:49.106356] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T17:21:49.174926] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-02. [2025-09-16T17:21:49.895441] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-09-16T17:21:49.974938] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-03. [2025-09-16T17:21:50.548815] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T17:21:50.774900] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-08. [2025-09-16T17:21:51.092145] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-09-16T17:21:51.574909] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-03. [2025-09-16T17:21:52.374942] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-04. [2025-09-16T17:21:52.739680] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-09-16T17:21:53.174923] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-09. [2025-09-16T17:21:53.228248] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-09-16T17:21:53.371043] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-09-16T17:21:53.974896] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-05. [2025-09-16T17:21:54.374164] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-09-16T17:21:54.380170] [INFO ] [REMOTE ] BANC: Full fetch complete (55599 records). [2025-09-16T17:21:54.380269] [DEBUG] [VALIDATOR ] BANC: Validating primary data. [2025-09-16T17:21:54.755493] [DEBUG] [VALIDATOR ] BANC: All expected trading days are present. [2025-09-16T17:21:54.755570] [DEBUG] [VALIDATOR ] BANC: Primary data validation passed. [2025-09-16T17:21:54.756186] [INFO ] [MAIN ] BANC: Fetched primary data is valid. [2025-09-16T17:21:54.756232] [DEBUG] [LOCAL ] BANC: Saving primary data. [2025-09-16T17:21:54.757331] [DEBUG] [LOCAL ] BANC: Trimmed 55599 records to 37899 (keeping 150 days of data) [2025-09-16T17:21:54.813932] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-04. [2025-09-16T17:21:54.884252] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T17:21:55.014624] [DEBUG] [LOCAL ] BANC: Primary data saved (37899 records). [2025-09-16T17:21:55.014790] [INFO ] [MAIN ] BANC: Full history saved. [2025-09-16T17:21:55.014890] [INFO ] [MAIN ] Update Progress: 213/2369 symbols completed. BANC took 20.9s. Estimated remaining: 3h 41m 19s. Rate: 9.7 symbols/min [2025-09-16T17:21:55.187753] [INFO ] [MAIN ] BB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:21:55.225490] [DEBUG] [LOCAL ] BB: Loaded 38976 existing records from primary storage. [2025-09-16T17:21:55.225667] [INFO ] [MAIN ] BB: Loaded 38976 existing records. [2025-09-16T17:21:55.401454] [WARN ] [MAIN ] BB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:21:55.401725] [INFO ] [MAIN ] BB: Wiping existing primary data before full refresh. [2025-09-16T17:21:55.401777] [INFO ] [LOCAL ] BB: Starting deletion of primary data. [2025-09-16T17:21:55.403690] [DEBUG] [LOCAL ] BB: Deleted primary data file. [2025-09-16T17:21:55.403754] [INFO ] [MAIN ] BB: Starting full primary refresh. [2025-09-16T17:21:55.581946] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-03. [2025-09-16T17:21:55.756918] [INFO ] [REMOTE ] BB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:21:56.372820] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T17:21:56.380692] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-06. [2025-09-16T17:21:57.174871] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-02. [2025-09-16T17:21:57.295907] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T17:21:57.374060] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-09-16T17:21:57.974904] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-04. [2025-09-16T17:21:58.774899] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-05. [2025-09-16T17:21:59.369957] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-02 (2.20 seconds). [2025-09-16T17:21:59.574906] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-07. [2025-09-16T17:21:59.759583] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-09-16T17:21:59.846594] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-09-16T17:22:00.374905] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-05. [2025-09-16T17:22:00.516719] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-09-16T17:22:01.174915] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-03. [2025-09-16T17:22:01.976638] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-08. [2025-09-16T17:22:01.983274] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T17:22:02.774891] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-06. [2025-09-16T17:22:02.957772] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-09-16T17:22:03.238949] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-03 (2.06 seconds). [2025-09-16T17:22:03.574877] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-09. [2025-09-16T17:22:04.244400] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-09-16T17:22:04.374945] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-04. [2025-09-16T17:22:04.889605] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T17:22:04.893428] [INFO ] [REMOTE ] BAP: Full fetch complete (28711 records). [2025-09-16T17:22:04.893507] [DEBUG] [VALIDATOR ] BAP: Validating primary data. [2025-09-16T17:22:05.181690] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-07. [2025-09-16T17:22:05.290084] [DEBUG] [VALIDATOR ] BAP: All expected trading days are present. [2025-09-16T17:22:05.290158] [DEBUG] [VALIDATOR ] BAP: Primary data validation passed. [2025-09-16T17:22:05.290535] [INFO ] [MAIN ] BAP: Fetched primary data is valid. [2025-09-16T17:22:05.290592] [DEBUG] [LOCAL ] BAP: Saving primary data. [2025-09-16T17:22:05.291607] [DEBUG] [LOCAL ] BAP: Trimmed 28711 records to 19126 (keeping 150 days of data) [2025-09-16T17:22:05.410054] [DEBUG] [LOCAL ] BAP: Primary data saved (19126 records). [2025-09-16T17:22:05.410194] [INFO ] [MAIN ] BAP: Full history saved. [2025-09-16T17:22:05.410276] [INFO ] [MAIN ] Update Progress: 214/2369 symbols completed. BAP took 23.7s. Estimated remaining: 3h 41m 56s. Rate: 9.7 symbols/min [2025-09-16T17:22:05.579309] [INFO ] [MAIN ] BBAI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:22:05.622382] [DEBUG] [LOCAL ] BBAI: Loaded 39780 existing records from primary storage. [2025-09-16T17:22:05.622546] [INFO ] [MAIN ] BBAI: Loaded 39780 existing records. [2025-09-16T17:22:05.801458] [WARN ] [MAIN ] BBAI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:22:05.801746] [INFO ] [MAIN ] BBAI: Wiping existing primary data before full refresh. [2025-09-16T17:22:05.801801] [INFO ] [LOCAL ] BBAI: Starting deletion of primary data. [2025-09-16T17:22:05.804324] [DEBUG] [LOCAL ] BBAI: Deleted primary data file. [2025-09-16T17:22:05.804400] [INFO ] [MAIN ] BBAI: Starting full primary refresh. [2025-09-16T17:22:05.976128] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-06. [2025-09-16T17:22:06.159017] [INFO ] [REMOTE ] BBAI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:22:06.216412] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T17:22:06.774914] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-02. [2025-09-16T17:22:07.136457] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-09-16T17:22:07.574879] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-08. [2025-09-16T17:22:07.613617] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T17:22:08.374994] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-07. [2025-09-16T17:22:09.181154] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-05. [2025-09-16T17:22:09.202353] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-09-16T17:22:09.343813] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-02 (2.57 seconds). [2025-09-16T17:22:09.942247] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T17:22:09.974864] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-08. [2025-09-16T17:22:10.774949] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-09. [2025-09-16T17:22:11.159724] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-09-16T17:22:11.192354] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-09-16T17:22:11.574924] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-09. [2025-09-16T17:22:12.312707] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T17:22:12.319197] [INFO ] [REMOTE ] BAX: Full fetch complete (58161 records). [2025-09-16T17:22:12.319306] [DEBUG] [VALIDATOR ] BAX: Validating primary data. [2025-09-16T17:22:12.383822] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-03. [2025-09-16T17:22:12.720786] [DEBUG] [VALIDATOR ] BAX: All expected trading days are present. [2025-09-16T17:22:12.720901] [DEBUG] [VALIDATOR ] BAX: Primary data validation passed. [2025-09-16T17:22:12.722150] [INFO ] [MAIN ] BAX: Fetched primary data is valid. [2025-09-16T17:22:12.722205] [DEBUG] [LOCAL ] BAX: Saving primary data. [2025-09-16T17:22:12.723807] [DEBUG] [LOCAL ] BAX: Trimmed 58161 records to 39905 (keeping 150 days of data) [2025-09-16T17:22:12.964078] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T17:22:12.970112] [INFO ] [REMOTE ] BARK: Full fetch complete (42111 records). [2025-09-16T17:22:12.970794] [DEBUG] [VALIDATOR ] BARK: Validating primary data. [2025-09-16T17:22:13.179272] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-06. [2025-09-16T17:22:13.400354] [DEBUG] [VALIDATOR ] BARK: All expected trading days are present. [2025-09-16T17:22:13.400804] [DEBUG] [VALIDATOR ] BARK: Primary data validation passed. [2025-09-16T17:22:13.402260] [INFO ] [MAIN ] BARK: Fetched primary data is valid. [2025-09-16T17:22:13.402609] [DEBUG] [LOCAL ] BARK: Saving primary data. [2025-09-16T17:22:13.406959] [DEBUG] [LOCAL ] BARK: Trimmed 42111 records to 28629 (keeping 150 days of data) [2025-09-16T17:22:13.500037] [DEBUG] [LOCAL ] BAX: Primary data saved (39905 records). [2025-09-16T17:22:13.502135] [INFO ] [MAIN ] BAX: Full history saved. [2025-09-16T17:22:13.502947] [INFO ] [MAIN ] Update Progress: 215/2369 symbols completed. BAX took 25.8s. Estimated remaining: 3h 42m 9s. Rate: 9.7 symbols/min [2025-09-16T17:22:13.693038] [INFO ] [MAIN ] BBAR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:22:13.767821] [DEBUG] [LOCAL ] BBAR: Loaded 30538 existing records from primary storage. [2025-09-16T17:22:13.769897] [INFO ] [MAIN ] BBAR: Loaded 30538 existing records. [2025-09-16T17:22:13.960943] [WARN ] [MAIN ] BBAR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:22:13.969415] [DEBUG] [LOCAL ] BARK: Primary data saved (28629 records). [2025-09-16T17:22:13.969855] [INFO ] [MAIN ] BARK: Full history saved. [2025-09-16T17:22:13.969996] [INFO ] [MAIN ] Update Progress: 216/2369 symbols completed. BARK took 27.4s. Estimated remaining: 3h 41m 6s. Rate: 9.7 symbols/min [2025-09-16T17:22:13.969615] [INFO ] [MAIN ] BBAR: Wiping existing primary data before full refresh. [2025-09-16T17:22:13.970177] [INFO ] [LOCAL ] BBAR: Starting deletion of primary data. [2025-09-16T17:22:13.975781] [DEBUG] [LOCAL ] BBAR: Deleted primary data file. [2025-09-16T17:22:14.036724] [INFO ] [MAIN ] BBAR: Starting full primary refresh. [2025-09-16T17:22:14.272886] [INFO ] [MAIN ] BBD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:22:14.540035] [INFO ] [REMOTE ] BBAR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:22:14.540433] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-02. [2025-09-16T17:22:14.554827] [DEBUG] [LOCAL ] BBD: Loaded 39316 existing records from primary storage. [2025-09-16T17:22:14.554987] [INFO ] [MAIN ] BBD: Loaded 39316 existing records. [2025-09-16T17:22:14.740610] [WARN ] [MAIN ] BBD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:22:14.741013] [INFO ] [MAIN ] BBD: Wiping existing primary data before full refresh. [2025-09-16T17:22:14.741116] [INFO ] [LOCAL ] BBD: Starting deletion of primary data. [2025-09-16T17:22:14.742805] [DEBUG] [LOCAL ] BBD: Deleted primary data file. [2025-09-16T17:22:14.742866] [INFO ] [MAIN ] BBD: Starting full primary refresh. [2025-09-16T17:22:15.018933] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-03 (2.64 seconds). [2025-09-16T17:22:15.074348] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-04. [2025-09-16T17:22:15.147867] [INFO ] [REMOTE ] BBD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:22:15.183827] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-09-16T17:22:15.574908] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-02. [2025-09-16T17:22:15.947149] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-09-16T17:22:16.374915] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-07. [2025-09-16T17:22:17.175032] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-03. [2025-09-16T17:22:17.313022] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T17:22:17.431220] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-04 (2.36 seconds). [2025-09-16T17:22:17.974908] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-05. [2025-09-16T17:22:18.160622] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-09-16T17:22:18.370355] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T17:22:18.774927] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-08. [2025-09-16T17:22:19.574929] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-03. [2025-09-16T17:22:20.374954] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-04. [2025-09-16T17:22:20.694085] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-09-16T17:22:20.817233] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-05 (2.84 seconds). [2025-09-16T17:22:20.920194] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T17:22:21.174932] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-04. [2025-09-16T17:22:21.723387] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T17:22:21.974914] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-09. [2025-09-16T17:22:22.746742] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-09-16T17:22:22.774843] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-05. [2025-09-16T17:22:23.574883] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-06. [2025-09-16T17:22:23.985169] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-09-16T17:22:23.991719] [INFO ] [REMOTE ] BB: Full fetch complete (57610 records). [2025-09-16T17:22:23.991835] [DEBUG] [VALIDATOR ] BB: Validating primary data. [2025-09-16T17:22:24.382121] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-05. [2025-09-16T17:22:24.384161] [DEBUG] [VALIDATOR ] BB: All expected trading days are present. [2025-09-16T17:22:24.384286] [DEBUG] [VALIDATOR ] BB: Primary data validation passed. [2025-09-16T17:22:24.385009] [INFO ] [MAIN ] BB: Fetched primary data is valid. [2025-09-16T17:22:24.385062] [DEBUG] [LOCAL ] BB: Saving primary data. [2025-09-16T17:22:24.387756] [DEBUG] [LOCAL ] BB: Trimmed 57610 records to 39362 (keeping 150 days of data) [2025-09-16T17:22:24.558327] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-09-16T17:22:24.682984] [DEBUG] [LOCAL ] BB: Primary data saved (39362 records). [2025-09-16T17:22:24.683174] [INFO ] [MAIN ] BB: Full history saved. [2025-09-16T17:22:24.683342] [INFO ] [MAIN ] Update Progress: 217/2369 symbols completed. BB took 29.7s. Estimated remaining: 3h 41m 45s. Rate: 9.7 symbols/min [2025-09-16T17:22:24.861930] [INFO ] [MAIN ] BBDC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:22:24.888258] [DEBUG] [LOCAL ] BBDC: Loaded 23228 existing records from primary storage. [2025-09-16T17:22:24.888411] [INFO ] [MAIN ] BBDC: Loaded 23228 existing records. [2025-09-16T17:22:25.072227] [WARN ] [MAIN ] BBDC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:22:25.072502] [INFO ] [MAIN ] BBDC: Wiping existing primary data before full refresh. [2025-09-16T17:22:25.072551] [INFO ] [LOCAL ] BBDC: Starting deletion of primary data. [2025-09-16T17:22:25.074123] [DEBUG] [LOCAL ] BBDC: Deleted primary data file. [2025-09-16T17:22:25.074194] [INFO ] [MAIN ] BBDC: Starting full primary refresh. [2025-09-16T17:22:25.179980] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-06. [2025-09-16T17:22:25.447378] [INFO ] [REMOTE ] BBDC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:22:25.579371] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T17:22:25.974941] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-06. [2025-09-16T17:22:26.205589] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-06 (2.63 seconds). [2025-09-16T17:22:26.777682] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-07. [2025-09-16T17:22:26.780310] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-09-16T17:22:27.561168] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T17:22:27.574910] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-07. [2025-09-16T17:22:28.375013] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-07. [2025-09-16T17:22:28.952075] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-09-16T17:22:29.174969] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-08. [2025-09-16T17:22:29.543098] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-07 (2.77 seconds). [2025-09-16T17:22:29.974965] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-08. [2025-09-16T17:22:30.336071] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-09-16T17:22:30.414118] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T17:22:30.774903] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-09. [2025-09-16T17:22:31.574952] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-02. [2025-09-16T17:22:32.324408] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-08 (2.35 seconds). [2025-09-16T17:22:32.363700] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T17:22:32.368982] [INFO ] [REMOTE ] BBAR: Full fetch complete (44658 records). [2025-09-16T17:22:32.369082] [DEBUG] [VALIDATOR ] BBAR: Validating primary data. [2025-09-16T17:22:32.381288] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-08. [2025-09-16T17:22:32.771177] [DEBUG] [VALIDATOR ] BBAR: All expected trading days are present. [2025-09-16T17:22:32.771393] [DEBUG] [VALIDATOR ] BBAR: Primary data validation passed. [2025-09-16T17:22:32.771974] [INFO ] [MAIN ] BBAR: Fetched primary data is valid. [2025-09-16T17:22:32.772031] [DEBUG] [LOCAL ] BBAR: Saving primary data. [2025-09-16T17:22:32.784079] [DEBUG] [LOCAL ] BBAR: Trimmed 44658 records to 30859 (keeping 150 days of data) [2025-09-16T17:22:32.802083] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-09-16T17:22:32.997918] [DEBUG] [LOCAL ] BBAR: Primary data saved (30859 records). [2025-09-16T17:22:32.998062] [INFO ] [MAIN ] BBAR: Full history saved. [2025-09-16T17:22:32.999125] [INFO ] [MAIN ] Update Progress: 218/2369 symbols completed. BBAR took 19.5s. Estimated remaining: 3h 41m 60s. Rate: 9.7 symbols/min [2025-09-16T17:22:33.169399] [INFO ] [MAIN ] BBDO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:22:33.175520] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-09. [2025-09-16T17:22:33.175857] [DEBUG] [LOCAL ] BBDO: Loaded 3549 existing records from primary storage. [2025-09-16T17:22:33.176824] [INFO ] [MAIN ] BBDO: Loaded 3549 existing records. [2025-09-16T17:22:33.356395] [WARN ] [MAIN ] BBDO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:22:33.356694] [INFO ] [MAIN ] BBDO: Wiping existing primary data before full refresh. [2025-09-16T17:22:33.356743] [INFO ] [LOCAL ] BBDO: Starting deletion of primary data. [2025-09-16T17:22:33.357039] [DEBUG] [LOCAL ] BBDO: Deleted primary data file. [2025-09-16T17:22:33.357083] [INFO ] [MAIN ] BBDO: Starting full primary refresh. [2025-09-16T17:22:33.723820] [INFO ] [REMOTE ] BBDO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:22:33.896361] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-09-16T17:22:33.974929] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-03. [2025-09-16T17:22:34.774935] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-09. [2025-09-16T17:22:35.118302] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T17:22:35.574926] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-02. [2025-09-16T17:22:35.923880] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-09 (2.75 seconds). [2025-09-16T17:22:35.932370] [INFO ] [REMOTE ] BBAI: Full fetch complete (58500 records). [2025-09-16T17:22:35.932550] [DEBUG] [VALIDATOR ] BBAI: Validating primary data. [2025-09-16T17:22:36.238484] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-09-16T17:22:36.327814] [DEBUG] [VALIDATOR ] BBAI: All expected trading days are present. [2025-09-16T17:22:36.327884] [DEBUG] [VALIDATOR ] BBAI: Primary data validation passed. [2025-09-16T17:22:36.328534] [INFO ] [MAIN ] BBAI: Fetched primary data is valid. [2025-09-16T17:22:36.328580] [DEBUG] [LOCAL ] BBAI: Saving primary data. [2025-09-16T17:22:36.329733] [DEBUG] [LOCAL ] BBAI: Trimmed 58500 records to 40170 (keeping 150 days of data) [2025-09-16T17:22:36.385501] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-04. [2025-09-16T17:22:36.566839] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-09-16T17:22:36.575212] [INFO ] [REMOTE ] BBD: Full fetch complete (57832 records). [2025-09-16T17:22:36.576170] [DEBUG] [VALIDATOR ] BBD: Validating primary data. [2025-09-16T17:22:37.005281] [DEBUG] [VALIDATOR ] BBD: All expected trading days are present. [2025-09-16T17:22:37.005778] [DEBUG] [VALIDATOR ] BBD: Primary data validation passed. [2025-09-16T17:22:37.007266] [INFO ] [MAIN ] BBD: Fetched primary data is valid. [2025-09-16T17:22:37.012813] [DEBUG] [LOCAL ] BBD: Saving primary data. [2025-09-16T17:22:37.012948] [DEBUG] [LOCAL ] BBAI: Primary data saved (40170 records). [2025-09-16T17:22:37.013107] [INFO ] [MAIN ] BBAI: Full history saved. [2025-09-16T17:22:37.014762] [INFO ] [MAIN ] Update Progress: 219/2369 symbols completed. BBAI took 31.6s. Estimated remaining: 3h 41m 32s. Rate: 9.7 symbols/min [2025-09-16T17:22:37.065694] [DEBUG] [LOCAL ] BBD: Trimmed 57832 records to 39702 (keeping 150 days of data) [2025-09-16T17:22:37.208217] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-03. [2025-09-16T17:22:37.302934] [INFO ] [MAIN ] BBN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:22:37.322224] [DEBUG] [LOCAL ] BBN: Loaded 13404 existing records from primary storage. [2025-09-16T17:22:37.322845] [INFO ] [MAIN ] BBN: Loaded 13404 existing records. [2025-09-16T17:22:37.531441] [WARN ] [MAIN ] BBN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:22:37.533537] [INFO ] [MAIN ] BBN: Wiping existing primary data before full refresh. [2025-09-16T17:22:37.534946] [INFO ] [LOCAL ] BBN: Starting deletion of primary data. [2025-09-16T17:22:37.537254] [DEBUG] [LOCAL ] BBN: Deleted primary data file. [2025-09-16T17:22:37.537424] [INFO ] [MAIN ] BBN: Starting full primary refresh. [2025-09-16T17:22:38.015868] [INFO ] [REMOTE ] BBN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:22:38.016610] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-02. [2025-09-16T17:22:38.017096] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-09-16T17:22:38.046250] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T17:22:38.087839] [DEBUG] [LOCAL ] BBD: Primary data saved (39702 records). [2025-09-16T17:22:38.088003] [INFO ] [MAIN ] BBD: Full history saved. [2025-09-16T17:22:38.088093] [INFO ] [MAIN ] Update Progress: 220/2369 symbols completed. BBD took 24.1s. Estimated remaining: 3h 40m 36s. Rate: 9.7 symbols/min [2025-09-16T17:22:38.268229] [INFO ] [MAIN ] BBT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:22:38.268446] [DEBUG] [LOCAL ] BBT: No existing data file found in primary storage. [2025-09-16T17:22:38.268485] [WARN ] [MAIN ] BBT: No existing data found, performing full refresh instead. [2025-09-16T17:22:38.268607] [INFO ] [MAIN ] BBT: Wiping existing primary data before full refresh. [2025-09-16T17:22:38.268666] [INFO ] [LOCAL ] BBT: Starting deletion of primary data. [2025-09-16T17:22:38.268711] [DEBUG] [LOCAL ] BBT: No primary data file found to delete. [2025-09-16T17:22:38.268753] [INFO ] [MAIN ] BBT: Starting full primary refresh. [2025-09-16T17:22:38.635083] [INFO ] [REMOTE ] BBT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:22:38.762529] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T17:22:38.774841] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-03. [2025-09-16T17:22:39.400137] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-09-16T17:22:39.574878] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-04. [2025-09-16T17:22:40.374937] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-05. [2025-09-16T17:22:40.384773] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-09-16T17:22:41.174929] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-04. [2025-09-16T17:22:41.271793] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-09-16T17:22:41.911099] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-09-16T17:22:41.974878] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-02. [2025-09-16T17:22:42.214918] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:22:42.215222] [WARN ] [REMOTE ] BBT: No data returned for month 2025-02 [2025-09-16T17:22:42.774891] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-03. [2025-09-16T17:22:42.989694] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:22:42.990002] [WARN ] [REMOTE ] BBT: No data returned for month 2025-03 [2025-09-16T17:22:43.574901] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-05. [2025-09-16T17:22:44.374946] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-05. [2025-09-16T17:22:44.506565] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-09-16T17:22:45.108059] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T17:22:45.174911] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-04. [2025-09-16T17:22:45.414475] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:22:45.414808] [WARN ] [REMOTE ] BBT: No data returned for month 2025-04 [2025-09-16T17:22:45.974897] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-06. [2025-09-16T17:22:46.774906] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-06. [2025-09-16T17:22:47.178050] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T17:22:47.279506] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-09-16T17:22:47.574904] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-07. [2025-09-16T17:22:48.374906] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-05. [2025-09-16T17:22:48.482262] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-09-16T17:22:48.587400] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:22:48.587793] [WARN ] [REMOTE ] BBT: No data returned for month 2025-05 [2025-09-16T17:22:49.174895] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-06. [2025-09-16T17:22:49.449695] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:22:49.450019] [WARN ] [REMOTE ] BBT: No data returned for month 2025-06 [2025-09-16T17:22:49.974900] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-08. [2025-09-16T17:22:50.774937] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-07. [2025-09-16T17:22:50.854592] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-09-16T17:22:51.203635] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-09-16T17:22:51.574900] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-08. [2025-09-16T17:22:51.958811] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-09-16T17:22:52.374905] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-07. [2025-09-16T17:22:52.603204] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-07: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:22:52.603503] [WARN ] [REMOTE ] BBT: No data returned for month 2025-07 [2025-09-16T17:22:53.174915] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-09. [2025-09-16T17:22:53.974906] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-06. [2025-09-16T17:22:54.556799] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T17:22:54.561331] [INFO ] [REMOTE ] BBDC: Full fetch complete (36808 records). [2025-09-16T17:22:54.561440] [DEBUG] [VALIDATOR ] BBDC: Validating primary data. [2025-09-16T17:22:54.780092] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-09. [2025-09-16T17:22:54.953515] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T17:22:54.952599] [DEBUG] [VALIDATOR ] BBDC: All expected trading days are present. [2025-09-16T17:22:54.957377] [DEBUG] [VALIDATOR ] BBDC: Primary data validation passed. [2025-09-16T17:22:54.958210] [INFO ] [MAIN ] BBDC: Fetched primary data is valid. [2025-09-16T17:22:54.958278] [DEBUG] [LOCAL ] BBDC: Saving primary data. [2025-09-16T17:22:54.959443] [DEBUG] [LOCAL ] BBDC: Trimmed 36808 records to 23506 (keeping 150 days of data) [2025-09-16T17:22:55.103267] [DEBUG] [LOCAL ] BBDC: Primary data saved (23506 records). [2025-09-16T17:22:55.103402] [INFO ] [MAIN ] BBDC: Full history saved. [2025-09-16T17:22:55.103497] [INFO ] [MAIN ] Update Progress: 221/2369 symbols completed. BBDC took 30.4s. Estimated remaining: 3h 42m 15s. Rate: 9.7 symbols/min [2025-09-16T17:22:55.280196] [INFO ] [MAIN ] BBU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:22:55.280396] [DEBUG] [LOCAL ] BBU: No existing data file found in primary storage. [2025-09-16T17:22:55.280437] [WARN ] [MAIN ] BBU: No existing data found, performing full refresh instead. [2025-09-16T17:22:55.280599] [INFO ] [MAIN ] BBU: Wiping existing primary data before full refresh. [2025-09-16T17:22:55.280668] [INFO ] [LOCAL ] BBU: Starting deletion of primary data. [2025-09-16T17:22:55.280724] [DEBUG] [LOCAL ] BBU: No primary data file found to delete. [2025-09-16T17:22:55.280760] [INFO ] [MAIN ] BBU: Starting full primary refresh. [2025-09-16T17:22:55.391241] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-09-16T17:22:55.403223] [INFO ] [REMOTE ] BBDO: Full fetch complete (5511 records). [2025-09-16T17:22:55.418680] [DEBUG] [VALIDATOR ] BBDO: Validating primary data. [2025-09-16T17:22:55.591248] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-07. [2025-09-16T17:22:55.949998] [INFO ] [REMOTE ] BBU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:22:56.027358] [DEBUG] [VALIDATOR ] BBDO: All expected trading days are present. [2025-09-16T17:22:56.027438] [DEBUG] [VALIDATOR ] BBDO: Primary data validation passed. [2025-09-16T17:22:56.027593] [INFO ] [MAIN ] BBDO: Fetched primary data is valid. [2025-09-16T17:22:56.027634] [DEBUG] [LOCAL ] BBDO: Saving primary data. [2025-09-16T17:22:56.028056] [DEBUG] [LOCAL ] BBDO: Trimmed 5511 records to 3607 (keeping 150 days of data) [2025-09-16T17:22:56.050141] [DEBUG] [LOCAL ] BBDO: Primary data saved (3607 records). [2025-09-16T17:22:56.050274] [INFO ] [MAIN ] BBDO: Full history saved. [2025-09-16T17:22:56.050369] [INFO ] [MAIN ] Update Progress: 222/2369 symbols completed. BBDO took 23.1s. Estimated remaining: 3h 41m 18s. Rate: 9.7 symbols/min [2025-09-16T17:22:56.267025] [INFO ] [MAIN ] BBUC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:22:56.278153] [DEBUG] [LOCAL ] BBUC: Loaded 9248 existing records from primary storage. [2025-09-16T17:22:56.278275] [INFO ] [MAIN ] BBUC: Loaded 9248 existing records. [2025-09-16T17:22:56.379994] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-02. [2025-09-16T17:22:56.469216] [WARN ] [MAIN ] BBUC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:22:56.469604] [INFO ] [MAIN ] BBUC: Wiping existing primary data before full refresh. [2025-09-16T17:22:56.469686] [INFO ] [LOCAL ] BBUC: Starting deletion of primary data. [2025-09-16T17:22:56.470160] [DEBUG] [LOCAL ] BBUC: Deleted primary data file. [2025-09-16T17:22:56.470217] [INFO ] [MAIN ] BBUC: Starting full primary refresh. [2025-09-16T17:22:56.595139] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T17:22:56.860411] [INFO ] [REMOTE ] BBUC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:22:57.132051] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T17:22:57.174899] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-02. [2025-09-16T17:22:57.809093] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-09-16T17:22:57.974910] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-03. [2025-09-16T17:22:58.774922] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-08. [2025-09-16T17:22:58.798468] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-09-16T17:22:59.574940] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-08. [2025-09-16T17:23:00.374897] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-04. [2025-09-16T17:23:01.174901] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-03. [2025-09-16T17:23:03.463329] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-08: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:23:03.463635] [WARN ] [REMOTE ] BBT: No data returned for month 2025-08 [2025-09-16T17:23:03.464132] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-09. [2025-09-16T17:23:04.516080] [INFO ] [REMOTE ] BBT: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T17:23:04.521066] [INFO ] [REMOTE ] BBT: Full fetch complete (3381 records). [2025-09-16T17:23:04.521166] [DEBUG] [VALIDATOR ] BBT: Validating primary data. [2025-09-16T17:23:04.641760] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-03 (3.47 seconds). [2025-09-16T17:23:04.643687] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-04. [2025-09-16T17:23:04.730227] [WARN ] [VALIDATOR ] BBT: Validation failed - insufficient historical data. Have 11 unique days, need 150 days [2025-09-16T17:23:04.731098] [FATAL] [MAIN ] BBT: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:23:04.742441] [WARN ] [MAIN ] BBT: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:23:04.743030] [INFO ] [LOCAL ] BBT: Starting deletion of primary data. [2025-09-16T17:23:04.743466] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-08 (5.97 seconds). [2025-09-16T17:23:04.747699] [DEBUG] [LOCAL ] BBT: No primary data file found to delete. [2025-09-16T17:23:04.748149] [INFO ] [LOCAL ] BBT: Starting deletion of derivative data. [2025-09-16T17:23:04.748422] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-09. [2025-09-16T17:23:04.748599] [DEBUG] [LOCAL ] BBT: No derivative data file found to delete. [2025-09-16T17:23:04.749496] [INFO ] [MAIN ] Update Progress: 223/2369 symbols completed. BBT took 26.7s. Estimated remaining: 3h 41m 36s. Rate: 9.7 symbols/min [2025-09-16T17:23:04.929750] [INFO ] [MAIN ] BBVA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:04.968369] [DEBUG] [LOCAL ] BBVA: Loaded 36011 existing records from primary storage. [2025-09-16T17:23:04.968511] [INFO ] [MAIN ] BBVA: Loaded 36011 existing records. [2025-09-16T17:23:05.158627] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-09-16T17:23:05.163931] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-05. [2025-09-16T17:23:05.164121] [WARN ] [MAIN ] BBVA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:23:05.165348] [INFO ] [MAIN ] BBVA: Wiping existing primary data before full refresh. [2025-09-16T17:23:05.173125] [INFO ] [LOCAL ] BBVA: Starting deletion of primary data. [2025-09-16T17:23:05.175403] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-04 (4.80 seconds). [2025-09-16T17:23:05.178770] [DEBUG] [LOCAL ] BBVA: Deleted primary data file. [2025-09-16T17:23:05.179345] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-05. [2025-09-16T17:23:05.179403] [INFO ] [MAIN ] BBVA: Starting full primary refresh. [2025-09-16T17:23:05.541197] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-09-16T17:23:05.546122] [INFO ] [REMOTE ] BBVA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:05.620052] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-09-16T17:23:05.622735] [INFO ] [REMOTE ] BBN: Full fetch complete (19511 records). [2025-09-16T17:23:05.622813] [DEBUG] [VALIDATOR ] BBN: Validating primary data. [2025-09-16T17:23:05.983768] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-02. [2025-09-16T17:23:05.994020] [DEBUG] [VALIDATOR ] BBN: All expected trading days are present. [2025-09-16T17:23:06.002008] [DEBUG] [VALIDATOR ] BBN: Primary data validation passed. [2025-09-16T17:23:06.002371] [INFO ] [MAIN ] BBN: Fetched primary data is valid. [2025-09-16T17:23:06.002849] [DEBUG] [LOCAL ] BBN: Saving primary data. [2025-09-16T17:23:06.003111] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T17:23:06.007241] [DEBUG] [LOCAL ] BBN: Trimmed 19511 records to 13520 (keeping 150 days of data) [2025-09-16T17:23:06.088978] [DEBUG] [LOCAL ] BBN: Primary data saved (13520 records). [2025-09-16T17:23:06.089111] [INFO ] [MAIN ] BBN: Full history saved. [2025-09-16T17:23:06.089195] [INFO ] [MAIN ] Update Progress: 224/2369 symbols completed. BBN took 29.1s. Estimated remaining: 3h 40m 43s. Rate: 9.7 symbols/min [2025-09-16T17:23:06.262994] [INFO ] [MAIN ] BBW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:06.280708] [DEBUG] [LOCAL ] BBW: Loaded 18611 existing records from primary storage. [2025-09-16T17:23:06.280820] [INFO ] [MAIN ] BBW: Loaded 18611 existing records. [2025-09-16T17:23:06.454002] [WARN ] [MAIN ] BBW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:23:06.454243] [INFO ] [MAIN ] BBW: Wiping existing primary data before full refresh. [2025-09-16T17:23:06.454288] [INFO ] [LOCAL ] BBW: Starting deletion of primary data. [2025-09-16T17:23:06.454907] [DEBUG] [LOCAL ] BBW: Deleted primary data file. [2025-09-16T17:23:06.454955] [INFO ] [MAIN ] BBW: Starting full primary refresh. [2025-09-16T17:23:06.783809] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-06. [2025-09-16T17:23:06.802480] [INFO ] [REMOTE ] BBW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:07.294786] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-09-16T17:23:07.574897] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-02. [2025-09-16T17:23:07.896230] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-09-16T17:23:08.374869] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-07. [2025-09-16T17:23:08.786383] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-09-16T17:23:09.165480] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T17:23:09.174846] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-03. [2025-09-16T17:23:09.974898] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-06. [2025-09-16T17:23:10.544045] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-09-16T17:23:10.774907] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-03. [2025-09-16T17:23:10.912185] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T17:23:11.574912] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-08. [2025-09-16T17:23:11.920665] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T17:23:12.112038] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-09-16T17:23:12.374924] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-07. [2025-09-16T17:23:13.109005] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-09-16T17:23:13.174927] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-08. [2025-09-16T17:23:13.795866] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-09-16T17:23:13.974931] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-09. [2025-09-16T17:23:14.539838] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T17:23:14.541688] [INFO ] [REMOTE ] BBUC: Full fetch complete (14114 records). [2025-09-16T17:23:14.541759] [DEBUG] [VALIDATOR ] BBUC: Validating primary data. [2025-09-16T17:23:14.779988] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-04. [2025-09-16T17:23:14.914264] [DEBUG] [VALIDATOR ] BBUC: All expected trading days are present. [2025-09-16T17:23:14.914341] [DEBUG] [VALIDATOR ] BBUC: Primary data validation passed. [2025-09-16T17:23:14.914608] [INFO ] [MAIN ] BBUC: Fetched primary data is valid. [2025-09-16T17:23:14.914668] [DEBUG] [LOCAL ] BBUC: Saving primary data. [2025-09-16T17:23:14.915157] [DEBUG] [LOCAL ] BBUC: Trimmed 14114 records to 9313 (keeping 150 days of data) [2025-09-16T17:23:14.974037] [DEBUG] [LOCAL ] BBUC: Primary data saved (9313 records). [2025-09-16T17:23:14.974193] [INFO ] [MAIN ] BBUC: Full history saved. [2025-09-16T17:23:14.974281] [INFO ] [MAIN ] Update Progress: 225/2369 symbols completed. BBUC took 18.9s. Estimated remaining: 3h 41m 3s. Rate: 9.7 symbols/min [2025-09-16T17:23:15.146860] [INFO ] [MAIN ] BBWI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:15.186024] [DEBUG] [LOCAL ] BBWI: Loaded 39551 existing records from primary storage. [2025-09-16T17:23:15.186198] [INFO ] [MAIN ] BBWI: Loaded 39551 existing records. [2025-09-16T17:23:15.362915] [WARN ] [MAIN ] BBWI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:23:15.363146] [INFO ] [MAIN ] BBWI: Wiping existing primary data before full refresh. [2025-09-16T17:23:15.363194] [INFO ] [LOCAL ] BBWI: Starting deletion of primary data. [2025-09-16T17:23:15.364322] [DEBUG] [LOCAL ] BBWI: Deleted primary data file. [2025-09-16T17:23:15.364376] [INFO ] [MAIN ] BBWI: Starting full primary refresh. [2025-09-16T17:23:15.580044] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-04. [2025-09-16T17:23:15.726540] [INFO ] [REMOTE ] BBWI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:16.125041] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T17:23:16.374912] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-09. [2025-09-16T17:23:16.859579] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-09-16T17:23:16.860850] [INFO ] [REMOTE ] BBU: Full fetch complete (4407 records). [2025-09-16T17:23:16.860921] [DEBUG] [VALIDATOR ] BBU: Validating primary data. [2025-09-16T17:23:17.186074] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-02. [2025-09-16T17:23:17.223824] [DEBUG] [VALIDATOR ] BBU: All expected trading days are present. [2025-09-16T17:23:17.223989] [DEBUG] [VALIDATOR ] BBU: Primary data validation passed. [2025-09-16T17:23:17.224235] [INFO ] [MAIN ] BBU: Fetched primary data is valid. [2025-09-16T17:23:17.224413] [DEBUG] [LOCAL ] BBU: Saving primary data. [2025-09-16T17:23:17.225031] [DEBUG] [LOCAL ] BBU: Trimmed 4407 records to 3205 (keeping 150 days of data) [2025-09-16T17:23:17.272071] [DEBUG] [LOCAL ] BBU: Primary data saved (3205 records). [2025-09-16T17:23:17.272230] [INFO ] [MAIN ] BBU: Full history saved. [2025-09-16T17:23:17.272395] [INFO ] [MAIN ] Update Progress: 226/2369 symbols completed. BBU took 22.2s. Estimated remaining: 3h 40m 20s. Rate: 9.7 symbols/min [2025-09-16T17:23:17.273965] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T17:23:17.459673] [INFO ] [MAIN ] BBY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:17.459879] [DEBUG] [LOCAL ] BBY: No existing data file found in primary storage. [2025-09-16T17:23:17.459920] [WARN ] [MAIN ] BBY: No existing data found, performing full refresh instead. [2025-09-16T17:23:17.460044] [INFO ] [MAIN ] BBY: Wiping existing primary data before full refresh. [2025-09-16T17:23:17.460084] [INFO ] [LOCAL ] BBY: Starting deletion of primary data. [2025-09-16T17:23:17.460128] [DEBUG] [LOCAL ] BBY: No primary data file found to delete. [2025-09-16T17:23:17.460164] [INFO ] [MAIN ] BBY: Starting full primary refresh. [2025-09-16T17:23:17.826572] [INFO ] [REMOTE ] BBY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:17.974871] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-02. [2025-09-16T17:23:18.774919] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-05. [2025-09-16T17:23:18.979229] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-09-16T17:23:19.574951] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-05. [2025-09-16T17:23:19.855933] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-09-16T17:23:20.374927] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-03. [2025-09-16T17:23:20.536817] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T17:23:20.590703] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-09-16T17:23:21.174973] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-06. [2025-09-16T17:23:21.786104] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-09-16T17:23:21.974982] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-06. [2025-09-16T17:23:22.744918] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T17:23:22.774913] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-07. [2025-09-16T17:23:22.827962] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T17:23:23.574904] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-07. [2025-09-16T17:23:23.895473] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T17:23:24.374909] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-04. [2025-09-16T17:23:25.174955] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-08. [2025-09-16T17:23:25.288389] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T17:23:25.728022] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T17:23:25.974916] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-05. [2025-09-16T17:23:26.024292] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-09-16T17:23:26.774960] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-09. [2025-09-16T17:23:27.434902] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T17:23:27.574959] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-08. [2025-09-16T17:23:28.248194] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T17:23:28.252620] [INFO ] [REMOTE ] BBW: Full fetch complete (28546 records). [2025-09-16T17:23:28.252783] [DEBUG] [VALIDATOR ] BBW: Validating primary data. [2025-09-16T17:23:28.379995] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-06. [2025-09-16T17:23:28.618125] [DEBUG] [VALIDATOR ] BBW: All expected trading days are present. [2025-09-16T17:23:28.618200] [DEBUG] [VALIDATOR ] BBW: Primary data validation passed. [2025-09-16T17:23:28.618569] [INFO ] [MAIN ] BBW: Fetched primary data is valid. [2025-09-16T17:23:28.618610] [DEBUG] [LOCAL ] BBW: Saving primary data. [2025-09-16T17:23:28.619272] [DEBUG] [LOCAL ] BBW: Trimmed 28546 records to 18907 (keeping 150 days of data) [2025-09-16T17:23:28.731159] [DEBUG] [LOCAL ] BBW: Primary data saved (18907 records). [2025-09-16T17:23:28.731294] [INFO ] [MAIN ] BBW: Full history saved. [2025-09-16T17:23:28.731375] [INFO ] [MAIN ] Update Progress: 227/2369 symbols completed. BBW took 22.6s. Estimated remaining: 3h 41m 4s. Rate: 9.7 symbols/min [2025-09-16T17:23:28.904447] [INFO ] [MAIN ] BC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:28.940105] [DEBUG] [LOCAL ] BC: Loaded 32606 existing records from primary storage. [2025-09-16T17:23:28.940241] [INFO ] [MAIN ] BC: Loaded 32606 existing records. [2025-09-16T17:23:29.116558] [WARN ] [MAIN ] BC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:23:29.116818] [INFO ] [MAIN ] BC: Wiping existing primary data before full refresh. [2025-09-16T17:23:29.116865] [INFO ] [LOCAL ] BC: Starting deletion of primary data. [2025-09-16T17:23:29.117790] [DEBUG] [LOCAL ] BC: Deleted primary data file. [2025-09-16T17:23:29.117841] [INFO ] [MAIN ] BC: Starting full primary refresh. [2025-09-16T17:23:29.180136] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-03. [2025-09-16T17:23:29.474656] [INFO ] [REMOTE ] BC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:29.503526] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T17:23:29.974905] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-09. [2025-09-16T17:23:30.307491] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-09-16T17:23:30.774881] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-07. [2025-09-16T17:23:30.989623] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T17:23:31.317236] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T17:23:31.322945] [INFO ] [REMOTE ] BBVA: Full fetch complete (53901 records). [2025-09-16T17:23:31.323036] [DEBUG] [VALIDATOR ] BBVA: Validating primary data. [2025-09-16T17:23:31.580076] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-04. [2025-09-16T17:23:31.703741] [DEBUG] [VALIDATOR ] BBVA: All expected trading days are present. [2025-09-16T17:23:31.703820] [DEBUG] [VALIDATOR ] BBVA: Primary data validation passed. [2025-09-16T17:23:31.704382] [INFO ] [MAIN ] BBVA: Fetched primary data is valid. [2025-09-16T17:23:31.704424] [DEBUG] [LOCAL ] BBVA: Saving primary data. [2025-09-16T17:23:31.705449] [DEBUG] [LOCAL ] BBVA: Trimmed 53901 records to 36371 (keeping 150 days of data) [2025-09-16T17:23:31.926793] [DEBUG] [LOCAL ] BBVA: Primary data saved (36371 records). [2025-09-16T17:23:31.927065] [INFO ] [MAIN ] BBVA: Full history saved. [2025-09-16T17:23:31.927160] [INFO ] [MAIN ] Update Progress: 228/2369 symbols completed. BBVA took 27.2s. Estimated remaining: 3h 40m 30s. Rate: 9.7 symbols/min [2025-09-16T17:23:32.097542] [INFO ] [MAIN ] BCAT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:32.121358] [DEBUG] [LOCAL ] BCAT: Loaded 25058 existing records from primary storage. [2025-09-16T17:23:32.121474] [INFO ] [MAIN ] BCAT: Loaded 25058 existing records. [2025-09-16T17:23:32.296425] [WARN ] [MAIN ] BCAT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:23:32.296680] [INFO ] [MAIN ] BCAT: Wiping existing primary data before full refresh. [2025-09-16T17:23:32.296724] [INFO ] [LOCAL ] BCAT: Starting deletion of primary data. [2025-09-16T17:23:32.297601] [DEBUG] [LOCAL ] BCAT: Deleted primary data file. [2025-09-16T17:23:32.297672] [INFO ] [MAIN ] BCAT: Starting full primary refresh. [2025-09-16T17:23:32.387457] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-02. [2025-09-16T17:23:32.658427] [INFO ] [REMOTE ] BCAT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:32.693559] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T17:23:33.074015] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T17:23:33.174859] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-05. [2025-09-16T17:23:33.519982] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-09-16T17:23:33.974896] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-08. [2025-09-16T17:23:34.774882] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-03. [2025-09-16T17:23:34.809961] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T17:23:35.567493] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T17:23:35.577817] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-02. [2025-09-16T17:23:36.032436] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T17:23:36.374916] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-04. [2025-09-16T17:23:36.781971] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-09-16T17:23:37.174905] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-03. [2025-09-16T17:23:37.974930] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-09. [2025-09-16T17:23:37.974867] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T17:23:38.249999] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T17:23:38.774896] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-04. [2025-09-16T17:23:39.577391] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-05. [2025-09-16T17:23:39.579336] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T17:23:39.585947] [INFO ] [REMOTE ] BBWI: Full fetch complete (58231 records). [2025-09-16T17:23:39.586050] [DEBUG] [VALIDATOR ] BBWI: Validating primary data. [2025-09-16T17:23:40.006206] [DEBUG] [VALIDATOR ] BBWI: All expected trading days are present. [2025-09-16T17:23:40.006424] [DEBUG] [VALIDATOR ] BBWI: Primary data validation passed. [2025-09-16T17:23:40.008183] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T17:23:40.007440] [INFO ] [MAIN ] BBWI: Fetched primary data is valid. [2025-09-16T17:23:40.008310] [DEBUG] [LOCAL ] BBWI: Saving primary data. [2025-09-16T17:23:40.016522] [DEBUG] [LOCAL ] BBWI: Trimmed 58231 records to 39941 (keeping 150 days of data) [2025-09-16T17:23:40.259766] [DEBUG] [LOCAL ] BBWI: Primary data saved (39941 records). [2025-09-16T17:23:40.259901] [INFO ] [MAIN ] BBWI: Full history saved. [2025-09-16T17:23:40.261545] [INFO ] [MAIN ] Update Progress: 229/2369 symbols completed. BBWI took 25.3s. Estimated remaining: 3h 40m 43s. Rate: 9.7 symbols/min [2025-09-16T17:23:40.380028] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-05. [2025-09-16T17:23:40.440403] [INFO ] [MAIN ] BCC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:40.461211] [DEBUG] [LOCAL ] BCC: Loaded 20178 existing records from primary storage. [2025-09-16T17:23:40.461337] [INFO ] [MAIN ] BCC: Loaded 20178 existing records. [2025-09-16T17:23:40.637296] [WARN ] [MAIN ] BCC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:23:40.637523] [INFO ] [MAIN ] BCC: Wiping existing primary data before full refresh. [2025-09-16T17:23:40.637568] [INFO ] [LOCAL ] BCC: Starting deletion of primary data. [2025-09-16T17:23:40.638150] [DEBUG] [LOCAL ] BCC: Deleted primary data file. [2025-09-16T17:23:40.638197] [INFO ] [MAIN ] BCC: Starting full primary refresh. [2025-09-16T17:23:40.991918] [INFO ] [REMOTE ] BCC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:41.174860] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-02. [2025-09-16T17:23:41.220135] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T17:23:41.637500] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-09-16T17:23:41.974914] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-06. [2025-09-16T17:23:42.490236] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-09-16T17:23:42.774984] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-06. [2025-09-16T17:23:43.228447] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T17:23:43.574969] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-03. [2025-09-16T17:23:44.210404] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T17:23:44.374876] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-07. [2025-09-16T17:23:44.834394] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T17:23:45.174916] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-06. [2025-09-16T17:23:45.880794] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T17:23:45.974939] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-07. [2025-09-16T17:23:46.774922] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-04. [2025-09-16T17:23:47.220401] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-06 (2.05 seconds). [2025-09-16T17:23:47.475815] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T17:23:47.567617] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-09-16T17:23:47.574846] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-05. [2025-09-16T17:23:48.374919] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-08. [2025-09-16T17:23:48.675207] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-09-16T17:23:49.174954] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-07. [2025-09-16T17:23:49.402816] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T17:23:49.974920] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-09. [2025-09-16T17:23:50.774911] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-06. [2025-09-16T17:23:50.807241] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-09-16T17:23:50.811723] [INFO ] [REMOTE ] BCAT: Full fetch complete (37561 records). [2025-09-16T17:23:50.811844] [DEBUG] [VALIDATOR ] BCAT: Validating primary data. [2025-09-16T17:23:51.209812] [DEBUG] [VALIDATOR ] BCAT: All expected trading days are present. [2025-09-16T17:23:51.209915] [DEBUG] [VALIDATOR ] BCAT: Primary data validation passed. [2025-09-16T17:23:51.210542] [INFO ] [MAIN ] BCAT: Fetched primary data is valid. [2025-09-16T17:23:51.210611] [DEBUG] [LOCAL ] BCAT: Saving primary data. [2025-09-16T17:23:51.211624] [DEBUG] [LOCAL ] BCAT: Trimmed 37561 records to 25335 (keeping 150 days of data) [2025-09-16T17:23:51.332949] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-09-16T17:23:51.416295] [DEBUG] [LOCAL ] BCAT: Primary data saved (25335 records). [2025-09-16T17:23:51.416453] [INFO ] [MAIN ] BCAT: Full history saved. [2025-09-16T17:23:51.416546] [INFO ] [MAIN ] Update Progress: 230/2369 symbols completed. BCAT took 19.5s. Estimated remaining: 3h 41m 23s. Rate: 9.7 symbols/min [2025-09-16T17:23:51.584055] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-08. [2025-09-16T17:23:51.614143] [INFO ] [MAIN ] BCE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:51.653025] [DEBUG] [LOCAL ] BCE: Loaded 38550 existing records from primary storage. [2025-09-16T17:23:51.653164] [INFO ] [MAIN ] BCE: Loaded 38550 existing records. [2025-09-16T17:23:51.832149] [WARN ] [MAIN ] BCE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:23:51.832553] [INFO ] [MAIN ] BCE: Wiping existing primary data before full refresh. [2025-09-16T17:23:51.832754] [INFO ] [LOCAL ] BCE: Starting deletion of primary data. [2025-09-16T17:23:51.838857] [DEBUG] [LOCAL ] BCE: Deleted primary data file. [2025-09-16T17:23:51.847171] [INFO ] [MAIN ] BCE: Starting full primary refresh. [2025-09-16T17:23:51.880920] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-09-16T17:23:52.213357] [INFO ] [REMOTE ] BCE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:52.374889] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-02. [2025-09-16T17:23:53.174950] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-07. [2025-09-16T17:23:53.477566] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-09-16T17:23:53.974871] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-08. [2025-09-16T17:23:54.002928] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T17:23:54.138671] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-09-16T17:23:54.774902] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-09. [2025-09-16T17:23:55.311690] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T17:23:55.574905] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-09. [2025-09-16T17:23:56.046980] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T17:23:56.053395] [INFO ] [REMOTE ] BBY: Full fetch complete (58084 records). [2025-09-16T17:23:56.053503] [DEBUG] [VALIDATOR ] BBY: Validating primary data. [2025-09-16T17:23:56.379988] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-08. [2025-09-16T17:23:56.454207] [DEBUG] [VALIDATOR ] BBY: All expected trading days are present. [2025-09-16T17:23:56.454292] [DEBUG] [VALIDATOR ] BBY: Primary data validation passed. [2025-09-16T17:23:56.455144] [INFO ] [MAIN ] BBY: Fetched primary data is valid. [2025-09-16T17:23:56.455198] [DEBUG] [LOCAL ] BBY: Saving primary data. [2025-09-16T17:23:56.456624] [DEBUG] [LOCAL ] BBY: Trimmed 58084 records to 39860 (keeping 150 days of data) [2025-09-16T17:23:56.701071] [DEBUG] [LOCAL ] BBY: Primary data saved (39860 records). [2025-09-16T17:23:56.701231] [INFO ] [MAIN ] BBY: Full history saved. [2025-09-16T17:23:56.701330] [INFO ] [MAIN ] Update Progress: 231/2369 symbols completed. BBY took 39.4s. Estimated remaining: 3h 41m 9s. Rate: 9.7 symbols/min [2025-09-16T17:23:56.877782] [INFO ] [MAIN ] BCH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:56.891858] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T17:23:56.897084] [INFO ] [REMOTE ] BC: Full fetch complete (48513 records). [2025-09-16T17:23:56.897222] [DEBUG] [VALIDATOR ] BC: Validating primary data. [2025-09-16T17:23:57.079106] [DEBUG] [LOCAL ] BCH: Loaded 21393 existing records from primary storage. [2025-09-16T17:23:57.085342] [INFO ] [MAIN ] BCH: Loaded 21393 existing records. [2025-09-16T17:23:57.179966] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-03. [2025-09-16T17:23:57.434209] [WARN ] [MAIN ] BCH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:23:57.511874] [INFO ] [MAIN ] BCH: Wiping existing primary data before full refresh. [2025-09-16T17:23:57.514395] [DEBUG] [VALIDATOR ] BC: All expected trading days are present. [2025-09-16T17:23:57.514484] [DEBUG] [VALIDATOR ] BC: Primary data validation passed. [2025-09-16T17:23:57.515243] [INFO ] [LOCAL ] BCH: Starting deletion of primary data. [2025-09-16T17:23:57.515198] [INFO ] [MAIN ] BC: Fetched primary data is valid. [2025-09-16T17:23:57.515342] [DEBUG] [LOCAL ] BC: Saving primary data. [2025-09-16T17:23:57.516086] [DEBUG] [LOCAL ] BCH: Deleted primary data file. [2025-09-16T17:23:57.516171] [INFO ] [MAIN ] BCH: Starting full primary refresh. [2025-09-16T17:23:57.598678] [DEBUG] [LOCAL ] BC: Trimmed 48513 records to 32937 (keeping 150 days of data) [2025-09-16T17:23:57.638538] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-09-16T17:23:57.979616] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-09. [2025-09-16T17:23:58.003909] [INFO ] [REMOTE ] BCH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:58.113510] [DEBUG] [LOCAL ] BC: Primary data saved (32937 records). [2025-09-16T17:23:58.113680] [INFO ] [MAIN ] BC: Full history saved. [2025-09-16T17:23:58.114490] [INFO ] [MAIN ] Update Progress: 232/2369 symbols completed. BC took 29.4s. Estimated remaining: 3h 40m 18s. Rate: 9.7 symbols/min [2025-09-16T17:23:58.284748] [INFO ] [MAIN ] BCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:58.305357] [DEBUG] [LOCAL ] BCO: Loaded 19118 existing records from primary storage. [2025-09-16T17:23:58.305503] [INFO ] [MAIN ] BCO: Loaded 19118 existing records. [2025-09-16T17:23:58.487183] [WARN ] [MAIN ] BCO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:23:58.487430] [INFO ] [MAIN ] BCO: Wiping existing primary data before full refresh. [2025-09-16T17:23:58.487492] [INFO ] [LOCAL ] BCO: Starting deletion of primary data. [2025-09-16T17:23:58.488064] [DEBUG] [LOCAL ] BCO: Deleted primary data file. [2025-09-16T17:23:58.488123] [INFO ] [MAIN ] BCO: Starting full primary refresh. [2025-09-16T17:23:58.779936] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-02. [2025-09-16T17:23:58.852219] [INFO ] [REMOTE ] BCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:58.931837] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T17:23:58.996383] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T17:23:58.999687] [INFO ] [REMOTE ] BCC: Full fetch complete (30866 records). [2025-09-16T17:23:58.999769] [DEBUG] [VALIDATOR ] BCC: Validating primary data. [2025-09-16T17:23:59.000292] [FATAL] [VALIDATOR ] BCC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:23:59.000370] [FATAL] [MAIN ] BCC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:23:59.000406] [WARN ] [MAIN ] BCC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:23:59.000445] [INFO ] [LOCAL ] BCC: Starting deletion of primary data. [2025-09-16T17:23:59.000503] [DEBUG] [LOCAL ] BCC: No primary data file found to delete. [2025-09-16T17:23:59.000549] [INFO ] [LOCAL ] BCC: Starting deletion of derivative data. [2025-09-16T17:23:59.000924] [DEBUG] [LOCAL ] BCC: Deleted derivative data file. [2025-09-16T17:23:59.001006] [INFO ] [MAIN ] Update Progress: 233/2369 symbols completed. BCC took 18.7s. Estimated remaining: 3h 39m 24s. Rate: 9.7 symbols/min [2025-09-16T17:23:59.172996] [INFO ] [MAIN ] BCS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:23:59.173176] [DEBUG] [LOCAL ] BCS: No existing data file found in primary storage. [2025-09-16T17:23:59.173213] [WARN ] [MAIN ] BCS: No existing data found, performing full refresh instead. [2025-09-16T17:23:59.173329] [INFO ] [MAIN ] BCS: Wiping existing primary data before full refresh. [2025-09-16T17:23:59.173373] [INFO ] [LOCAL ] BCS: Starting deletion of primary data. [2025-09-16T17:23:59.173414] [DEBUG] [LOCAL ] BCS: No primary data file found to delete. [2025-09-16T17:23:59.173447] [INFO ] [MAIN ] BCS: Starting full primary refresh. [2025-09-16T17:23:59.525708] [INFO ] [REMOTE ] BCS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:23:59.574853] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-02. [2025-09-16T17:24:00.080388] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-09-16T17:24:00.374892] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-03. [2025-09-16T17:24:01.174900] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-02. [2025-09-16T17:24:01.329620] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T17:24:01.821384] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T17:24:01.947052] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-09-16T17:24:01.974844] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-03. [2025-09-16T17:24:02.774917] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-03. [2025-09-16T17:24:03.242187] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T17:24:03.574965] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-04. [2025-09-16T17:24:04.212353] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T17:24:04.374907] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-04. [2025-09-16T17:24:04.834786] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-09-16T17:24:05.174942] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-04. [2025-09-16T17:24:05.662329] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-09-16T17:24:05.974903] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-04. [2025-09-16T17:24:06.571117] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T17:24:06.774932] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-05. [2025-09-16T17:24:07.349757] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T17:24:07.574347] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-09-16T17:24:07.580222] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-05. [2025-09-16T17:24:08.374906] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-06. [2025-09-16T17:24:09.174906] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-05. [2025-09-16T17:24:09.459417] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-09-16T17:24:09.516601] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T17:24:09.974903] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-07. [2025-09-16T17:24:10.183761] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T17:24:10.774908] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-06. [2025-09-16T17:24:11.455467] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T17:24:11.574911] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-06. [2025-09-16T17:24:12.034472] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T17:24:12.374912] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-07. [2025-09-16T17:24:13.174889] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-05. [2025-09-16T17:24:13.231819] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-09-16T17:24:13.395222] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-09-16T17:24:13.974968] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-08. [2025-09-16T17:24:14.725507] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-09-16T17:24:14.765350] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T17:24:14.774926] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-08. [2025-09-16T17:24:15.444263] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-09-16T17:24:15.574907] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-06. [2025-09-16T17:24:16.374921] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-07. [2025-09-16T17:24:17.174925] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-09. [2025-09-16T17:24:17.524692] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-09-16T17:24:17.920376] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-09-16T17:24:17.971476] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T17:24:17.974782] [INFO ] [REMOTE ] BCO: Full fetch complete (28711 records). [2025-09-16T17:24:17.974928] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-09. [2025-09-16T17:24:17.976297] [DEBUG] [VALIDATOR ] BCO: Validating primary data. [2025-09-16T17:24:18.362714] [DEBUG] [VALIDATOR ] BCO: All expected trading days are present. [2025-09-16T17:24:18.362813] [DEBUG] [VALIDATOR ] BCO: Primary data validation passed. [2025-09-16T17:24:18.363278] [INFO ] [MAIN ] BCO: Fetched primary data is valid. [2025-09-16T17:24:18.363338] [DEBUG] [LOCAL ] BCO: Saving primary data. [2025-09-16T17:24:18.364441] [DEBUG] [LOCAL ] BCO: Trimmed 28711 records to 19242 (keeping 150 days of data) [2025-09-16T17:24:18.495404] [DEBUG] [LOCAL ] BCO: Primary data saved (19242 records). [2025-09-16T17:24:18.495553] [INFO ] [MAIN ] BCO: Full history saved. [2025-09-16T17:24:18.495664] [INFO ] [MAIN ] Update Progress: 234/2369 symbols completed. BCO took 20.4s. Estimated remaining: 3h 41m 19s. Rate: 9.6 symbols/min [2025-09-16T17:24:18.664496] [INFO ] [MAIN ] BCSF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:24:18.683502] [DEBUG] [LOCAL ] BCSF: Loaded 19270 existing records from primary storage. [2025-09-16T17:24:18.683610] [INFO ] [MAIN ] BCSF: Loaded 19270 existing records. [2025-09-16T17:24:18.781894] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-08. [2025-09-16T17:24:18.862418] [WARN ] [MAIN ] BCSF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:24:18.862677] [INFO ] [MAIN ] BCSF: Wiping existing primary data before full refresh. [2025-09-16T17:24:18.862724] [INFO ] [LOCAL ] BCSF: Starting deletion of primary data. [2025-09-16T17:24:18.863265] [DEBUG] [LOCAL ] BCSF: Deleted primary data file. [2025-09-16T17:24:18.863311] [INFO ] [MAIN ] BCSF: Starting full primary refresh. [2025-09-16T17:24:19.218401] [INFO ] [REMOTE ] BCSF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:24:19.318621] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T17:24:19.322677] [INFO ] [REMOTE ] BCH: Full fetch complete (32024 records). [2025-09-16T17:24:19.322763] [DEBUG] [VALIDATOR ] BCH: Validating primary data. [2025-09-16T17:24:19.580122] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-07. [2025-09-16T17:24:19.691502] [DEBUG] [VALIDATOR ] BCH: All expected trading days are present. [2025-09-16T17:24:19.691587] [DEBUG] [VALIDATOR ] BCH: Primary data validation passed. [2025-09-16T17:24:19.692031] [INFO ] [MAIN ] BCH: Fetched primary data is valid. [2025-09-16T17:24:19.692075] [DEBUG] [LOCAL ] BCH: Saving primary data. [2025-09-16T17:24:19.692922] [DEBUG] [LOCAL ] BCH: Trimmed 32024 records to 21687 (keeping 150 days of data) [2025-09-16T17:24:19.818551] [DEBUG] [LOCAL ] BCH: Primary data saved (21687 records). [2025-09-16T17:24:19.818694] [INFO ] [MAIN ] BCH: Full history saved. [2025-09-16T17:24:19.818784] [INFO ] [MAIN ] Update Progress: 235/2369 symbols completed. BCH took 23.1s. Estimated remaining: 3h 40m 28s. Rate: 9.7 symbols/min [2025-09-16T17:24:19.986507] [INFO ] [MAIN ] BCX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:24:20.001573] [DEBUG] [LOCAL ] BCX: Loaded 15307 existing records from primary storage. [2025-09-16T17:24:20.001713] [INFO ] [MAIN ] BCX: Loaded 15307 existing records. [2025-09-16T17:24:20.182696] [WARN ] [MAIN ] BCX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:24:20.182970] [INFO ] [MAIN ] BCX: Wiping existing primary data before full refresh. [2025-09-16T17:24:20.183032] [INFO ] [LOCAL ] BCX: Starting deletion of primary data. [2025-09-16T17:24:20.183459] [DEBUG] [LOCAL ] BCX: Deleted primary data file. [2025-09-16T17:24:20.183503] [INFO ] [MAIN ] BCX: Starting full primary refresh. [2025-09-16T17:24:20.390060] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-02. [2025-09-16T17:24:20.540383] [INFO ] [REMOTE ] BCX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:24:20.586033] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-09-16T17:24:20.949388] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T17:24:21.174917] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-08. [2025-09-16T17:24:21.601549] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-09-16T17:24:21.974953] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-02. [2025-09-16T17:24:22.774874] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-09. [2025-09-16T17:24:22.824252] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T17:24:22.920934] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-09-16T17:24:23.574902] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-09. [2025-09-16T17:24:24.173833] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:24:24.179669] [INFO ] [REMOTE ] BCS: Full fetch complete (57755 records). [2025-09-16T17:24:24.179765] [DEBUG] [VALIDATOR ] BCS: Validating primary data. [2025-09-16T17:24:24.180427] [FATAL] [VALIDATOR ] BCS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:24:24.180504] [FATAL] [MAIN ] BCS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:24:24.180565] [WARN ] [MAIN ] BCS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:24:24.180612] [INFO ] [LOCAL ] BCS: Starting deletion of primary data. [2025-09-16T17:24:24.180703] [DEBUG] [LOCAL ] BCS: No primary data file found to delete. [2025-09-16T17:24:24.180741] [INFO ] [LOCAL ] BCS: Starting deletion of derivative data. [2025-09-16T17:24:24.180790] [DEBUG] [LOCAL ] BCS: No derivative data file found to delete. [2025-09-16T17:24:24.180854] [INFO ] [MAIN ] Update Progress: 236/2369 symbols completed. BCS took 25.2s. Estimated remaining: 3h 40m 6s. Rate: 9.7 symbols/min [2025-09-16T17:24:24.349453] [INFO ] [MAIN ] BDC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:24:24.367501] [DEBUG] [LOCAL ] BDC: Loaded 17657 existing records from primary storage. [2025-09-16T17:24:24.367611] [INFO ] [MAIN ] BDC: Loaded 17657 existing records. [2025-09-16T17:24:24.380157] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-03. [2025-09-16T17:24:24.544565] [WARN ] [MAIN ] BDC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:24:24.544822] [INFO ] [MAIN ] BDC: Wiping existing primary data before full refresh. [2025-09-16T17:24:24.544867] [INFO ] [LOCAL ] BDC: Starting deletion of primary data. [2025-09-16T17:24:24.545506] [DEBUG] [LOCAL ] BDC: Deleted primary data file. [2025-09-16T17:24:24.545553] [INFO ] [MAIN ] BDC: Starting full primary refresh. [2025-09-16T17:24:24.901746] [INFO ] [REMOTE ] BDC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:24:25.174912] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-02. [2025-09-16T17:24:25.192250] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T17:24:25.198026] [INFO ] [REMOTE ] BCE: Full fetch complete (56612 records). [2025-09-16T17:24:25.198156] [DEBUG] [VALIDATOR ] BCE: Validating primary data. [2025-09-16T17:24:25.198791] [FATAL] [VALIDATOR ] BCE: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:24:25.198869] [FATAL] [MAIN ] BCE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:24:25.198906] [WARN ] [MAIN ] BCE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:24:25.198965] [INFO ] [LOCAL ] BCE: Starting deletion of primary data. [2025-09-16T17:24:25.199025] [DEBUG] [LOCAL ] BCE: No primary data file found to delete. [2025-09-16T17:24:25.199090] [INFO ] [LOCAL ] BCE: Starting deletion of derivative data. [2025-09-16T17:24:25.199328] [DEBUG] [LOCAL ] BCE: Deleted derivative data file. [2025-09-16T17:24:25.199458] [INFO ] [MAIN ] Update Progress: 237/2369 symbols completed. BCE took 33.8s. Estimated remaining: 3h 39m 13s. Rate: 9.7 symbols/min [2025-09-16T17:24:25.826413] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T17:24:25.974951] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-04. [2025-09-16T17:24:26.417225] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-09-16T17:24:26.774912] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-03. [2025-09-16T17:24:27.304683] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T17:24:27.484089] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-09-16T17:24:27.574869] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-04. [2025-09-16T17:24:28.374923] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-05. [2025-09-16T17:24:28.726908] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T17:24:29.174871] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-05. [2025-09-16T17:24:29.561867] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-09-16T17:24:29.777925] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-09-16T17:24:29.974904] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-06. [2025-09-16T17:24:30.774900] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-06. [2025-09-16T17:24:31.058850] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T17:24:31.548498] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T17:24:31.574845] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-07. [2025-09-16T17:24:32.374868] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-07. [2025-09-16T17:24:32.696393] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T17:24:33.174903] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-08. [2025-09-16T17:24:33.396773] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T17:24:33.974893] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-03. [2025-09-16T17:24:33.986967] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T17:24:34.774912] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-09. [2025-09-16T17:24:35.010878] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-09-16T17:24:35.574910] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-08. [2025-09-16T17:24:35.695340] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T17:24:35.697922] [INFO ] [REMOTE ] BCX: Full fetch complete (24154 records). [2025-09-16T17:24:35.697995] [DEBUG] [VALIDATOR ] BCX: Validating primary data. [2025-09-16T17:24:36.064424] [DEBUG] [VALIDATOR ] BCX: All expected trading days are present. [2025-09-16T17:24:36.064508] [DEBUG] [VALIDATOR ] BCX: Primary data validation passed. [2025-09-16T17:24:36.064951] [INFO ] [MAIN ] BCX: Fetched primary data is valid. [2025-09-16T17:24:36.065002] [DEBUG] [LOCAL ] BCX: Saving primary data. [2025-09-16T17:24:36.065910] [DEBUG] [LOCAL ] BCX: Trimmed 24154 records to 15406 (keeping 150 days of data) [2025-09-16T17:24:36.169034] [DEBUG] [LOCAL ] BCX: Primary data saved (15406 records). [2025-09-16T17:24:36.169180] [INFO ] [MAIN ] BCX: Full history saved. [2025-09-16T17:24:36.169305] [INFO ] [MAIN ] Update Progress: 238/2369 symbols completed. BCX took 16.4s. Estimated remaining: 3h 39m 50s. Rate: 9.7 symbols/min [2025-09-16T17:24:36.374981] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-04. [2025-09-16T17:24:36.602138] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T17:24:37.174960] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-09. [2025-09-16T17:24:37.346194] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-09-16T17:24:37.974856] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-05. [2025-09-16T17:24:38.356128] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T17:24:38.359566] [INFO ] [REMOTE ] BCSF: Full fetch complete (29771 records). [2025-09-16T17:24:38.359728] [DEBUG] [VALIDATOR ] BCSF: Validating primary data. [2025-09-16T17:24:38.730580] [DEBUG] [VALIDATOR ] BCSF: All expected trading days are present. [2025-09-16T17:24:38.730686] [DEBUG] [VALIDATOR ] BCSF: Primary data validation passed. [2025-09-16T17:24:38.731099] [INFO ] [MAIN ] BCSF: Fetched primary data is valid. [2025-09-16T17:24:38.731150] [DEBUG] [LOCAL ] BCSF: Saving primary data. [2025-09-16T17:24:38.732114] [DEBUG] [LOCAL ] BCSF: Trimmed 29771 records to 19518 (keeping 150 days of data) [2025-09-16T17:24:38.807130] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T17:24:38.839076] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-06. [2025-09-16T17:24:38.870787] [DEBUG] [LOCAL ] BCSF: Primary data saved (19518 records). [2025-09-16T17:24:38.870899] [INFO ] [MAIN ] BCSF: Full history saved. [2025-09-16T17:24:38.870988] [INFO ] [MAIN ] Update Progress: 239/2369 symbols completed. BCSF took 20.4s. Estimated remaining: 3h 39m 12s. Rate: 9.7 symbols/min [2025-09-16T17:24:39.979580] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T17:24:39.987376] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-07. [2025-09-16T17:24:41.171415] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-09-16T17:24:41.177311] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-08. [2025-09-16T17:24:42.284331] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-09-16T17:24:42.290045] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-09. [2025-09-16T17:24:43.436774] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T17:24:43.439816] [INFO ] [REMOTE ] BDC: Full fetch complete (27297 records). [2025-09-16T17:24:43.439899] [DEBUG] [VALIDATOR ] BDC: Validating primary data. [2025-09-16T17:24:43.801853] [DEBUG] [VALIDATOR ] BDC: All expected trading days are present. [2025-09-16T17:24:43.801932] [DEBUG] [VALIDATOR ] BDC: Primary data validation passed. [2025-09-16T17:24:43.802309] [INFO ] [MAIN ] BDC: Fetched primary data is valid. [2025-09-16T17:24:43.802350] [DEBUG] [LOCAL ] BDC: Saving primary data. [2025-09-16T17:24:43.803034] [DEBUG] [LOCAL ] BDC: Trimmed 27297 records to 17786 (keeping 150 days of data) [2025-09-16T17:24:43.919893] [DEBUG] [LOCAL ] BDC: Primary data saved (17786 records). [2025-09-16T17:24:43.920086] [INFO ] [MAIN ] BDC: Full history saved. [2025-09-16T17:24:43.920175] [INFO ] [MAIN ] Update Progress: 240/2369 symbols completed. BDC took 19.7s. Estimated remaining: 3h 38m 56s. Rate: 9.7 symbols/min [2025-09-16T17:24:44.543542] [INFO ] [MAIN ] BE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:24:44.631243] [INFO ] [MAIN ] BDN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:24:44.714804] [INFO ] [MAIN ] BDJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:24:44.718635] [INFO ] [MAIN ] BDX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:24:44.772879] [DEBUG] [LOCAL ] BDJ: Loaded 21245 existing records from primary storage. [2025-09-16T17:24:44.773157] [INFO ] [MAIN ] BDJ: Loaded 21245 existing records. [2025-09-16T17:24:44.886865] [DEBUG] [LOCAL ] BDN: Loaded 34811 existing records from primary storage. [2025-09-16T17:24:44.901610] [INFO ] [MAIN ] BDN: Loaded 34811 existing records. [2025-09-16T17:24:44.914228] [DEBUG] [LOCAL ] BE: Loaded 39622 existing records from primary storage. [2025-09-16T17:24:44.962492] [INFO ] [MAIN ] BE: Loaded 39622 existing records. [2025-09-16T17:24:45.110690] [WARN ] [MAIN ] BDJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:24:45.182360] [INFO ] [MAIN ] BDJ: Wiping existing primary data before full refresh. [2025-09-16T17:24:45.187631] [DEBUG] [LOCAL ] BDX: Loaded 38949 existing records from primary storage. [2025-09-16T17:24:45.193504] [INFO ] [MAIN ] BDX: Loaded 38949 existing records. [2025-09-16T17:24:45.253924] [INFO ] [LOCAL ] BDJ: Starting deletion of primary data. [2025-09-16T17:24:45.374774] [WARN ] [MAIN ] BDN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:24:45.395836] [DEBUG] [LOCAL ] BDJ: Deleted primary data file. [2025-09-16T17:24:45.463506] [INFO ] [MAIN ] BDJ: Starting full primary refresh. [2025-09-16T17:24:45.478788] [INFO ] [MAIN ] BDN: Wiping existing primary data before full refresh. [2025-09-16T17:24:45.482236] [INFO ] [LOCAL ] BDN: Starting deletion of primary data. [2025-09-16T17:24:45.473543] [WARN ] [MAIN ] BE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:24:45.516537] [DEBUG] [LOCAL ] BDN: Deleted primary data file. [2025-09-16T17:24:45.547407] [INFO ] [MAIN ] BE: Wiping existing primary data before full refresh. [2025-09-16T17:24:45.542200] [INFO ] [MAIN ] BDN: Starting full primary refresh. [2025-09-16T17:24:45.576029] [INFO ] [LOCAL ] BE: Starting deletion of primary data. [2025-09-16T17:24:45.695708] [WARN ] [MAIN ] BDX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:24:45.706963] [INFO ] [MAIN ] BDX: Wiping existing primary data before full refresh. [2025-09-16T17:24:45.725956] [INFO ] [LOCAL ] BDX: Starting deletion of primary data. [2025-09-16T17:24:45.690364] [DEBUG] [LOCAL ] BE: Deleted primary data file. [2025-09-16T17:24:45.763552] [INFO ] [MAIN ] BE: Starting full primary refresh. [2025-09-16T17:24:45.763725] [DEBUG] [LOCAL ] BDX: Deleted primary data file. [2025-09-16T17:24:45.774502] [INFO ] [MAIN ] BDX: Starting full primary refresh. [2025-09-16T17:24:46.738540] [INFO ] [REMOTE ] BDJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:24:46.824457] [INFO ] [REMOTE ] BDN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:24:46.829923] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-02. [2025-09-16T17:24:46.866425] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-02. [2025-09-16T17:24:47.039527] [INFO ] [REMOTE ] BDX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:24:47.045694] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-02. [2025-09-16T17:24:47.064995] [INFO ] [REMOTE ] BE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:24:47.066986] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-02. [2025-09-16T17:24:48.234858] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-09-16T17:24:48.243923] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-03. [2025-09-16T17:24:48.371095] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-09-16T17:24:48.380231] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-03. [2025-09-16T17:24:48.755911] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-09-16T17:24:48.766177] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-03. [2025-09-16T17:24:48.826945] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-09-16T17:24:48.837784] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-03. [2025-09-16T17:24:49.169632] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-09-16T17:24:49.176110] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-04. [2025-09-16T17:24:49.827942] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T17:24:49.838072] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-04. [2025-09-16T17:24:49.941328] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-09-16T17:24:49.974864] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-04. [2025-09-16T17:24:50.497977] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T17:24:50.657088] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-09-16T17:24:50.774891] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-04. [2025-09-16T17:24:51.582303] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-05. [2025-09-16T17:24:51.602884] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T17:24:51.791450] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T17:24:52.380007] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-05. [2025-09-16T17:24:52.394111] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T17:24:52.718895] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T17:24:53.174900] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-06. [2025-09-16T17:24:53.731152] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T17:24:53.974869] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-05. [2025-09-16T17:24:54.120843] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-09-16T17:24:54.774906] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-06. [2025-09-16T17:24:55.574894] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-05. [2025-09-16T17:24:56.006249] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-09-16T17:24:56.274855] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T17:24:56.374914] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-07. [2025-09-16T17:24:56.842302] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T17:24:57.174899] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-06. [2025-09-16T17:24:57.357528] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-09-16T17:24:57.974904] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-07. [2025-09-16T17:24:58.779989] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-08. [2025-09-16T17:24:58.798075] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T17:24:59.574902] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-07. [2025-09-16T17:24:59.772600] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T17:24:59.912487] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-09-16T17:25:00.374919] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-09. [2025-09-16T17:25:01.174903] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-08. [2025-09-16T17:25:01.436135] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-09-16T17:25:01.610287] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T17:25:01.614314] [INFO ] [REMOTE ] BDJ: Full fetch complete (31704 records). [2025-09-16T17:25:01.614450] [DEBUG] [VALIDATOR ] BDJ: Validating primary data. [2025-09-16T17:25:01.979982] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-08. [2025-09-16T17:25:02.024352] [DEBUG] [VALIDATOR ] BDJ: All expected trading days are present. [2025-09-16T17:25:02.024453] [DEBUG] [VALIDATOR ] BDJ: Primary data validation passed. [2025-09-16T17:25:02.025057] [INFO ] [MAIN ] BDJ: Fetched primary data is valid. [2025-09-16T17:25:02.025156] [DEBUG] [LOCAL ] BDJ: Saving primary data. [2025-09-16T17:25:02.026330] [DEBUG] [LOCAL ] BDJ: Trimmed 31704 records to 21383 (keeping 150 days of data) [2025-09-16T17:25:02.162754] [DEBUG] [LOCAL ] BDJ: Primary data saved (21383 records). [2025-09-16T17:25:02.162887] [INFO ] [MAIN ] BDJ: Full history saved. [2025-09-16T17:25:02.162973] [INFO ] [MAIN ] Update Progress: 241/2369 symbols completed. BDJ took 18.2s. Estimated remaining: 3h 40m 37s. Rate: 9.6 symbols/min [2025-09-16T17:25:02.334266] [INFO ] [MAIN ] BEDU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:02.334478] [DEBUG] [LOCAL ] BEDU: No existing data file found in primary storage. [2025-09-16T17:25:02.334518] [WARN ] [MAIN ] BEDU: No existing data found, performing full refresh instead. [2025-09-16T17:25:02.334640] [INFO ] [MAIN ] BEDU: Wiping existing primary data before full refresh. [2025-09-16T17:25:02.334700] [INFO ] [LOCAL ] BEDU: Starting deletion of primary data. [2025-09-16T17:25:02.334741] [DEBUG] [LOCAL ] BEDU: No primary data file found to delete. [2025-09-16T17:25:02.334773] [INFO ] [MAIN ] BEDU: Starting full primary refresh. [2025-09-16T17:25:02.690238] [INFO ] [REMOTE ] BEDU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:02.774896] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-06. [2025-09-16T17:25:03.574932] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-02. [2025-09-16T17:25:04.006357] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-09-16T17:25:04.374948] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-03. [2025-09-16T17:25:04.698803] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-09-16T17:25:04.775687] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-09-16T17:25:05.174912] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-04. [2025-09-16T17:25:05.472479] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-09-16T17:25:05.974924] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-05. [2025-09-16T17:25:06.071543] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-08 (4.90 seconds). [2025-09-16T17:25:06.303508] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-09-16T17:25:06.774927] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-07. [2025-09-16T17:25:07.094583] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-08 (5.11 seconds). [2025-09-16T17:25:07.574966] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-09. [2025-09-16T17:25:08.374872] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-09. [2025-09-16T17:25:08.453235] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-09-16T17:25:09.167993] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T17:25:09.173738] [INFO ] [REMOTE ] BDX: Full fetch complete (56535 records). [2025-09-16T17:25:09.173895] [DEBUG] [VALIDATOR ] BDX: Validating primary data. [2025-09-16T17:25:09.174628] [FATAL] [VALIDATOR ] BDX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:25:09.174732] [FATAL] [MAIN ] BDX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:25:09.174770] [WARN ] [MAIN ] BDX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:25:09.174819] [INFO ] [LOCAL ] BDX: Starting deletion of primary data. [2025-09-16T17:25:09.174869] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-08. [2025-09-16T17:25:09.175013] [DEBUG] [LOCAL ] BDX: No primary data file found to delete. [2025-09-16T17:25:09.175839] [INFO ] [LOCAL ] BDX: Starting deletion of derivative data. [2025-09-16T17:25:09.176261] [DEBUG] [LOCAL ] BDX: Deleted derivative data file. [2025-09-16T17:25:09.176409] [INFO ] [MAIN ] Update Progress: 242/2369 symbols completed. BDX took 25.2s. Estimated remaining: 3h 40m 37s. Rate: 9.6 symbols/min [2025-09-16T17:25:09.360318] [INFO ] [MAIN ] BEKE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:09.400238] [DEBUG] [LOCAL ] BEKE: Loaded 39242 existing records from primary storage. [2025-09-16T17:25:09.400387] [INFO ] [MAIN ] BEKE: Loaded 39242 existing records. [2025-09-16T17:25:09.589140] [WARN ] [MAIN ] BEKE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:25:09.589420] [INFO ] [MAIN ] BEKE: Wiping existing primary data before full refresh. [2025-09-16T17:25:09.589491] [INFO ] [LOCAL ] BEKE: Starting deletion of primary data. [2025-09-16T17:25:09.595868] [DEBUG] [LOCAL ] BEKE: Deleted primary data file. [2025-09-16T17:25:09.601995] [INFO ] [MAIN ] BEKE: Starting full primary refresh. [2025-09-16T17:25:09.664286] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T17:25:09.718576] [INFO ] [REMOTE ] BDN: Full fetch complete (51397 records). [2025-09-16T17:25:09.718756] [DEBUG] [VALIDATOR ] BDN: Validating primary data. [2025-09-16T17:25:09.735255] [FATAL] [VALIDATOR ] BDN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:25:09.797582] [FATAL] [MAIN ] BDN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:25:09.849720] [WARN ] [MAIN ] BDN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:25:09.883356] [INFO ] [LOCAL ] BDN: Starting deletion of primary data. [2025-09-16T17:25:09.912993] [DEBUG] [LOCAL ] BDN: No primary data file found to delete. [2025-09-16T17:25:09.918737] [INFO ] [LOCAL ] BDN: Starting deletion of derivative data. [2025-09-16T17:25:09.929327] [DEBUG] [LOCAL ] BDN: Deleted derivative data file. [2025-09-16T17:25:09.954365] [INFO ] [MAIN ] Update Progress: 243/2369 symbols completed. BDN took 26.0s. Estimated remaining: 3h 39m 44s. Rate: 9.7 symbols/min [2025-09-16T17:25:09.986795] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-06. [2025-09-16T17:25:09.992035] [INFO ] [REMOTE ] BEKE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:10.187309] [INFO ] [MAIN ] BEN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:10.228738] [DEBUG] [LOCAL ] BEN: Loaded 39192 existing records from primary storage. [2025-09-16T17:25:10.228889] [INFO ] [MAIN ] BEN: Loaded 39192 existing records. [2025-09-16T17:25:10.337069] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-09-16T17:25:10.415611] [WARN ] [MAIN ] BEN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:25:10.415912] [INFO ] [MAIN ] BEN: Wiping existing primary data before full refresh. [2025-09-16T17:25:10.415963] [INFO ] [LOCAL ] BEN: Starting deletion of primary data. [2025-09-16T17:25:10.417021] [DEBUG] [LOCAL ] BEN: Deleted primary data file. [2025-09-16T17:25:10.417078] [INFO ] [MAIN ] BEN: Starting full primary refresh. [2025-09-16T17:25:10.780511] [INFO ] [REMOTE ] BEN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:10.777981] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-02. [2025-09-16T17:25:11.472274] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-08 (2.30 seconds). [2025-09-16T17:25:11.574852] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-09. [2025-09-16T17:25:12.357541] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T17:25:12.374870] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-07. [2025-09-16T17:25:12.625861] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-07 (0.25 seconds). [2025-09-16T17:25:13.174948] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-08. [2025-09-16T17:25:13.397504] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-08 (0.22 seconds). [2025-09-16T17:25:13.839979] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-09 (2.27 seconds). [2025-09-16T17:25:13.845813] [INFO ] [REMOTE ] BE: Full fetch complete (57934 records). [2025-09-16T17:25:13.845907] [DEBUG] [VALIDATOR ] BE: Validating primary data. [2025-09-16T17:25:13.846579] [FATAL] [VALIDATOR ] BE: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:25:13.846675] [FATAL] [MAIN ] BE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:25:13.846716] [WARN ] [MAIN ] BE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:25:13.846753] [INFO ] [LOCAL ] BE: Starting deletion of primary data. [2025-09-16T17:25:13.846834] [DEBUG] [LOCAL ] BE: No primary data file found to delete. [2025-09-16T17:25:13.846871] [INFO ] [LOCAL ] BE: Starting deletion of derivative data. [2025-09-16T17:25:13.847148] [DEBUG] [LOCAL ] BE: Deleted derivative data file. [2025-09-16T17:25:13.847227] [INFO ] [MAIN ] Update Progress: 244/2369 symbols completed. BE took 29.9s. Estimated remaining: 3h 39m 17s. Rate: 9.7 symbols/min [2025-09-16T17:25:13.986847] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-02. [2025-09-16T17:25:14.016517] [INFO ] [MAIN ] BEP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:14.048306] [DEBUG] [LOCAL ] BEP: Loaded 27576 existing records from primary storage. [2025-09-16T17:25:14.048460] [INFO ] [MAIN ] BEP: Loaded 27576 existing records. [2025-09-16T17:25:14.222272] [WARN ] [MAIN ] BEP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:25:14.222517] [INFO ] [MAIN ] BEP: Wiping existing primary data before full refresh. [2025-09-16T17:25:14.222562] [INFO ] [LOCAL ] BEP: Starting deletion of primary data. [2025-09-16T17:25:14.223405] [DEBUG] [LOCAL ] BEP: Deleted primary data file. [2025-09-16T17:25:14.223455] [INFO ] [MAIN ] BEP: Starting full primary refresh. [2025-09-16T17:25:14.573731] [INFO ] [REMOTE ] BEP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:14.774939] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-02. [2025-09-16T17:25:15.340459] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-09-16T17:25:15.574967] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-03. [2025-09-16T17:25:15.962779] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T17:25:16.374958] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-09. [2025-09-16T17:25:16.920446] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-09-16T17:25:16.921238] [INFO ] [REMOTE ] BEDU: Full fetch complete (933 records). [2025-09-16T17:25:16.921308] [DEBUG] [VALIDATOR ] BEDU: Validating primary data. [2025-09-16T17:25:17.110304] [WARN ] [VALIDATOR ] BEDU: Validation failed - insufficient historical data. Have 122 unique days, need 150 days [2025-09-16T17:25:17.110550] [FATAL] [MAIN ] BEDU: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:25:17.110626] [WARN ] [MAIN ] BEDU: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:25:17.110856] [INFO ] [LOCAL ] BEDU: Starting deletion of primary data. [2025-09-16T17:25:17.111010] [DEBUG] [LOCAL ] BEDU: No primary data file found to delete. [2025-09-16T17:25:17.111141] [INFO ] [LOCAL ] BEDU: Starting deletion of derivative data. [2025-09-16T17:25:17.111299] [DEBUG] [LOCAL ] BEDU: No derivative data file found to delete. [2025-09-16T17:25:17.111382] [INFO ] [MAIN ] Update Progress: 245/2369 symbols completed. BEDU took 14.9s. Estimated remaining: 3h 38m 46s. Rate: 9.7 symbols/min [2025-09-16T17:25:17.179948] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-03. [2025-09-16T17:25:17.294417] [INFO ] [MAIN ] BEPC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:17.327784] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T17:25:17.370777] [DEBUG] [LOCAL ] BEPC: Loaded 33680 existing records from primary storage. [2025-09-16T17:25:17.370920] [INFO ] [MAIN ] BEPC: Loaded 33680 existing records. [2025-09-16T17:25:17.548585] [WARN ] [MAIN ] BEPC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:25:17.548847] [INFO ] [MAIN ] BEPC: Wiping existing primary data before full refresh. [2025-09-16T17:25:17.548893] [INFO ] [LOCAL ] BEPC: Starting deletion of primary data. [2025-09-16T17:25:17.549797] [DEBUG] [LOCAL ] BEPC: Deleted primary data file. [2025-09-16T17:25:17.549847] [INFO ] [MAIN ] BEPC: Starting full primary refresh. [2025-09-16T17:25:17.904261] [INFO ] [REMOTE ] BEPC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:17.974854] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-02. [2025-09-16T17:25:18.775874] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T17:25:18.775979] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-04. [2025-09-16T17:25:19.574937] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-03. [2025-09-16T17:25:19.657437] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T17:25:20.334865] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T17:25:20.374926] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-04. [2025-09-16T17:25:20.760006] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-09-16T17:25:21.174915] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-03. [2025-09-16T17:25:21.802658] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:25:21.974954] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-05. [2025-09-16T17:25:22.481087] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T17:25:22.774908] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-04. [2025-09-16T17:25:23.574937] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-04. [2025-09-16T17:25:23.800428] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-09-16T17:25:24.115814] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-09-16T17:25:24.374914] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-05. [2025-09-16T17:25:24.838820] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-09-16T17:25:25.174964] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-05. [2025-09-16T17:25:25.979544] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-06. [2025-09-16T17:25:25.986672] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T17:25:26.353478] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-09-16T17:25:26.774906] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-06. [2025-09-16T17:25:27.574908] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-05. [2025-09-16T17:25:27.800701] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-09-16T17:25:28.004055] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T17:25:28.374921] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-06. [2025-09-16T17:25:29.157425] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T17:25:29.174855] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-06. [2025-09-16T17:25:29.790833] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T17:25:29.974909] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-07. [2025-09-16T17:25:30.780055] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-07. [2025-09-16T17:25:30.795258] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T17:25:31.459854] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T17:25:31.574887] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-08. [2025-09-16T17:25:32.374897] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-07. [2025-09-16T17:25:32.605662] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-09-16T17:25:32.915698] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T17:25:33.174924] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-07. [2025-09-16T17:25:33.632608] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T17:25:33.974917] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-08. [2025-09-16T17:25:34.484600] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T17:25:34.774950] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-09. [2025-09-16T17:25:35.574902] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-08. [2025-09-16T17:25:35.905627] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T17:25:36.375054] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-08. [2025-09-16T17:25:36.454029] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T17:25:36.461115] [INFO ] [REMOTE ] BEN: Full fetch complete (57719 records). [2025-09-16T17:25:36.461367] [DEBUG] [VALIDATOR ] BEN: Validating primary data. [2025-09-16T17:25:36.862606] [DEBUG] [VALIDATOR ] BEN: All expected trading days are present. [2025-09-16T17:25:36.862719] [DEBUG] [VALIDATOR ] BEN: Primary data validation passed. [2025-09-16T17:25:36.863367] [INFO ] [MAIN ] BEN: Fetched primary data is valid. [2025-09-16T17:25:36.863411] [DEBUG] [LOCAL ] BEN: Saving primary data. [2025-09-16T17:25:36.864740] [DEBUG] [LOCAL ] BEN: Trimmed 57719 records to 39572 (keeping 150 days of data) [2025-09-16T17:25:37.133373] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T17:25:37.169057] [DEBUG] [LOCAL ] BEN: Primary data saved (39572 records). [2025-09-16T17:25:37.169205] [INFO ] [MAIN ] BEN: Full history saved. [2025-09-16T17:25:37.169296] [INFO ] [MAIN ] Update Progress: 246/2369 symbols completed. BEN took 27.2s. Estimated remaining: 3h 40m 39s. Rate: 9.6 symbols/min [2025-09-16T17:25:37.180947] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-09. [2025-09-16T17:25:37.347074] [INFO ] [MAIN ] BEPH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:37.352155] [DEBUG] [LOCAL ] BEPH: Loaded 3476 existing records from primary storage. [2025-09-16T17:25:37.352243] [INFO ] [MAIN ] BEPH: Loaded 3476 existing records. [2025-09-16T17:25:37.531200] [WARN ] [MAIN ] BEPH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:25:37.531482] [INFO ] [MAIN ] BEPH: Wiping existing primary data before full refresh. [2025-09-16T17:25:37.531531] [INFO ] [LOCAL ] BEPH: Starting deletion of primary data. [2025-09-16T17:25:37.531955] [DEBUG] [LOCAL ] BEPH: Deleted primary data file. [2025-09-16T17:25:37.532005] [INFO ] [MAIN ] BEPH: Starting full primary refresh. [2025-09-16T17:25:37.892909] [INFO ] [REMOTE ] BEPH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:37.931780] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T17:25:37.974924] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-09. [2025-09-16T17:25:38.774895] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-02. [2025-09-16T17:25:39.158622] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-09-16T17:25:39.165043] [INFO ] [REMOTE ] BEKE: Full fetch complete (57946 records). [2025-09-16T17:25:39.165158] [DEBUG] [VALIDATOR ] BEKE: Validating primary data. [2025-09-16T17:25:39.275634] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-09-16T17:25:39.492756] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T17:25:39.538279] [INFO ] [REMOTE ] BEP: Full fetch complete (42614 records). [2025-09-16T17:25:39.549371] [DEBUG] [VALIDATOR ] BEP: Validating primary data. [2025-09-16T17:25:39.580376] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-09. [2025-09-16T17:25:39.652004] [DEBUG] [VALIDATOR ] BEKE: All expected trading days are present. [2025-09-16T17:25:39.666734] [DEBUG] [VALIDATOR ] BEKE: Primary data validation passed. [2025-09-16T17:25:39.693977] [INFO ] [MAIN ] BEKE: Fetched primary data is valid. [2025-09-16T17:25:39.705885] [DEBUG] [LOCAL ] BEKE: Saving primary data. [2025-09-16T17:25:39.782267] [DEBUG] [LOCAL ] BEKE: Trimmed 57946 records to 39625 (keeping 150 days of data) [2025-09-16T17:25:40.085715] [DEBUG] [VALIDATOR ] BEP: All expected trading days are present. [2025-09-16T17:25:40.088550] [DEBUG] [VALIDATOR ] BEP: Primary data validation passed. [2025-09-16T17:25:40.092260] [INFO ] [MAIN ] BEP: Fetched primary data is valid. [2025-09-16T17:25:40.092823] [DEBUG] [LOCAL ] BEP: Saving primary data. [2025-09-16T17:25:40.107095] [DEBUG] [LOCAL ] BEP: Trimmed 42614 records to 27781 (keeping 150 days of data) [2025-09-16T17:25:40.351838] [DEBUG] [LOCAL ] BEKE: Primary data saved (39625 records). [2025-09-16T17:25:40.355234] [INFO ] [MAIN ] BEKE: Full history saved. [2025-09-16T17:25:40.356824] [INFO ] [MAIN ] Update Progress: 247/2369 symbols completed. BEKE took 31.2s. Estimated remaining: 3h 40m 7s. Rate: 9.6 symbols/min [2025-09-16T17:25:40.407019] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-03. [2025-09-16T17:25:40.589893] [INFO ] [MAIN ] BEPI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:40.600922] [DEBUG] [LOCAL ] BEP: Primary data saved (27781 records). [2025-09-16T17:25:40.601035] [INFO ] [MAIN ] BEP: Full history saved. [2025-09-16T17:25:40.599694] [DEBUG] [LOCAL ] BEPI: Loaded 2662 existing records from primary storage. [2025-09-16T17:25:40.602805] [INFO ] [MAIN ] Update Progress: 248/2369 symbols completed. BEP took 26.8s. Estimated remaining: 3h 39m 9s. Rate: 9.7 symbols/min [2025-09-16T17:25:40.602886] [INFO ] [MAIN ] BEPI: Loaded 2662 existing records. [2025-09-16T17:25:40.945559] [INFO ] [MAIN ] BEPJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:40.963213] [WARN ] [MAIN ] BEPI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:25:40.964079] [INFO ] [MAIN ] BEPI: Wiping existing primary data before full refresh. [2025-09-16T17:25:40.964150] [INFO ] [LOCAL ] BEPI: Starting deletion of primary data. [2025-09-16T17:25:40.964410] [DEBUG] [LOCAL ] BEPI: Deleted primary data file. [2025-09-16T17:25:40.964471] [INFO ] [MAIN ] BEPI: Starting full primary refresh. [2025-09-16T17:25:41.066031] [DEBUG] [LOCAL ] BEPJ: Loaded 2575 existing records from primary storage. [2025-09-16T17:25:41.077719] [INFO ] [MAIN ] BEPJ: Loaded 2575 existing records. [2025-09-16T17:25:41.383149] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T17:25:41.496456] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-04. [2025-09-16T17:25:41.496147] [WARN ] [MAIN ] BEPJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:25:41.523061] [INFO ] [MAIN ] BEPJ: Wiping existing primary data before full refresh. [2025-09-16T17:25:41.526514] [INFO ] [REMOTE ] BEPI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:41.526756] [INFO ] [LOCAL ] BEPJ: Starting deletion of primary data. [2025-09-16T17:25:41.532910] [DEBUG] [LOCAL ] BEPJ: Deleted primary data file. [2025-09-16T17:25:41.541730] [INFO ] [MAIN ] BEPJ: Starting full primary refresh. [2025-09-16T17:25:41.548064] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-09-16T17:25:41.623958] [INFO ] [REMOTE ] BEPC: Full fetch complete (50895 records). [2025-09-16T17:25:41.649027] [DEBUG] [VALIDATOR ] BEPC: Validating primary data. [2025-09-16T17:25:41.977017] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-02. [2025-09-16T17:25:42.229851] [INFO ] [REMOTE ] BEPJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:42.295515] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-09-16T17:25:42.329568] [DEBUG] [VALIDATOR ] BEPC: All expected trading days are present. [2025-09-16T17:25:42.329685] [DEBUG] [VALIDATOR ] BEPC: Primary data validation passed. [2025-09-16T17:25:42.330335] [INFO ] [MAIN ] BEPC: Fetched primary data is valid. [2025-09-16T17:25:42.330379] [DEBUG] [LOCAL ] BEPC: Saving primary data. [2025-09-16T17:25:42.331499] [DEBUG] [LOCAL ] BEPC: Trimmed 50895 records to 34009 (keeping 150 days of data) [2025-09-16T17:25:42.557948] [DEBUG] [LOCAL ] BEPC: Primary data saved (34009 records). [2025-09-16T17:25:42.558235] [INFO ] [MAIN ] BEPC: Full history saved. [2025-09-16T17:25:42.558326] [INFO ] [MAIN ] Update Progress: 249/2369 symbols completed. BEPC took 25.4s. Estimated remaining: 3h 38m 27s. Rate: 9.7 symbols/min [2025-09-16T17:25:42.663332] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T17:25:42.739167] [INFO ] [MAIN ] BFAM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:42.764631] [DEBUG] [LOCAL ] BFAM: Loaded 24414 existing records from primary storage. [2025-09-16T17:25:42.764844] [INFO ] [MAIN ] BFAM: Loaded 24414 existing records. [2025-09-16T17:25:42.780040] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-05. [2025-09-16T17:25:42.961044] [WARN ] [MAIN ] BFAM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:25:42.961339] [INFO ] [MAIN ] BFAM: Wiping existing primary data before full refresh. [2025-09-16T17:25:42.961386] [INFO ] [LOCAL ] BFAM: Starting deletion of primary data. [2025-09-16T17:25:42.962217] [DEBUG] [LOCAL ] BFAM: Deleted primary data file. [2025-09-16T17:25:42.962281] [INFO ] [MAIN ] BFAM: Starting full primary refresh. [2025-09-16T17:25:43.353278] [INFO ] [REMOTE ] BFAM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:43.440403] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-09-16T17:25:43.574903] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-02. [2025-09-16T17:25:44.374894] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-06. [2025-09-16T17:25:44.956876] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-09-16T17:25:44.991077] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T17:25:45.174910] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-02. [2025-09-16T17:25:45.683315] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-09-16T17:25:45.974905] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-03. [2025-09-16T17:25:46.774907] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-07. [2025-09-16T17:25:47.177435] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-09-16T17:25:47.225773] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-09-16T17:25:47.574896] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-03. [2025-09-16T17:25:48.003003] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-09-16T17:25:48.374908] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-03. [2025-09-16T17:25:48.825267] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-09-16T17:25:49.174907] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-04. [2025-09-16T17:25:49.974900] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-04. [2025-09-16T17:25:50.270567] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-09-16T17:25:50.584410] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-09-16T17:25:50.774881] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-04. [2025-09-16T17:25:51.327592] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T17:25:51.574936] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-05. [2025-09-16T17:25:51.956727] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-09-16T17:25:52.374892] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-05. [2025-09-16T17:25:53.174908] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-08. [2025-09-16T17:25:53.510299] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T17:25:53.595894] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-09-16T17:25:53.974927] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-06. [2025-09-16T17:25:54.566756] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T17:25:54.774934] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-09. [2025-09-16T17:25:55.399849] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-09-16T17:25:55.400876] [INFO ] [REMOTE ] BEPH: Full fetch complete (5755 records). [2025-09-16T17:25:55.400944] [DEBUG] [VALIDATOR ] BEPH: Validating primary data. [2025-09-16T17:25:55.580210] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-06. [2025-09-16T17:25:55.771840] [DEBUG] [VALIDATOR ] BEPH: All expected trading days are present. [2025-09-16T17:25:55.771921] [DEBUG] [VALIDATOR ] BEPH: Primary data validation passed. [2025-09-16T17:25:55.772097] [INFO ] [MAIN ] BEPH: Fetched primary data is valid. [2025-09-16T17:25:55.772145] [DEBUG] [LOCAL ] BEPH: Saving primary data. [2025-09-16T17:25:55.772533] [DEBUG] [LOCAL ] BEPH: Trimmed 5755 records to 3492 (keeping 150 days of data) [2025-09-16T17:25:55.794268] [DEBUG] [LOCAL ] BEPH: Primary data saved (3492 records). [2025-09-16T17:25:55.794358] [INFO ] [MAIN ] BEPH: Full history saved. [2025-09-16T17:25:55.794440] [INFO ] [MAIN ] Update Progress: 250/2369 symbols completed. BEPH took 18.6s. Estimated remaining: 3h 39m 21s. Rate: 9.7 symbols/min [2025-09-16T17:25:55.972566] [INFO ] [MAIN ] BFH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:25:56.006204] [DEBUG] [LOCAL ] BFH: Loaded 30069 existing records from primary storage. [2025-09-16T17:25:56.006399] [INFO ] [MAIN ] BFH: Loaded 30069 existing records. [2025-09-16T17:25:56.191090] [WARN ] [MAIN ] BFH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:25:56.191348] [INFO ] [MAIN ] BFH: Wiping existing primary data before full refresh. [2025-09-16T17:25:56.191396] [INFO ] [LOCAL ] BFH: Starting deletion of primary data. [2025-09-16T17:25:56.192318] [DEBUG] [LOCAL ] BFH: Deleted primary data file. [2025-09-16T17:25:56.192370] [INFO ] [MAIN ] BFH: Starting full primary refresh. [2025-09-16T17:25:56.381601] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-05. [2025-09-16T17:25:56.557362] [INFO ] [REMOTE ] BFH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:25:57.033140] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T17:25:57.174890] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-07. [2025-09-16T17:25:57.218082] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T17:25:57.974909] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-02. [2025-09-16T17:25:58.239873] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-09-16T17:25:58.774936] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-08. [2025-09-16T17:25:59.368751] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T17:25:59.574899] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-07. [2025-09-16T17:26:00.262707] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T17:26:00.358238] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-09-16T17:26:00.374850] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-08. [2025-09-16T17:26:01.080542] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-09-16T17:26:01.174963] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-09. [2025-09-16T17:26:01.711217] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-09-16T17:26:01.712142] [INFO ] [REMOTE ] BEPJ: Full fetch complete (3515 records). [2025-09-16T17:26:01.712224] [DEBUG] [VALIDATOR ] BEPJ: Validating primary data. [2025-09-16T17:26:01.712703] [FATAL] [VALIDATOR ] BEPJ: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:26:01.712782] [FATAL] [MAIN ] BEPJ: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:26:01.712821] [WARN ] [MAIN ] BEPJ: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:26:01.712863] [INFO ] [LOCAL ] BEPJ: Starting deletion of primary data. [2025-09-16T17:26:01.712922] [DEBUG] [LOCAL ] BEPJ: No primary data file found to delete. [2025-09-16T17:26:01.712959] [INFO ] [LOCAL ] BEPJ: Starting deletion of derivative data. [2025-09-16T17:26:01.713279] [DEBUG] [LOCAL ] BEPJ: Deleted derivative data file. [2025-09-16T17:26:01.713366] [INFO ] [MAIN ] Update Progress: 251/2369 symbols completed. BEPJ took 21.1s. Estimated remaining: 3h 39m 12s. Rate: 9.7 symbols/min [2025-09-16T17:26:01.903286] [INFO ] [MAIN ] BFK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:01.914584] [DEBUG] [LOCAL ] BFK: Loaded 9574 existing records from primary storage. [2025-09-16T17:26:01.914764] [INFO ] [MAIN ] BFK: Loaded 9574 existing records. [2025-09-16T17:26:01.980035] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-03. [2025-09-16T17:26:02.095886] [WARN ] [MAIN ] BFK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:02.096158] [INFO ] [MAIN ] BFK: Wiping existing primary data before full refresh. [2025-09-16T17:26:02.096206] [INFO ] [LOCAL ] BFK: Starting deletion of primary data. [2025-09-16T17:26:02.096528] [DEBUG] [LOCAL ] BFK: Deleted primary data file. [2025-09-16T17:26:02.096573] [INFO ] [MAIN ] BFK: Starting full primary refresh. [2025-09-16T17:26:02.455591] [INFO ] [REMOTE ] BFK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:02.774902] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-06. [2025-09-16T17:26:03.244706] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-09-16T17:26:03.574910] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-07. [2025-09-16T17:26:03.603163] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T17:26:04.133212] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-09-16T17:26:04.374911] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-02. [2025-09-16T17:26:05.052838] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-09-16T17:26:05.174890] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-03. [2025-09-16T17:26:05.727000] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-09-16T17:26:05.974923] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-09. [2025-09-16T17:26:06.774958] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-04. [2025-09-16T17:26:07.290217] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T17:26:07.294427] [INFO ] [REMOTE ] BFAM: Full fetch complete (35656 records). [2025-09-16T17:26:07.294563] [DEBUG] [VALIDATOR ] BFAM: Validating primary data. [2025-09-16T17:26:07.580794] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-04. [2025-09-16T17:26:07.671680] [DEBUG] [VALIDATOR ] BFAM: All expected trading days are present. [2025-09-16T17:26:07.671772] [DEBUG] [VALIDATOR ] BFAM: Primary data validation passed. [2025-09-16T17:26:07.672367] [INFO ] [MAIN ] BFAM: Fetched primary data is valid. [2025-09-16T17:26:07.672424] [DEBUG] [LOCAL ] BFAM: Saving primary data. [2025-09-16T17:26:07.673349] [DEBUG] [LOCAL ] BFAM: Trimmed 35656 records to 24751 (keeping 150 days of data) [2025-09-16T17:26:07.826128] [DEBUG] [LOCAL ] BFAM: Primary data saved (24751 records). [2025-09-16T17:26:07.826287] [INFO ] [MAIN ] BFAM: Full history saved. [2025-09-16T17:26:07.826396] [INFO ] [MAIN ] Update Progress: 252/2369 symbols completed. BFAM took 25.3s. Estimated remaining: 3h 39m 5s. Rate: 9.7 symbols/min [2025-09-16T17:26:07.995984] [INFO ] [MAIN ] BFLY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:08.031266] [DEBUG] [LOCAL ] BFLY: Loaded 38151 existing records from primary storage. [2025-09-16T17:26:08.031429] [INFO ] [MAIN ] BFLY: Loaded 38151 existing records. [2025-09-16T17:26:08.207549] [WARN ] [MAIN ] BFLY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:08.207789] [INFO ] [MAIN ] BFLY: Wiping existing primary data before full refresh. [2025-09-16T17:26:08.207835] [INFO ] [LOCAL ] BFLY: Starting deletion of primary data. [2025-09-16T17:26:08.208759] [DEBUG] [LOCAL ] BFLY: Deleted primary data file. [2025-09-16T17:26:08.208808] [INFO ] [MAIN ] BFLY: Starting full primary refresh. [2025-09-16T17:26:08.376592] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-08. [2025-09-16T17:26:08.575597] [INFO ] [REMOTE ] BFLY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:08.654691] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T17:26:08.936311] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-09-16T17:26:09.095798] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-09-16T17:26:09.174858] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-09. [2025-09-16T17:26:09.811933] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-09-16T17:26:09.812855] [INFO ] [REMOTE ] BEPI: Full fetch complete (3680 records). [2025-09-16T17:26:09.812935] [DEBUG] [VALIDATOR ] BEPI: Validating primary data. [2025-09-16T17:26:09.984234] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-05. [2025-09-16T17:26:10.167390] [DEBUG] [VALIDATOR ] BEPI: All expected trading days are present. [2025-09-16T17:26:10.167466] [DEBUG] [VALIDATOR ] BEPI: Primary data validation passed. [2025-09-16T17:26:10.167580] [INFO ] [MAIN ] BEPI: Fetched primary data is valid. [2025-09-16T17:26:10.167618] [DEBUG] [LOCAL ] BEPI: Saving primary data. [2025-09-16T17:26:10.167974] [DEBUG] [LOCAL ] BEPI: Trimmed 3680 records to 2679 (keeping 150 days of data) [2025-09-16T17:26:10.184324] [DEBUG] [LOCAL ] BEPI: Primary data saved (2679 records). [2025-09-16T17:26:10.184388] [INFO ] [MAIN ] BEPI: Full history saved. [2025-09-16T17:26:10.184474] [INFO ] [MAIN ] Update Progress: 253/2369 symbols completed. BEPI took 29.8s. Estimated remaining: 3h 38m 27s. Rate: 9.7 symbols/min [2025-09-16T17:26:10.356985] [INFO ] [MAIN ] BFS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:10.365718] [DEBUG] [LOCAL ] BFS: Loaded 6716 existing records from primary storage. [2025-09-16T17:26:10.365838] [INFO ] [MAIN ] BFS: Loaded 6716 existing records. [2025-09-16T17:26:10.542313] [WARN ] [MAIN ] BFS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:10.542569] [INFO ] [MAIN ] BFS: Wiping existing primary data before full refresh. [2025-09-16T17:26:10.542615] [INFO ] [LOCAL ] BFS: Starting deletion of primary data. [2025-09-16T17:26:10.542998] [DEBUG] [LOCAL ] BFS: Deleted primary data file. [2025-09-16T17:26:10.543044] [INFO ] [MAIN ] BFS: Starting full primary refresh. [2025-09-16T17:26:10.781099] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-05. [2025-09-16T17:26:10.846983] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T17:26:10.912428] [INFO ] [REMOTE ] BFS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:11.574918] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-02. [2025-09-16T17:26:11.890859] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-09-16T17:26:12.374900] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-02. [2025-09-16T17:26:12.949983] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-09-16T17:26:13.159038] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T17:26:13.174841] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-03. [2025-09-16T17:26:13.974902] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-06. [2025-09-16T17:26:14.674560] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-09-16T17:26:14.721662] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T17:26:14.774879] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-03. [2025-09-16T17:26:15.524217] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-09-16T17:26:15.574859] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-04. [2025-09-16T17:26:16.100573] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-09-16T17:26:16.374886] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-07. [2025-09-16T17:26:17.021802] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-09-16T17:26:17.174884] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-04. [2025-09-16T17:26:17.974946] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-05. [2025-09-16T17:26:18.602772] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-09-16T17:26:18.774975] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-06. [2025-09-16T17:26:18.925859] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-09-16T17:26:19.239550] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-09-16T17:26:19.574957] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-05. [2025-09-16T17:26:20.374951] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-06. [2025-09-16T17:26:20.942461] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-09-16T17:26:21.174940] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-06. [2025-09-16T17:26:21.946667] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T17:26:21.974919] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-08. [2025-09-16T17:26:22.509907] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T17:26:22.774994] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-07. [2025-09-16T17:26:22.921909] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-09-16T17:26:23.574902] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-09. [2025-09-16T17:26:24.296701] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-09-16T17:26:24.299073] [INFO ] [REMOTE ] BFK: Full fetch complete (14613 records). [2025-09-16T17:26:24.299142] [DEBUG] [VALIDATOR ] BFK: Validating primary data. [2025-09-16T17:26:24.379882] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-07. [2025-09-16T17:26:24.674542] [DEBUG] [VALIDATOR ] BFK: All expected trading days are present. [2025-09-16T17:26:24.674691] [DEBUG] [VALIDATOR ] BFK: Primary data validation passed. [2025-09-16T17:26:24.674984] [INFO ] [MAIN ] BFK: Fetched primary data is valid. [2025-09-16T17:26:24.675126] [DEBUG] [LOCAL ] BFK: Saving primary data. [2025-09-16T17:26:24.675867] [DEBUG] [LOCAL ] BFK: Trimmed 14613 records to 9721 (keeping 150 days of data) [2025-09-16T17:26:24.761670] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-09-16T17:26:24.777430] [DEBUG] [LOCAL ] BFK: Primary data saved (9721 records). [2025-09-16T17:26:24.777604] [INFO ] [MAIN ] BFK: Full history saved. [2025-09-16T17:26:24.777704] [INFO ] [MAIN ] Update Progress: 254/2369 symbols completed. BFK took 23.1s. Estimated remaining: 3h 39m 30s. Rate: 9.6 symbols/min [2025-09-16T17:26:24.947028] [INFO ] [MAIN ] BFZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:24.956061] [DEBUG] [LOCAL ] BFZ: Loaded 8249 existing records from primary storage. [2025-09-16T17:26:24.956149] [INFO ] [MAIN ] BFZ: Loaded 8249 existing records. [2025-09-16T17:26:25.131096] [WARN ] [MAIN ] BFZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:25.131313] [INFO ] [MAIN ] BFZ: Wiping existing primary data before full refresh. [2025-09-16T17:26:25.131361] [INFO ] [LOCAL ] BFZ: Starting deletion of primary data. [2025-09-16T17:26:25.131758] [DEBUG] [LOCAL ] BFZ: Deleted primary data file. [2025-09-16T17:26:25.131813] [INFO ] [MAIN ] BFZ: Starting full primary refresh. [2025-09-16T17:26:25.187285] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-08. [2025-09-16T17:26:25.487547] [INFO ] [REMOTE ] BFZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:25.638970] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T17:26:25.974915] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-07. [2025-09-16T17:26:26.595356] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-09-16T17:26:26.750006] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T17:26:26.774860] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-09. [2025-09-16T17:26:27.574883] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-08. [2025-09-16T17:26:28.088484] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T17:26:28.094446] [INFO ] [REMOTE ] BFLY: Full fetch complete (56085 records). [2025-09-16T17:26:28.094560] [DEBUG] [VALIDATOR ] BFLY: Validating primary data. [2025-09-16T17:26:28.095286] [FATAL] [VALIDATOR ] BFLY: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:26:28.095367] [FATAL] [MAIN ] BFLY: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:26:28.095404] [WARN ] [MAIN ] BFLY: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:26:28.095451] [INFO ] [LOCAL ] BFLY: Starting deletion of primary data. [2025-09-16T17:26:28.095536] [DEBUG] [LOCAL ] BFLY: No primary data file found to delete. [2025-09-16T17:26:28.095597] [INFO ] [LOCAL ] BFLY: Starting deletion of derivative data. [2025-09-16T17:26:28.095977] [DEBUG] [LOCAL ] BFLY: Deleted derivative data file. [2025-09-16T17:26:28.096104] [INFO ] [MAIN ] Update Progress: 255/2369 symbols completed. BFLY took 20.3s. Estimated remaining: 3h 39m 0s. Rate: 9.7 symbols/min [2025-09-16T17:26:28.300873] [INFO ] [MAIN ] BG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:28.335850] [DEBUG] [LOCAL ] BG: Loaded 37332 existing records from primary storage. [2025-09-16T17:26:28.335991] [INFO ] [MAIN ] BG: Loaded 37332 existing records. [2025-09-16T17:26:28.380080] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-02. [2025-09-16T17:26:28.513090] [WARN ] [MAIN ] BG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:28.513377] [INFO ] [MAIN ] BG: Wiping existing primary data before full refresh. [2025-09-16T17:26:28.513441] [INFO ] [LOCAL ] BG: Starting deletion of primary data. [2025-09-16T17:26:28.514386] [DEBUG] [LOCAL ] BG: Deleted primary data file. [2025-09-16T17:26:28.514441] [INFO ] [MAIN ] BG: Starting full primary refresh. [2025-09-16T17:26:28.865103] [INFO ] [REMOTE ] BG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:28.885743] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T17:26:29.174914] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-02. [2025-09-16T17:26:29.238438] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T17:26:29.974915] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-09. [2025-09-16T17:26:30.658909] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T17:26:30.774887] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-03. [2025-09-16T17:26:31.067341] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T17:26:31.072380] [INFO ] [REMOTE ] BFH: Full fetch complete (45180 records). [2025-09-16T17:26:31.072506] [DEBUG] [VALIDATOR ] BFH: Validating primary data. [2025-09-16T17:26:31.073213] [FATAL] [VALIDATOR ] BFH: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:26:31.073324] [FATAL] [MAIN ] BFH: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:26:31.073709] [WARN ] [MAIN ] BFH: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:26:31.073955] [INFO ] [LOCAL ] BFH: Starting deletion of primary data. [2025-09-16T17:26:31.074051] [DEBUG] [LOCAL ] BFH: No primary data file found to delete. [2025-09-16T17:26:31.074095] [INFO ] [LOCAL ] BFH: Starting deletion of derivative data. [2025-09-16T17:26:31.074377] [DEBUG] [LOCAL ] BFH: Deleted derivative data file. [2025-09-16T17:26:31.074541] [INFO ] [MAIN ] Update Progress: 256/2369 symbols completed. BFH took 35.3s. Estimated remaining: 3h 38m 27s. Rate: 9.7 symbols/min [2025-09-16T17:26:31.276337] [INFO ] [MAIN ] BGB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:31.292780] [DEBUG] [LOCAL ] BGB: Loaded 12257 existing records from primary storage. [2025-09-16T17:26:31.292963] [INFO ] [MAIN ] BGB: Loaded 12257 existing records. [2025-09-16T17:26:31.469325] [WARN ] [MAIN ] BGB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:31.469558] [INFO ] [MAIN ] BGB: Wiping existing primary data before full refresh. [2025-09-16T17:26:31.469604] [INFO ] [LOCAL ] BGB: Starting deletion of primary data. [2025-09-16T17:26:31.470030] [DEBUG] [LOCAL ] BGB: Deleted primary data file. [2025-09-16T17:26:31.470076] [INFO ] [MAIN ] BGB: Starting full primary refresh. [2025-09-16T17:26:31.580060] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-03. [2025-09-16T17:26:31.829759] [INFO ] [REMOTE ] BGB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:32.161274] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T17:26:32.374905] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-04. [2025-09-16T17:26:32.636394] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-09-16T17:26:32.905839] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-09-16T17:26:33.174913] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-02. [2025-09-16T17:26:33.925164] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T17:26:33.974849] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-03. [2025-09-16T17:26:34.774904] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-05. [2025-09-16T17:26:34.842416] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-09-16T17:26:35.522538] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-09-16T17:26:35.574873] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-04. [2025-09-16T17:26:36.374882] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-04. [2025-09-16T17:26:36.845513] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-09-16T17:26:37.174937] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-06. [2025-09-16T17:26:37.842110] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T17:26:37.974908] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-05. [2025-09-16T17:26:37.994613] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-09-16T17:26:38.724740] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-09-16T17:26:38.774872] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-06. [2025-09-16T17:26:39.574973] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-07. [2025-09-16T17:26:39.666847] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T17:26:40.326373] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-09-16T17:26:40.374892] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-05. [2025-09-16T17:26:41.174970] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-07. [2025-09-16T17:26:41.975010] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-08. [2025-09-16T17:26:42.159892] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-09-16T17:26:42.240361] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-09-16T17:26:42.776985] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-06. [2025-09-16T17:26:42.778041] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-09-16T17:26:43.574937] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-08. [2025-09-16T17:26:44.083918] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T17:26:44.369858] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T17:26:44.374835] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-09. [2025-09-16T17:26:45.174906] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-09. [2025-09-16T17:26:45.783559] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-09-16T17:26:45.785781] [INFO ] [REMOTE ] BFZ: Full fetch complete (12628 records). [2025-09-16T17:26:45.785856] [DEBUG] [VALIDATOR ] BFZ: Validating primary data. [2025-09-16T17:26:45.982697] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-07. [2025-09-16T17:26:46.091272] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-09-16T17:26:46.117602] [INFO ] [REMOTE ] BGB: Full fetch complete (19454 records). [2025-09-16T17:26:46.139962] [DEBUG] [VALIDATOR ] BGB: Validating primary data. [2025-09-16T17:26:46.272175] [DEBUG] [VALIDATOR ] BFZ: All expected trading days are present. [2025-09-16T17:26:46.280263] [DEBUG] [VALIDATOR ] BFZ: Primary data validation passed. [2025-09-16T17:26:46.283088] [INFO ] [MAIN ] BFZ: Fetched primary data is valid. [2025-09-16T17:26:46.283204] [DEBUG] [LOCAL ] BFZ: Saving primary data. [2025-09-16T17:26:46.295034] [DEBUG] [LOCAL ] BFZ: Trimmed 12628 records to 8443 (keeping 150 days of data) [2025-09-16T17:26:46.550588] [DEBUG] [LOCAL ] BFZ: Primary data saved (8443 records). [2025-09-16T17:26:46.574677] [INFO ] [MAIN ] BFZ: Full history saved. [2025-09-16T17:26:46.582855] [INFO ] [MAIN ] Update Progress: 257/2369 symbols completed. BFZ took 21.8s. Estimated remaining: 3h 39m 37s. Rate: 9.6 symbols/min [2025-09-16T17:26:46.683480] [DEBUG] [VALIDATOR ] BGB: All expected trading days are present. [2025-09-16T17:26:46.699383] [DEBUG] [VALIDATOR ] BGB: Primary data validation passed. [2025-09-16T17:26:46.710045] [INFO ] [MAIN ] BGB: Fetched primary data is valid. [2025-09-16T17:26:46.710336] [DEBUG] [LOCAL ] BGB: Saving primary data. [2025-09-16T17:26:46.731559] [DEBUG] [LOCAL ] BGB: Trimmed 19454 records to 12351 (keeping 150 days of data) [2025-09-16T17:26:46.780377] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-08. [2025-09-16T17:26:46.783229] [INFO ] [MAIN ] BGH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:46.854891] [DEBUG] [LOCAL ] BGH: Loaded 12045 existing records from primary storage. [2025-09-16T17:26:46.856852] [INFO ] [MAIN ] BGH: Loaded 12045 existing records. [2025-09-16T17:26:47.043487] [WARN ] [MAIN ] BGH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:47.047616] [DEBUG] [LOCAL ] BGB: Primary data saved (12351 records). [2025-09-16T17:26:47.047796] [INFO ] [MAIN ] BGB: Full history saved. [2025-09-16T17:26:47.048080] [INFO ] [MAIN ] Update Progress: 258/2369 symbols completed. BGB took 16.0s. Estimated remaining: 3h 38m 44s. Rate: 9.7 symbols/min [2025-09-16T17:26:47.047998] [INFO ] [MAIN ] BGH: Wiping existing primary data before full refresh. [2025-09-16T17:26:47.091246] [INFO ] [LOCAL ] BGH: Starting deletion of primary data. [2025-09-16T17:26:47.139015] [DEBUG] [LOCAL ] BGH: Deleted primary data file. [2025-09-16T17:26:47.149355] [INFO ] [MAIN ] BGH: Starting full primary refresh. [2025-09-16T17:26:47.343663] [INFO ] [MAIN ] BGR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:47.523326] [DEBUG] [LOCAL ] BGR: Loaded 9621 existing records from primary storage. [2025-09-16T17:26:47.523929] [INFO ] [MAIN ] BGR: Loaded 9621 existing records. [2025-09-16T17:26:47.697402] [INFO ] [REMOTE ] BGH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:47.714838] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-02. [2025-09-16T17:26:47.811902] [WARN ] [MAIN ] BGR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:47.812769] [INFO ] [MAIN ] BGR: Wiping existing primary data before full refresh. [2025-09-16T17:26:47.813921] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T17:26:47.814007] [INFO ] [LOCAL ] BGR: Starting deletion of primary data. [2025-09-16T17:26:47.817744] [DEBUG] [LOCAL ] BGR: Deleted primary data file. [2025-09-16T17:26:47.817855] [INFO ] [MAIN ] BGR: Starting full primary refresh. [2025-09-16T17:26:48.221170] [INFO ] [REMOTE ] BGR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:48.226256] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-07 (2.24 seconds). [2025-09-16T17:26:48.374853] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-08. [2025-09-16T17:26:48.732611] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T17:26:49.174910] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-09. [2025-09-16T17:26:49.920082] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-09-16T17:26:49.921868] [INFO ] [REMOTE ] BFS: Full fetch complete (9682 records). [2025-09-16T17:26:49.921940] [DEBUG] [VALIDATOR ] BFS: Validating primary data. [2025-09-16T17:26:49.984488] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-02. [2025-09-16T17:26:50.285851] [DEBUG] [VALIDATOR ] BFS: All expected trading days are present. [2025-09-16T17:26:50.286010] [DEBUG] [VALIDATOR ] BFS: Primary data validation passed. [2025-09-16T17:26:50.286289] [INFO ] [MAIN ] BFS: Fetched primary data is valid. [2025-09-16T17:26:50.286342] [DEBUG] [LOCAL ] BFS: Saving primary data. [2025-09-16T17:26:50.287168] [DEBUG] [LOCAL ] BFS: Trimmed 9682 records to 6786 (keeping 150 days of data) [2025-09-16T17:26:50.349691] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-09-16T17:26:50.371495] [DEBUG] [LOCAL ] BFS: Primary data saved (6786 records). [2025-09-16T17:26:50.371619] [INFO ] [MAIN ] BFS: Full history saved. [2025-09-16T17:26:50.371732] [INFO ] [MAIN ] Update Progress: 259/2369 symbols completed. BFS took 40.2s. Estimated remaining: 3h 38m 14s. Rate: 9.7 symbols/min [2025-09-16T17:26:50.541961] [INFO ] [MAIN ] BGS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:50.584228] [DEBUG] [LOCAL ] BGS: Loaded 37320 existing records from primary storage. [2025-09-16T17:26:50.584394] [INFO ] [MAIN ] BGS: Loaded 37320 existing records. [2025-09-16T17:26:50.760701] [WARN ] [MAIN ] BGS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:50.760952] [INFO ] [MAIN ] BGS: Wiping existing primary data before full refresh. [2025-09-16T17:26:50.761001] [INFO ] [LOCAL ] BGS: Starting deletion of primary data. [2025-09-16T17:26:50.761897] [DEBUG] [LOCAL ] BGS: Deleted primary data file. [2025-09-16T17:26:50.761953] [INFO ] [MAIN ] BGS: Starting full primary refresh. [2025-09-16T17:26:50.782461] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-03. [2025-09-16T17:26:51.111894] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-09-16T17:26:51.130668] [INFO ] [REMOTE ] BGS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:51.578342] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-03. [2025-09-16T17:26:51.579227] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T17:26:52.096820] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-09-16T17:26:52.374903] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-04. [2025-09-16T17:26:53.174902] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-02. [2025-09-16T17:26:53.232243] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-09-16T17:26:53.974903] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-04. [2025-09-16T17:26:54.526425] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-09-16T17:26:54.774900] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-09. [2025-09-16T17:26:54.984202] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-09-16T17:26:55.574896] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-05. [2025-09-16T17:26:56.374873] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-05. [2025-09-16T17:26:56.444297] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-09-16T17:26:56.812264] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-09-16T17:26:56.820328] [INFO ] [REMOTE ] BG: Full fetch complete (54772 records). [2025-09-16T17:26:56.820466] [DEBUG] [VALIDATOR ] BG: Validating primary data. [2025-09-16T17:26:57.107439] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T17:26:57.189504] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-06. [2025-09-16T17:26:57.241178] [DEBUG] [VALIDATOR ] BG: All expected trading days are present. [2025-09-16T17:26:57.241254] [DEBUG] [VALIDATOR ] BG: Primary data validation passed. [2025-09-16T17:26:57.241890] [INFO ] [MAIN ] BG: Fetched primary data is valid. [2025-09-16T17:26:57.241946] [DEBUG] [LOCAL ] BG: Saving primary data. [2025-09-16T17:26:57.243395] [DEBUG] [LOCAL ] BG: Trimmed 54772 records to 37699 (keeping 150 days of data) [2025-09-16T17:26:57.482759] [DEBUG] [LOCAL ] BG: Primary data saved (37699 records). [2025-09-16T17:26:57.482909] [INFO ] [MAIN ] BG: Full history saved. [2025-09-16T17:26:57.482992] [INFO ] [MAIN ] Update Progress: 260/2369 symbols completed. BG took 29.4s. Estimated remaining: 3h 38m 15s. Rate: 9.7 symbols/min [2025-09-16T17:26:57.653774] [INFO ] [MAIN ] BGSF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:26:57.660144] [DEBUG] [LOCAL ] BGSF: Loaded 4578 existing records from primary storage. [2025-09-16T17:26:57.660256] [INFO ] [MAIN ] BGSF: Loaded 4578 existing records. [2025-09-16T17:26:57.837784] [WARN ] [MAIN ] BGSF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:26:57.838038] [INFO ] [MAIN ] BGSF: Wiping existing primary data before full refresh. [2025-09-16T17:26:57.838106] [INFO ] [LOCAL ] BGSF: Starting deletion of primary data. [2025-09-16T17:26:57.838340] [DEBUG] [LOCAL ] BGSF: Deleted primary data file. [2025-09-16T17:26:57.838383] [INFO ] [MAIN ] BGSF: Starting full primary refresh. [2025-09-16T17:26:57.981521] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-03. [2025-09-16T17:26:58.197708] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T17:26:58.207392] [INFO ] [REMOTE ] BGSF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:26:58.774915] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-02. [2025-09-16T17:26:59.254330] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-09-16T17:26:59.574867] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-03. [2025-09-16T17:26:59.681653] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-09-16T17:26:59.972450] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-09-16T17:27:00.374911] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-06. [2025-09-16T17:27:01.174897] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-04. [2025-09-16T17:27:01.974914] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-07. [2025-09-16T17:27:02.100050] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T17:27:02.263754] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-09-16T17:27:02.774888] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-07. [2025-09-16T17:27:03.062249] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T17:27:03.574894] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-04. [2025-09-16T17:27:03.638722] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-09-16T17:27:04.374916] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-08. [2025-09-16T17:27:05.174910] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-05. [2025-09-16T17:27:05.210367] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-09-16T17:27:05.332891] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T17:27:05.617391] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-09-16T17:27:05.974901] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-05. [2025-09-16T17:27:06.774944] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-06. [2025-09-16T17:27:07.558821] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T17:27:07.574853] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-07. [2025-09-16T17:27:07.786839] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T17:27:08.328097] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-09-16T17:27:08.374956] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-09. [2025-09-16T17:27:09.153046] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-09-16T17:27:09.155814] [INFO ] [REMOTE ] BGR: Full fetch complete (14905 records). [2025-09-16T17:27:09.156166] [DEBUG] [VALIDATOR ] BGR: Validating primary data. [2025-09-16T17:27:09.180086] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-06. [2025-09-16T17:27:09.518162] [DEBUG] [VALIDATOR ] BGR: All expected trading days are present. [2025-09-16T17:27:09.518242] [DEBUG] [VALIDATOR ] BGR: Primary data validation passed. [2025-09-16T17:27:09.518524] [INFO ] [MAIN ] BGR: Fetched primary data is valid. [2025-09-16T17:27:09.518565] [DEBUG] [LOCAL ] BGR: Saving primary data. [2025-09-16T17:27:09.519168] [DEBUG] [LOCAL ] BGR: Trimmed 14905 records to 9713 (keeping 150 days of data) [2025-09-16T17:27:09.582815] [DEBUG] [LOCAL ] BGR: Primary data saved (9713 records). [2025-09-16T17:27:09.582955] [INFO ] [MAIN ] BGR: Full history saved. [2025-09-16T17:27:09.583047] [INFO ] [MAIN ] Update Progress: 261/2369 symbols completed. BGR took 22.5s. Estimated remaining: 3h 38m 57s. Rate: 9.6 symbols/min [2025-09-16T17:27:09.753605] [INFO ] [MAIN ] BGT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:27:09.766749] [DEBUG] [LOCAL ] BGT: Loaded 12264 existing records from primary storage. [2025-09-16T17:27:09.766880] [INFO ] [MAIN ] BGT: Loaded 12264 existing records. [2025-09-16T17:27:09.941317] [WARN ] [MAIN ] BGT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:27:09.941545] [INFO ] [MAIN ] BGT: Wiping existing primary data before full refresh. [2025-09-16T17:27:09.942372] [INFO ] [LOCAL ] BGT: Starting deletion of primary data. [2025-09-16T17:27:09.942818] [DEBUG] [LOCAL ] BGT: Deleted primary data file. [2025-09-16T17:27:09.942866] [INFO ] [MAIN ] BGT: Starting full primary refresh. [2025-09-16T17:27:09.979960] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-08. [2025-09-16T17:27:10.315671] [INFO ] [REMOTE ] BGT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:27:10.774976] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-08. [2025-09-16T17:27:11.047155] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-09-16T17:27:11.083569] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-09-16T17:27:11.452088] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-09-16T17:27:11.574934] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-09. [2025-09-16T17:27:12.374931] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-09. [2025-09-16T17:27:12.626191] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T17:27:12.628165] [INFO ] [REMOTE ] BGSF: Full fetch complete (5911 records). [2025-09-16T17:27:12.628233] [DEBUG] [VALIDATOR ] BGSF: Validating primary data. [2025-09-16T17:27:12.985505] [DEBUG] [VALIDATOR ] BGSF: All expected trading days are present. [2025-09-16T17:27:12.985580] [DEBUG] [VALIDATOR ] BGSF: Primary data validation passed. [2025-09-16T17:27:12.985757] [INFO ] [MAIN ] BGSF: Fetched primary data is valid. [2025-09-16T17:27:12.985801] [DEBUG] [LOCAL ] BGSF: Saving primary data. [2025-09-16T17:27:12.986183] [DEBUG] [LOCAL ] BGSF: Trimmed 5911 records to 4813 (keeping 150 days of data) [2025-09-16T17:27:13.013346] [DEBUG] [LOCAL ] BGSF: Primary data saved (4813 records). [2025-09-16T17:27:13.013428] [INFO ] [MAIN ] BGSF: Full history saved. [2025-09-16T17:27:13.013503] [INFO ] [MAIN ] Update Progress: 262/2369 symbols completed. BGSF took 15.5s. Estimated remaining: 3h 38m 28s. Rate: 9.6 symbols/min [2025-09-16T17:27:13.181519] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-07. [2025-09-16T17:27:13.183956] [INFO ] [MAIN ] BGX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:27:13.194583] [DEBUG] [LOCAL ] BGX: Loaded 7699 existing records from primary storage. [2025-09-16T17:27:13.194743] [INFO ] [MAIN ] BGX: Loaded 7699 existing records. [2025-09-16T17:27:13.378666] [WARN ] [MAIN ] BGX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:27:13.378931] [INFO ] [MAIN ] BGX: Wiping existing primary data before full refresh. [2025-09-16T17:27:13.378987] [INFO ] [LOCAL ] BGX: Starting deletion of primary data. [2025-09-16T17:27:13.379403] [DEBUG] [LOCAL ] BGX: Deleted primary data file. [2025-09-16T17:27:13.379466] [INFO ] [MAIN ] BGX: Starting full primary refresh. [2025-09-16T17:27:13.671881] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T17:27:13.692456] [INFO ] [REMOTE ] BGH: Full fetch complete (17396 records). [2025-09-16T17:27:13.726926] [DEBUG] [VALIDATOR ] BGH: Validating primary data. [2025-09-16T17:27:13.754983] [INFO ] [REMOTE ] BGX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:27:13.980020] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-02. [2025-09-16T17:27:14.130776] [DEBUG] [VALIDATOR ] BGH: All expected trading days are present. [2025-09-16T17:27:14.130875] [DEBUG] [VALIDATOR ] BGH: Primary data validation passed. [2025-09-16T17:27:14.131184] [INFO ] [MAIN ] BGH: Fetched primary data is valid. [2025-09-16T17:27:14.131223] [DEBUG] [LOCAL ] BGH: Saving primary data. [2025-09-16T17:27:14.131808] [DEBUG] [LOCAL ] BGH: Trimmed 17396 records to 12163 (keeping 150 days of data) [2025-09-16T17:27:14.207934] [DEBUG] [LOCAL ] BGH: Primary data saved (12163 records). [2025-09-16T17:27:14.208074] [INFO ] [MAIN ] BGH: Full history saved. [2025-09-16T17:27:14.208157] [INFO ] [MAIN ] Update Progress: 263/2369 symbols completed. BGH took 27.6s. Estimated remaining: 3h 37m 41s. Rate: 9.7 symbols/min [2025-09-16T17:27:14.379792] [INFO ] [MAIN ] BGY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:27:14.396455] [DEBUG] [LOCAL ] BGY: Loaded 16276 existing records from primary storage. [2025-09-16T17:27:14.396578] [INFO ] [MAIN ] BGY: Loaded 16276 existing records. [2025-09-16T17:27:14.584092] [WARN ] [MAIN ] BGY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:27:14.584543] [INFO ] [MAIN ] BGY: Wiping existing primary data before full refresh. [2025-09-16T17:27:14.584690] [INFO ] [LOCAL ] BGY: Starting deletion of primary data. [2025-09-16T17:27:14.585304] [DEBUG] [LOCAL ] BGY: Deleted primary data file. [2025-09-16T17:27:14.585390] [INFO ] [MAIN ] BGY: Starting full primary refresh. [2025-09-16T17:27:14.794850] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-02. [2025-09-16T17:27:14.875686] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T17:27:15.004627] [INFO ] [REMOTE ] BGY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:27:15.166567] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T17:27:15.497572] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-09-16T17:27:15.574909] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-03. [2025-09-16T17:27:16.374908] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-02. [2025-09-16T17:27:16.543456] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-09-16T17:27:17.174949] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-04. [2025-09-16T17:27:17.467293] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T17:27:17.776152] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-09-16T17:27:17.974913] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-03. [2025-09-16T17:27:18.774946] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-05. [2025-09-16T17:27:19.134248] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T17:27:19.455135] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-09-16T17:27:19.574914] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-06. [2025-09-16T17:27:20.374945] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-04. [2025-09-16T17:27:20.575378] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T17:27:21.148844] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-09-16T17:27:21.174952] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-03. [2025-09-16T17:27:21.759384] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T17:27:21.974940] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-04. [2025-09-16T17:27:22.757441] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-09-16T17:27:22.774897] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-05. [2025-09-16T17:27:23.420754] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-09-16T17:27:23.574924] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-08. [2025-09-16T17:27:24.374906] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-06. [2025-09-16T17:27:24.893581] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-09-16T17:27:25.108591] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T17:27:25.174892] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-09. [2025-09-16T17:27:25.974936] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-05. [2025-09-16T17:27:26.745611] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-09-16T17:27:26.772275] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T17:27:26.777863] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-07. [2025-09-16T17:27:26.779349] [INFO ] [REMOTE ] BGS: Full fetch complete (54816 records). [2025-09-16T17:27:26.779441] [DEBUG] [VALIDATOR ] BGS: Validating primary data. [2025-09-16T17:27:27.157161] [DEBUG] [VALIDATOR ] BGS: All expected trading days are present. [2025-09-16T17:27:27.157241] [DEBUG] [VALIDATOR ] BGS: Primary data validation passed. [2025-09-16T17:27:27.157842] [INFO ] [MAIN ] BGS: Fetched primary data is valid. [2025-09-16T17:27:27.157888] [DEBUG] [LOCAL ] BGS: Saving primary data. [2025-09-16T17:27:27.159033] [DEBUG] [LOCAL ] BGS: Trimmed 54816 records to 37704 (keeping 150 days of data) [2025-09-16T17:27:27.390230] [DEBUG] [LOCAL ] BGS: Primary data saved (37704 records). [2025-09-16T17:27:27.390397] [INFO ] [MAIN ] BGS: Full history saved. [2025-09-16T17:27:27.392095] [INFO ] [MAIN ] Update Progress: 264/2369 symbols completed. BGS took 37.0s. Estimated remaining: 3h 38m 31s. Rate: 9.6 symbols/min [2025-09-16T17:27:27.561436] [INFO ] [MAIN ] BH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:27:27.561639] [DEBUG] [LOCAL ] BH: No existing data file found in primary storage. [2025-09-16T17:27:27.561696] [WARN ] [MAIN ] BH: No existing data found, performing full refresh instead. [2025-09-16T17:27:27.561816] [INFO ] [MAIN ] BH: Wiping existing primary data before full refresh. [2025-09-16T17:27:27.561855] [INFO ] [LOCAL ] BH: Starting deletion of primary data. [2025-09-16T17:27:27.561895] [DEBUG] [LOCAL ] BH: No primary data file found to delete. [2025-09-16T17:27:27.561928] [INFO ] [MAIN ] BH: Starting full primary refresh. [2025-09-16T17:27:27.582196] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-06. [2025-09-16T17:27:27.926490] [INFO ] [REMOTE ] BH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:27:27.974031] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T17:27:28.374897] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-02. [2025-09-16T17:27:28.593787] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T17:27:29.132810] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T17:27:29.174938] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-08. [2025-09-16T17:27:29.974924] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-03. [2025-09-16T17:27:30.042533] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-09-16T17:27:30.656387] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-09-16T17:27:30.774871] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-04. [2025-09-16T17:27:31.449969] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T17:27:31.574884] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-09. [2025-09-16T17:27:32.374912] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-07. [2025-09-16T17:27:32.460519] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T17:27:32.462925] [INFO ] [REMOTE ] BGT: Full fetch complete (17989 records). [2025-09-16T17:27:32.462997] [DEBUG] [VALIDATOR ] BGT: Validating primary data. [2025-09-16T17:27:32.855977] [DEBUG] [VALIDATOR ] BGT: All expected trading days are present. [2025-09-16T17:27:32.856055] [DEBUG] [VALIDATOR ] BGT: Primary data validation passed. [2025-09-16T17:27:32.856381] [INFO ] [MAIN ] BGT: Fetched primary data is valid. [2025-09-16T17:27:32.856423] [DEBUG] [LOCAL ] BGT: Saving primary data. [2025-09-16T17:27:32.857007] [DEBUG] [LOCAL ] BGT: Trimmed 17989 records to 12384 (keeping 150 days of data) [2025-09-16T17:27:32.933664] [DEBUG] [LOCAL ] BGT: Primary data saved (12384 records). [2025-09-16T17:27:32.933838] [INFO ] [MAIN ] BGT: Full history saved. [2025-09-16T17:27:32.933925] [INFO ] [MAIN ] Update Progress: 265/2369 symbols completed. BGT took 23.4s. Estimated remaining: 3h 38m 19s. Rate: 9.6 symbols/min [2025-09-16T17:27:33.102054] [INFO ] [MAIN ] BHC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:27:33.137210] [DEBUG] [LOCAL ] BHC: Loaded 34175 existing records from primary storage. [2025-09-16T17:27:33.137342] [INFO ] [MAIN ] BHC: Loaded 34175 existing records. [2025-09-16T17:27:33.180020] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-05. [2025-09-16T17:27:33.313946] [WARN ] [MAIN ] BHC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:27:33.314190] [INFO ] [MAIN ] BHC: Wiping existing primary data before full refresh. [2025-09-16T17:27:33.314234] [INFO ] [LOCAL ] BHC: Starting deletion of primary data. [2025-09-16T17:27:33.315024] [DEBUG] [LOCAL ] BHC: Deleted primary data file. [2025-09-16T17:27:33.315071] [INFO ] [MAIN ] BHC: Starting full primary refresh. [2025-09-16T17:27:33.682453] [INFO ] [REMOTE ] BHC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:27:33.685380] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T17:27:33.974904] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-02. [2025-09-16T17:27:34.170163] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T17:27:34.774909] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-07. [2025-09-16T17:27:35.574922] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-06. [2025-09-16T17:27:35.651391] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-09-16T17:27:35.651101] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T17:27:36.374917] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-08. [2025-09-16T17:27:36.535760] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-09-16T17:27:37.135779] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-09-16T17:27:37.174915] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-08. [2025-09-16T17:27:37.974871] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-07. [2025-09-16T17:27:38.247788] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T17:27:38.752578] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-09-16T17:27:38.774853] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-08. [2025-09-16T17:27:39.574914] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-09. [2025-09-16T17:27:39.937244] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T17:27:40.328549] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-09-16T17:27:40.331234] [INFO ] [REMOTE ] BGY: Full fetch complete (25251 records). [2025-09-16T17:27:40.331355] [DEBUG] [VALIDATOR ] BGY: Validating primary data. [2025-09-16T17:27:40.379976] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-09. [2025-09-16T17:27:40.715949] [DEBUG] [VALIDATOR ] BGY: All expected trading days are present. [2025-09-16T17:27:40.716059] [DEBUG] [VALIDATOR ] BGY: Primary data validation passed. [2025-09-16T17:27:40.716460] [INFO ] [MAIN ] BGY: Fetched primary data is valid. [2025-09-16T17:27:40.716500] [DEBUG] [LOCAL ] BGY: Saving primary data. [2025-09-16T17:27:40.717194] [DEBUG] [LOCAL ] BGY: Trimmed 25251 records to 16395 (keeping 150 days of data) [2025-09-16T17:27:40.813857] [DEBUG] [LOCAL ] BGY: Primary data saved (16395 records). [2025-09-16T17:27:40.814007] [INFO ] [MAIN ] BGY: Full history saved. [2025-09-16T17:27:40.814091] [INFO ] [MAIN ] Update Progress: 266/2369 symbols completed. BGY took 26.6s. Estimated remaining: 3h 38m 26s. Rate: 9.6 symbols/min [2025-09-16T17:27:40.988397] [INFO ] [MAIN ] BHE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:27:41.005895] [DEBUG] [LOCAL ] BHE: Loaded 17441 existing records from primary storage. [2025-09-16T17:27:41.006048] [INFO ] [MAIN ] BHE: Loaded 17441 existing records. [2025-09-16T17:27:41.184743] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-03. [2025-09-16T17:27:41.190469] [WARN ] [MAIN ] BHE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:27:41.191383] [INFO ] [MAIN ] BHE: Wiping existing primary data before full refresh. [2025-09-16T17:27:41.191461] [INFO ] [LOCAL ] BHE: Starting deletion of primary data. [2025-09-16T17:27:41.192153] [DEBUG] [LOCAL ] BHE: Deleted primary data file. [2025-09-16T17:27:41.192214] [INFO ] [MAIN ] BHE: Starting full primary refresh. [2025-09-16T17:27:41.493957] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T17:27:41.540740] [INFO ] [REMOTE ] BGX: Full fetch complete (11921 records). [2025-09-16T17:27:41.569355] [DEBUG] [VALIDATOR ] BGX: Validating primary data. [2025-09-16T17:27:41.569257] [INFO ] [REMOTE ] BHE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:27:41.931870] [DEBUG] [VALIDATOR ] BGX: All expected trading days are present. [2025-09-16T17:27:41.931948] [DEBUG] [VALIDATOR ] BGX: Primary data validation passed. [2025-09-16T17:27:41.932241] [INFO ] [MAIN ] BGX: Fetched primary data is valid. [2025-09-16T17:27:41.932283] [DEBUG] [LOCAL ] BGX: Saving primary data. [2025-09-16T17:27:41.932777] [DEBUG] [LOCAL ] BGX: Trimmed 11921 records to 7749 (keeping 150 days of data) [2025-09-16T17:27:41.976955] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-09. [2025-09-16T17:27:41.982074] [DEBUG] [LOCAL ] BGX: Primary data saved (7749 records). [2025-09-16T17:27:41.982239] [INFO ] [MAIN ] BGX: Full history saved. [2025-09-16T17:27:41.982329] [INFO ] [MAIN ] Update Progress: 267/2369 symbols completed. BGX took 29.0s. Estimated remaining: 3h 37m 40s. Rate: 9.7 symbols/min [2025-09-16T17:27:42.155805] [INFO ] [MAIN ] BHK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:27:42.173536] [DEBUG] [LOCAL ] BHK: Loaded 18311 existing records from primary storage. [2025-09-16T17:27:42.173701] [INFO ] [MAIN ] BHK: Loaded 18311 existing records. [2025-09-16T17:27:42.360554] [WARN ] [MAIN ] BHK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:27:42.360840] [INFO ] [MAIN ] BHK: Wiping existing primary data before full refresh. [2025-09-16T17:27:42.360886] [INFO ] [LOCAL ] BHK: Starting deletion of primary data. [2025-09-16T17:27:42.361396] [DEBUG] [LOCAL ] BHK: Deleted primary data file. [2025-09-16T17:27:42.361441] [INFO ] [MAIN ] BHK: Starting full primary refresh. [2025-09-16T17:27:42.738596] [INFO ] [REMOTE ] BHK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:27:42.766427] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-09-16T17:27:42.776056] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-02. [2025-09-16T17:27:42.924676] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T17:27:42.927517] [INFO ] [REMOTE ] BH: Full fetch complete (15656 records). [2025-09-16T17:27:42.927610] [DEBUG] [VALIDATOR ] BH: Validating primary data. [2025-09-16T17:27:43.300266] [DEBUG] [VALIDATOR ] BH: All expected trading days are present. [2025-09-16T17:27:43.300378] [DEBUG] [VALIDATOR ] BH: Primary data validation passed. [2025-09-16T17:27:43.300730] [INFO ] [MAIN ] BH: Fetched primary data is valid. [2025-09-16T17:27:43.300774] [DEBUG] [LOCAL ] BH: Saving primary data. [2025-09-16T17:27:43.301309] [DEBUG] [LOCAL ] BH: Trimmed 15656 records to 13261 (keeping 150 days of data) [2025-09-16T17:27:43.380369] [DEBUG] [LOCAL ] BH: Primary data saved (13261 records). [2025-09-16T17:27:43.380517] [INFO ] [MAIN ] BH: Full history saved. [2025-09-16T17:27:43.380595] [INFO ] [MAIN ] Update Progress: 268/2369 symbols completed. BH took 16.0s. Estimated remaining: 3h 36m 56s. Rate: 9.7 symbols/min [2025-09-16T17:27:43.558034] [INFO ] [MAIN ] BHP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:27:43.582551] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-04. [2025-09-16T17:27:43.600514] [DEBUG] [LOCAL ] BHP: Loaded 39376 existing records from primary storage. [2025-09-16T17:27:43.600687] [INFO ] [MAIN ] BHP: Loaded 39376 existing records. [2025-09-16T17:27:43.782144] [WARN ] [MAIN ] BHP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:27:43.782589] [INFO ] [MAIN ] BHP: Wiping existing primary data before full refresh. [2025-09-16T17:27:43.782742] [INFO ] [LOCAL ] BHP: Starting deletion of primary data. [2025-09-16T17:27:43.790003] [DEBUG] [LOCAL ] BHP: Deleted primary data file. [2025-09-16T17:27:43.791554] [INFO ] [MAIN ] BHP: Starting full primary refresh. [2025-09-16T17:27:43.825539] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-09-16T17:27:44.157751] [INFO ] [REMOTE ] BHP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:27:44.374912] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-03. [2025-09-16T17:27:44.942045] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-09-16T17:27:45.174929] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-04. [2025-09-16T17:27:45.424285] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T17:27:45.974913] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-02. [2025-09-16T17:27:46.225902] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-09-16T17:27:46.774908] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-05. [2025-09-16T17:27:47.574876] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-02. [2025-09-16T17:27:47.607321] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T17:27:47.827054] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-09-16T17:27:48.374899] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-05. [2025-09-16T17:27:48.862175] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T17:27:49.174915] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-06. [2025-09-16T17:27:49.974912] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-03. [2025-09-16T17:27:50.008678] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-09-16T17:27:50.033772] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-09-16T17:27:50.774906] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-06. [2025-09-16T17:27:51.574942] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-07. [2025-09-16T17:27:52.175368] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-03 (2.20 seconds). [2025-09-16T17:27:52.374855] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-04. [2025-09-16T17:27:52.581194] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-09-16T17:27:52.755131] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-09-16T17:27:53.174870] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-07. [2025-09-16T17:27:53.928072] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-09-16T17:27:53.974855] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-05. [2025-09-16T17:27:54.774875] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-08. [2025-09-16T17:27:54.878564] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T17:27:55.504807] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T17:27:55.574858] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-06. [2025-09-16T17:27:55.903257] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T17:27:56.374932] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-08. [2025-09-16T17:27:57.158502] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T17:27:57.174851] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-07. [2025-09-16T17:27:57.974903] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-09. [2025-09-16T17:27:58.051113] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T17:27:58.774932] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-09. [2025-09-16T17:27:59.081205] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T17:27:59.085218] [INFO ] [REMOTE ] BHK: Full fetch complete (23529 records). [2025-09-16T17:27:59.085330] [DEBUG] [VALIDATOR ] BHK: Validating primary data. [2025-09-16T17:27:59.303401] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-09-16T17:27:59.502615] [DEBUG] [VALIDATOR ] BHK: All expected trading days are present. [2025-09-16T17:27:59.502708] [DEBUG] [VALIDATOR ] BHK: Primary data validation passed. [2025-09-16T17:27:59.502982] [INFO ] [MAIN ] BHK: Fetched primary data is valid. [2025-09-16T17:27:59.503023] [DEBUG] [LOCAL ] BHK: Saving primary data. [2025-09-16T17:27:59.503623] [DEBUG] [LOCAL ] BHK: Trimmed 23529 records to 18457 (keeping 150 days of data) [2025-09-16T17:27:59.586208] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-08. [2025-09-16T17:27:59.618724] [DEBUG] [LOCAL ] BHK: Primary data saved (18457 records). [2025-09-16T17:27:59.618852] [INFO ] [MAIN ] BHK: Full history saved. [2025-09-16T17:27:59.618936] [INFO ] [MAIN ] Update Progress: 269/2369 symbols completed. BHK took 17.6s. Estimated remaining: 3h 38m 8s. Rate: 9.6 symbols/min [2025-09-16T17:27:59.791036] [INFO ] [MAIN ] BHR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:27:59.806233] [DEBUG] [LOCAL ] BHR: Loaded 15154 existing records from primary storage. [2025-09-16T17:27:59.806356] [INFO ] [MAIN ] BHR: Loaded 15154 existing records. [2025-09-16T17:27:59.979518] [WARN ] [MAIN ] BHR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:27:59.979767] [INFO ] [MAIN ] BHR: Wiping existing primary data before full refresh. [2025-09-16T17:27:59.979821] [INFO ] [LOCAL ] BHR: Starting deletion of primary data. [2025-09-16T17:27:59.980380] [DEBUG] [LOCAL ] BHR: Deleted primary data file. [2025-09-16T17:27:59.980442] [INFO ] [MAIN ] BHR: Starting full primary refresh. [2025-09-16T17:28:00.330332] [INFO ] [REMOTE ] BHR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:28:00.346448] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T17:28:00.352091] [INFO ] [REMOTE ] BHC: Full fetch complete (50966 records). [2025-09-16T17:28:00.352203] [DEBUG] [VALIDATOR ] BHC: Validating primary data. [2025-09-16T17:28:00.383794] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-02. [2025-09-16T17:28:00.744464] [DEBUG] [VALIDATOR ] BHC: All expected trading days are present. [2025-09-16T17:28:00.744580] [DEBUG] [VALIDATOR ] BHC: Primary data validation passed. [2025-09-16T17:28:00.745371] [INFO ] [MAIN ] BHC: Fetched primary data is valid. [2025-09-16T17:28:00.745432] [DEBUG] [LOCAL ] BHC: Saving primary data. [2025-09-16T17:28:00.747001] [DEBUG] [LOCAL ] BHC: Trimmed 50966 records to 34553 (keeping 150 days of data) [2025-09-16T17:28:00.948528] [DEBUG] [LOCAL ] BHC: Primary data saved (34553 records). [2025-09-16T17:28:00.948700] [INFO ] [MAIN ] BHC: Full history saved. [2025-09-16T17:28:00.948809] [INFO ] [MAIN ] Update Progress: 270/2369 symbols completed. BHC took 28.0s. Estimated remaining: 3h 37m 24s. Rate: 9.7 symbols/min [2025-09-16T17:28:01.122257] [INFO ] [MAIN ] BHV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:28:01.122434] [DEBUG] [LOCAL ] BHV: No existing data file found in primary storage. [2025-09-16T17:28:01.122473] [WARN ] [MAIN ] BHV: No existing data found, performing full refresh instead. [2025-09-16T17:28:01.122600] [INFO ] [MAIN ] BHV: Wiping existing primary data before full refresh. [2025-09-16T17:28:01.122658] [INFO ] [LOCAL ] BHV: Starting deletion of primary data. [2025-09-16T17:28:01.122706] [DEBUG] [LOCAL ] BHV: No primary data file found to delete. [2025-09-16T17:28:01.122740] [INFO ] [MAIN ] BHV: Starting full primary refresh. [2025-09-16T17:28:01.187102] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-03. [2025-09-16T17:28:01.494556] [INFO ] [REMOTE ] BHV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:28:01.864111] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-08 (2.28 seconds). [2025-09-16T17:28:01.901485] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T17:28:01.974898] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-03. [2025-09-16T17:28:02.774879] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-09. [2025-09-16T17:28:02.858542] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T17:28:03.125405] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T17:28:03.575003] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-02. [2025-09-16T17:28:04.032610] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-09-16T17:28:04.374958] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-04. [2025-09-16T17:28:04.634804] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-09-16T17:28:04.641135] [INFO ] [REMOTE ] BHP: Full fetch complete (58045 records). [2025-09-16T17:28:04.641277] [DEBUG] [VALIDATOR ] BHP: Validating primary data. [2025-09-16T17:28:05.029741] [DEBUG] [VALIDATOR ] BHP: All expected trading days are present. [2025-09-16T17:28:05.029837] [DEBUG] [VALIDATOR ] BHP: Primary data validation passed. [2025-09-16T17:28:05.030500] [INFO ] [MAIN ] BHP: Fetched primary data is valid. [2025-09-16T17:28:05.030551] [DEBUG] [LOCAL ] BHP: Saving primary data. [2025-09-16T17:28:05.031809] [DEBUG] [LOCAL ] BHP: Trimmed 58045 records to 39761 (keeping 150 days of data) [2025-09-16T17:28:05.212285] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-04. [2025-09-16T17:28:05.284408] [DEBUG] [LOCAL ] BHP: Primary data saved (39761 records). [2025-09-16T17:28:05.284595] [INFO ] [MAIN ] BHP: Full history saved. [2025-09-16T17:28:05.285407] [INFO ] [MAIN ] Update Progress: 271/2369 symbols completed. BHP took 21.9s. Estimated remaining: 3h 37m 3s. Rate: 9.7 symbols/min [2025-09-16T17:28:05.459506] [INFO ] [MAIN ] BHVN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:28:05.498493] [DEBUG] [LOCAL ] BHVN: Loaded 37135 existing records from primary storage. [2025-09-16T17:28:05.498657] [INFO ] [MAIN ] BHVN: Loaded 37135 existing records. [2025-09-16T17:28:05.684168] [WARN ] [MAIN ] BHVN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:28:05.684511] [INFO ] [MAIN ] BHVN: Wiping existing primary data before full refresh. [2025-09-16T17:28:05.684665] [INFO ] [LOCAL ] BHVN: Starting deletion of primary data. [2025-09-16T17:28:05.685545] [DEBUG] [LOCAL ] BHVN: Deleted primary data file. [2025-09-16T17:28:05.685602] [INFO ] [MAIN ] BHVN: Starting full primary refresh. [2025-09-16T17:28:05.875439] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T17:28:05.975229] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-05. [2025-09-16T17:28:06.086376] [INFO ] [REMOTE ] BHVN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:28:06.256571] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T17:28:06.774931] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-03. [2025-09-16T17:28:07.256088] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-09-16T17:28:07.276688] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T17:28:07.574901] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-04. [2025-09-16T17:28:07.905369] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-09-16T17:28:08.374905] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-05. [2025-09-16T17:28:08.803315] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-09-16T17:28:09.174928] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-06. [2025-09-16T17:28:09.510208] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-09-16T17:28:09.974926] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-07. [2025-09-16T17:28:10.346070] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-09-16T17:28:10.774906] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-06. [2025-09-16T17:28:11.574873] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-08. [2025-09-16T17:28:11.893838] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T17:28:11.985420] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-09-16T17:28:12.374950] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-07. [2025-09-16T17:28:13.174918] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-02. [2025-09-16T17:28:13.461696] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T17:28:13.974908] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-08. [2025-09-16T17:28:14.774896] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-09. [2025-09-16T17:28:14.813433] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T17:28:15.107961] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T17:28:15.460823] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-09-16T17:28:15.461710] [INFO ] [REMOTE ] BHV: Full fetch complete (1085 records). [2025-09-16T17:28:15.461840] [DEBUG] [VALIDATOR ] BHV: Validating primary data. [2025-09-16T17:28:15.579977] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-05. [2025-09-16T17:28:15.653830] [WARN ] [VALIDATOR ] BHV: Validation failed - insufficient historical data. Have 147 unique days, need 150 days [2025-09-16T17:28:15.653996] [FATAL] [MAIN ] BHV: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:28:15.654045] [WARN ] [MAIN ] BHV: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:28:15.654091] [INFO ] [LOCAL ] BHV: Starting deletion of primary data. [2025-09-16T17:28:15.654151] [DEBUG] [LOCAL ] BHV: No primary data file found to delete. [2025-09-16T17:28:15.654201] [INFO ] [LOCAL ] BHV: Starting deletion of derivative data. [2025-09-16T17:28:15.654256] [DEBUG] [LOCAL ] BHV: No derivative data file found to delete. [2025-09-16T17:28:15.654336] [INFO ] [MAIN ] Update Progress: 272/2369 symbols completed. BHV took 14.7s. Estimated remaining: 3h 37m 29s. Rate: 9.6 symbols/min [2025-09-16T17:28:15.838209] [INFO ] [MAIN ] BILL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:28:15.874550] [DEBUG] [LOCAL ] BILL: Loaded 38625 existing records from primary storage. [2025-09-16T17:28:15.874727] [INFO ] [MAIN ] BILL: Loaded 38625 existing records. [2025-09-16T17:28:16.065396] [WARN ] [MAIN ] BILL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:28:16.065694] [INFO ] [MAIN ] BILL: Wiping existing primary data before full refresh. [2025-09-16T17:28:16.065752] [INFO ] [LOCAL ] BILL: Starting deletion of primary data. [2025-09-16T17:28:16.066590] [DEBUG] [LOCAL ] BILL: Deleted primary data file. [2025-09-16T17:28:16.066638] [INFO ] [MAIN ] BILL: Starting full primary refresh. [2025-09-16T17:28:16.379993] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-09. [2025-09-16T17:28:16.448718] [INFO ] [REMOTE ] BILL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:28:16.732860] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T17:28:17.148891] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-09-16T17:28:17.151793] [INFO ] [REMOTE ] BHE: Full fetch complete (27965 records). [2025-09-16T17:28:17.151869] [DEBUG] [VALIDATOR ] BHE: Validating primary data. [2025-09-16T17:28:17.181333] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-02. [2025-09-16T17:28:17.556454] [DEBUG] [VALIDATOR ] BHE: All expected trading days are present. [2025-09-16T17:28:17.556556] [DEBUG] [VALIDATOR ] BHE: Primary data validation passed. [2025-09-16T17:28:17.556955] [INFO ] [MAIN ] BHE: Fetched primary data is valid. [2025-09-16T17:28:17.556999] [DEBUG] [LOCAL ] BHE: Saving primary data. [2025-09-16T17:28:17.557707] [DEBUG] [LOCAL ] BHE: Trimmed 27965 records to 17617 (keeping 150 days of data) [2025-09-16T17:28:17.662898] [DEBUG] [LOCAL ] BHE: Primary data saved (17617 records). [2025-09-16T17:28:17.663040] [INFO ] [MAIN ] BHE: Full history saved. [2025-09-16T17:28:17.663125] [INFO ] [MAIN ] Update Progress: 273/2369 symbols completed. BHE took 36.8s. Estimated remaining: 3h 36m 50s. Rate: 9.7 symbols/min [2025-09-16T17:28:17.831405] [INFO ] [MAIN ] BIO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:28:17.849389] [DEBUG] [LOCAL ] BIO: Loaded 18273 existing records from primary storage. [2025-09-16T17:28:17.849509] [INFO ] [MAIN ] BIO: Loaded 18273 existing records. [2025-09-16T17:28:17.980022] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-03. [2025-09-16T17:28:18.033235] [WARN ] [MAIN ] BIO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:28:18.033502] [INFO ] [MAIN ] BIO: Wiping existing primary data before full refresh. [2025-09-16T17:28:18.033548] [INFO ] [LOCAL ] BIO: Starting deletion of primary data. [2025-09-16T17:28:18.034128] [DEBUG] [LOCAL ] BIO: Deleted primary data file. [2025-09-16T17:28:18.034178] [INFO ] [MAIN ] BIO: Starting full primary refresh. [2025-09-16T17:28:18.392011] [INFO ] [REMOTE ] BIO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:28:18.774882] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-06. [2025-09-16T17:28:19.017669] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-09-16T17:28:19.574927] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-02. [2025-09-16T17:28:19.779858] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T17:28:19.831142] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-09-16T17:28:20.374922] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-07. [2025-09-16T17:28:20.583914] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-09-16T17:28:21.133324] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-09-16T17:28:21.174853] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-08. [2025-09-16T17:28:21.974870] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-04. [2025-09-16T17:28:22.180595] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T17:28:22.774927] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-09. [2025-09-16T17:28:23.574879] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-03. [2025-09-16T17:28:23.696131] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T17:28:24.045947] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T17:28:24.049611] [INFO ] [REMOTE ] BHR: Full fetch complete (22674 records). [2025-09-16T17:28:24.049743] [DEBUG] [VALIDATOR ] BHR: Validating primary data. [2025-09-16T17:28:24.379468] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-03. [2025-09-16T17:28:24.419429] [DEBUG] [VALIDATOR ] BHR: All expected trading days are present. [2025-09-16T17:28:24.419515] [DEBUG] [VALIDATOR ] BHR: Primary data validation passed. [2025-09-16T17:28:24.419922] [INFO ] [MAIN ] BHR: Fetched primary data is valid. [2025-09-16T17:28:24.419968] [DEBUG] [LOCAL ] BHR: Saving primary data. [2025-09-16T17:28:24.420630] [DEBUG] [LOCAL ] BHR: Trimmed 22674 records to 15316 (keeping 150 days of data) [2025-09-16T17:28:24.508763] [DEBUG] [LOCAL ] BHR: Primary data saved (15316 records). [2025-09-16T17:28:24.508922] [INFO ] [MAIN ] BHR: Full history saved. [2025-09-16T17:28:24.509010] [INFO ] [MAIN ] Update Progress: 274/2369 symbols completed. BHR took 24.9s. Estimated remaining: 3h 36m 49s. Rate: 9.7 symbols/min [2025-09-16T17:28:24.683159] [INFO ] [MAIN ] BIP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:28:24.717078] [DEBUG] [LOCAL ] BIP: Loaded 27810 existing records from primary storage. [2025-09-16T17:28:24.717228] [INFO ] [MAIN ] BIP: Loaded 27810 existing records. [2025-09-16T17:28:24.897914] [WARN ] [MAIN ] BIP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:28:24.898208] [INFO ] [MAIN ] BIP: Wiping existing primary data before full refresh. [2025-09-16T17:28:24.898287] [INFO ] [LOCAL ] BIP: Starting deletion of primary data. [2025-09-16T17:28:24.899089] [DEBUG] [LOCAL ] BIP: Deleted primary data file. [2025-09-16T17:28:24.899142] [INFO ] [MAIN ] BIP: Starting full primary refresh. [2025-09-16T17:28:25.176765] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-05. [2025-09-16T17:28:25.275258] [INFO ] [REMOTE ] BIP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:28:25.377093] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T17:28:25.696847] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-09-16T17:28:25.974920] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-04. [2025-09-16T17:28:26.774928] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-02. [2025-09-16T17:28:26.988303] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T17:28:27.303129] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T17:28:27.574960] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-06. [2025-09-16T17:28:27.891600] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-09-16T17:28:28.374911] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-03. [2025-09-16T17:28:29.174921] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-04. [2025-09-16T17:28:29.238247] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T17:28:29.985934] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-07. [2025-09-16T17:28:30.009542] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T17:28:30.774912] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-04. [2025-09-16T17:28:30.826819] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T17:28:31.347628] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-09-16T17:28:31.575032] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-05. [2025-09-16T17:28:32.177092] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T17:28:32.374900] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-05. [2025-09-16T17:28:32.826539] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T17:28:33.174930] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-06. [2025-09-16T17:28:33.974916] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-08. [2025-09-16T17:28:34.177864] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-09-16T17:28:34.464067] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T17:28:34.774919] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-07. [2025-09-16T17:28:35.574919] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-05. [2025-09-16T17:28:35.635920] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-09-16T17:28:35.670992] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-09-16T17:28:36.374930] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-08. [2025-09-16T17:28:37.009322] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-09-16T17:28:37.174872] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-06. [2025-09-16T17:28:37.230374] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T17:28:37.974924] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-09. [2025-09-16T17:28:38.299814] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T17:28:38.774907] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-06. [2025-09-16T17:28:39.574940] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-07. [2025-09-16T17:28:39.754468] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T17:28:39.760391] [INFO ] [REMOTE ] BHVN: Full fetch complete (53279 records). [2025-09-16T17:28:39.760498] [DEBUG] [VALIDATOR ] BHVN: Validating primary data. [2025-09-16T17:28:40.168308] [DEBUG] [VALIDATOR ] BHVN: All expected trading days are present. [2025-09-16T17:28:40.168442] [DEBUG] [VALIDATOR ] BHVN: Primary data validation passed. [2025-09-16T17:28:40.169862] [INFO ] [MAIN ] BHVN: Fetched primary data is valid. [2025-09-16T17:28:40.169945] [DEBUG] [LOCAL ] BHVN: Saving primary data. [2025-09-16T17:28:40.172860] [DEBUG] [LOCAL ] BHVN: Trimmed 53279 records to 37493 (keeping 150 days of data) [2025-09-16T17:28:40.376461] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-09. [2025-09-16T17:28:40.423448] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T17:28:40.449403] [DEBUG] [LOCAL ] BHVN: Primary data saved (37493 records). [2025-09-16T17:28:40.449533] [INFO ] [MAIN ] BHVN: Full history saved. [2025-09-16T17:28:40.450452] [INFO ] [MAIN ] Update Progress: 275/2369 symbols completed. BHVN took 35.2s. Estimated remaining: 3h 37m 57s. Rate: 9.6 symbols/min [2025-09-16T17:28:40.631268] [INFO ] [MAIN ] BIPC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:28:40.664305] [DEBUG] [LOCAL ] BIPC: Loaded 29926 existing records from primary storage. [2025-09-16T17:28:40.664442] [INFO ] [MAIN ] BIPC: Loaded 29926 existing records. [2025-09-16T17:28:40.846222] [WARN ] [MAIN ] BIPC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:28:40.846465] [INFO ] [MAIN ] BIPC: Wiping existing primary data before full refresh. [2025-09-16T17:28:40.846520] [INFO ] [LOCAL ] BIPC: Starting deletion of primary data. [2025-09-16T17:28:40.847375] [DEBUG] [LOCAL ] BIPC: Deleted primary data file. [2025-09-16T17:28:40.847431] [INFO ] [MAIN ] BIPC: Starting full primary refresh. [2025-09-16T17:28:41.177871] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-07. [2025-09-16T17:28:41.236658] [INFO ] [REMOTE ] BIPC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:28:41.264827] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T17:28:41.281556] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-09-16T17:28:41.284386] [INFO ] [REMOTE ] BIO: Full fetch complete (28282 records). [2025-09-16T17:28:41.284476] [DEBUG] [VALIDATOR ] BIO: Validating primary data. [2025-09-16T17:28:41.658850] [DEBUG] [VALIDATOR ] BIO: All expected trading days are present. [2025-09-16T17:28:41.658930] [DEBUG] [VALIDATOR ] BIO: Primary data validation passed. [2025-09-16T17:28:41.659379] [INFO ] [MAIN ] BIO: Fetched primary data is valid. [2025-09-16T17:28:41.659444] [DEBUG] [LOCAL ] BIO: Saving primary data. [2025-09-16T17:28:41.660236] [DEBUG] [LOCAL ] BIO: Trimmed 28282 records to 18405 (keeping 150 days of data) [2025-09-16T17:28:41.771820] [DEBUG] [LOCAL ] BIO: Primary data saved (18405 records). [2025-09-16T17:28:41.771964] [INFO ] [MAIN ] BIO: Full history saved. [2025-09-16T17:28:41.772059] [INFO ] [MAIN ] Update Progress: 276/2369 symbols completed. BIO took 24.1s. Estimated remaining: 3h 37m 13s. Rate: 9.6 symbols/min [2025-09-16T17:28:41.953618] [INFO ] [MAIN ] BIPH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:28:41.956986] [DEBUG] [LOCAL ] BIPH: Loaded 1566 existing records from primary storage. [2025-09-16T17:28:41.957059] [INFO ] [MAIN ] BIPH: Loaded 1566 existing records. [2025-09-16T17:28:41.980045] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-08. [2025-09-16T17:28:42.144249] [WARN ] [MAIN ] BIPH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:28:42.144525] [INFO ] [MAIN ] BIPH: Wiping existing primary data before full refresh. [2025-09-16T17:28:42.144590] [INFO ] [LOCAL ] BIPH: Starting deletion of primary data. [2025-09-16T17:28:42.144860] [DEBUG] [LOCAL ] BIPH: Deleted primary data file. [2025-09-16T17:28:42.144954] [INFO ] [MAIN ] BIPH: Starting full primary refresh. [2025-09-16T17:28:42.508709] [INFO ] [REMOTE ] BIPH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:28:42.774855] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-02. [2025-09-16T17:28:42.990663] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T17:28:43.389627] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-09-16T17:28:43.548929] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T17:28:43.574862] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-09. [2025-09-16T17:28:44.374924] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-03. [2025-09-16T17:28:44.829011] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-09-16T17:28:45.144506] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T17:28:45.149290] [INFO ] [REMOTE ] BIP: Full fetch complete (43456 records). [2025-09-16T17:28:45.149393] [DEBUG] [VALIDATOR ] BIP: Validating primary data. [2025-09-16T17:28:45.179998] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-04. [2025-09-16T17:28:45.542003] [DEBUG] [VALIDATOR ] BIP: All expected trading days are present. [2025-09-16T17:28:45.542116] [DEBUG] [VALIDATOR ] BIP: Primary data validation passed. [2025-09-16T17:28:45.542665] [INFO ] [MAIN ] BIP: Fetched primary data is valid. [2025-09-16T17:28:45.542712] [DEBUG] [LOCAL ] BIP: Saving primary data. [2025-09-16T17:28:45.544523] [DEBUG] [LOCAL ] BIP: Trimmed 43456 records to 28099 (keeping 150 days of data) [2025-09-16T17:28:45.722763] [DEBUG] [LOCAL ] BIP: Primary data saved (28099 records). [2025-09-16T17:28:45.722905] [INFO ] [MAIN ] BIP: Full history saved. [2025-09-16T17:28:45.725884] [INFO ] [MAIN ] Update Progress: 277/2369 symbols completed. BIP took 21.2s. Estimated remaining: 3h 36m 50s. Rate: 9.6 symbols/min [2025-09-16T17:28:45.828755] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-09-16T17:28:45.974863] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-05. [2025-09-16T17:28:46.352391] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-09-16T17:28:46.774904] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-02. [2025-09-16T17:28:47.574904] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-08. [2025-09-16T17:28:48.323269] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T17:28:48.374922] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-03. [2025-09-16T17:28:49.027685] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T17:28:49.174899] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-06. [2025-09-16T17:28:49.614746] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-09-16T17:28:49.743746] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T17:28:49.974912] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-04. [2025-09-16T17:28:50.774945] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-09. [2025-09-16T17:28:51.267390] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-09-16T17:28:51.574894] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-05. [2025-09-16T17:28:52.374962] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-07. [2025-09-16T17:28:52.728768] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-09-16T17:28:52.730208] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-09-16T17:28:52.739197] [INFO ] [REMOTE ] BILL: Full fetch complete (56980 records). [2025-09-16T17:28:52.739369] [DEBUG] [VALIDATOR ] BILL: Validating primary data. [2025-09-16T17:28:53.149019] [DEBUG] [VALIDATOR ] BILL: All expected trading days are present. [2025-09-16T17:28:53.149208] [DEBUG] [VALIDATOR ] BILL: Primary data validation passed. [2025-09-16T17:28:53.150054] [INFO ] [MAIN ] BILL: Fetched primary data is valid. [2025-09-16T17:28:53.150121] [DEBUG] [LOCAL ] BILL: Saving primary data. [2025-09-16T17:28:53.151656] [DEBUG] [LOCAL ] BILL: Trimmed 56980 records to 39007 (keeping 150 days of data) [2025-09-16T17:28:53.209878] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-08. [2025-09-16T17:28:53.282473] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-09-16T17:28:53.427602] [DEBUG] [LOCAL ] BILL: Primary data saved (39007 records). [2025-09-16T17:28:53.427766] [INFO ] [MAIN ] BILL: Full history saved. [2025-09-16T17:28:53.427859] [INFO ] [MAIN ] Update Progress: 278/2369 symbols completed. BILL took 37.8s. Estimated remaining: 3h 36m 55s. Rate: 9.6 symbols/min [2025-09-16T17:28:53.690849] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-09-16T17:28:53.974923] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-06. [2025-09-16T17:28:54.774916] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-09. [2025-09-16T17:28:55.072315] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-09-16T17:28:55.240884] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-09-16T17:28:55.241984] [INFO ] [REMOTE ] BIPH: Full fetch complete (2794 records). [2025-09-16T17:28:55.242058] [DEBUG] [VALIDATOR ] BIPH: Validating primary data. [2025-09-16T17:28:55.580077] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-07. [2025-09-16T17:28:55.631009] [DEBUG] [VALIDATOR ] BIPH: All expected trading days are present. [2025-09-16T17:28:55.631078] [DEBUG] [VALIDATOR ] BIPH: Primary data validation passed. [2025-09-16T17:28:55.631191] [INFO ] [MAIN ] BIPH: Fetched primary data is valid. [2025-09-16T17:28:55.631229] [DEBUG] [LOCAL ] BIPH: Saving primary data. [2025-09-16T17:28:55.631566] [DEBUG] [LOCAL ] BIPH: Trimmed 2794 records to 1585 (keeping 150 days of data) [2025-09-16T17:28:55.642225] [DEBUG] [LOCAL ] BIPH: Primary data saved (1585 records). [2025-09-16T17:28:55.642292] [INFO ] [MAIN ] BIPH: Full history saved. [2025-09-16T17:28:55.642360] [INFO ] [MAIN ] Update Progress: 279/2369 symbols completed. BIPH took 13.9s. Estimated remaining: 3h 36m 19s. Rate: 9.7 symbols/min [2025-09-16T17:28:57.205341] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-09-16T17:28:57.214759] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-08. [2025-09-16T17:28:58.794521] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T17:28:58.803325] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-09. [2025-09-16T17:28:59.915906] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T17:28:59.920665] [INFO ] [REMOTE ] BIPC: Full fetch complete (45505 records). [2025-09-16T17:28:59.920779] [DEBUG] [VALIDATOR ] BIPC: Validating primary data. [2025-09-16T17:29:00.303430] [DEBUG] [VALIDATOR ] BIPC: All expected trading days are present. [2025-09-16T17:29:00.303552] [DEBUG] [VALIDATOR ] BIPC: Primary data validation passed. [2025-09-16T17:29:00.304213] [INFO ] [MAIN ] BIPC: Fetched primary data is valid. [2025-09-16T17:29:00.304301] [DEBUG] [LOCAL ] BIPC: Saving primary data. [2025-09-16T17:29:00.307134] [DEBUG] [LOCAL ] BIPC: Trimmed 45505 records to 30211 (keeping 150 days of data) [2025-09-16T17:29:00.498257] [DEBUG] [LOCAL ] BIPC: Primary data saved (30211 records). [2025-09-16T17:29:00.498391] [INFO ] [MAIN ] BIPC: Full history saved. [2025-09-16T17:29:00.499287] [INFO ] [MAIN ] Update Progress: 280/2369 symbols completed. BIPC took 20.0s. Estimated remaining: 3h 36m 2s. Rate: 9.7 symbols/min [2025-09-16T17:29:01.085952] [INFO ] [MAIN ] BIRK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:01.212570] [INFO ] [MAIN ] BIPJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:01.222597] [INFO ] [MAIN ] BIPI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:01.242425] [INFO ] [MAIN ] BIT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:01.250478] [DEBUG] [LOCAL ] BIPI: Loaded 2045 existing records from primary storage. [2025-09-16T17:29:01.248910] [DEBUG] [LOCAL ] BIPJ: Loaded 2114 existing records from primary storage. [2025-09-16T17:29:01.251002] [INFO ] [MAIN ] BIPI: Loaded 2045 existing records. [2025-09-16T17:29:01.261930] [INFO ] [MAIN ] BIPJ: Loaded 2114 existing records. [2025-09-16T17:29:01.480443] [DEBUG] [LOCAL ] BIT: Loaded 16393 existing records from primary storage. [2025-09-16T17:29:01.535853] [INFO ] [MAIN ] BIT: Loaded 16393 existing records. [2025-09-16T17:29:01.620564] [WARN ] [MAIN ] BIPI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:01.657351] [INFO ] [MAIN ] BIPI: Wiping existing primary data before full refresh. [2025-09-16T17:29:01.732668] [INFO ] [LOCAL ] BIPI: Starting deletion of primary data. [2025-09-16T17:29:01.732532] [WARN ] [MAIN ] BIPJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:01.783197] [INFO ] [MAIN ] BIPJ: Wiping existing primary data before full refresh. [2025-09-16T17:29:01.783091] [DEBUG] [LOCAL ] BIRK: Loaded 37594 existing records from primary storage. [2025-09-16T17:29:01.799139] [INFO ] [LOCAL ] BIPJ: Starting deletion of primary data. [2025-09-16T17:29:01.798994] [INFO ] [MAIN ] BIRK: Loaded 37594 existing records. [2025-09-16T17:29:01.788461] [DEBUG] [LOCAL ] BIPI: Deleted primary data file. [2025-09-16T17:29:01.849660] [DEBUG] [LOCAL ] BIPJ: Deleted primary data file. [2025-09-16T17:29:01.866098] [INFO ] [MAIN ] BIPJ: Starting full primary refresh. [2025-09-16T17:29:01.858951] [INFO ] [MAIN ] BIPI: Starting full primary refresh. [2025-09-16T17:29:01.911213] [WARN ] [MAIN ] BIT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:02.144675] [INFO ] [MAIN ] BIT: Wiping existing primary data before full refresh. [2025-09-16T17:29:02.216412] [INFO ] [LOCAL ] BIT: Starting deletion of primary data. [2025-09-16T17:29:02.436067] [WARN ] [MAIN ] BIRK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:02.394740] [DEBUG] [LOCAL ] BIT: Deleted primary data file. [2025-09-16T17:29:02.474556] [INFO ] [MAIN ] BIRK: Wiping existing primary data before full refresh. [2025-09-16T17:29:02.508438] [INFO ] [MAIN ] BIT: Starting full primary refresh. [2025-09-16T17:29:02.508504] [INFO ] [LOCAL ] BIRK: Starting deletion of primary data. [2025-09-16T17:29:02.593722] [DEBUG] [LOCAL ] BIRK: Deleted primary data file. [2025-09-16T17:29:02.698449] [INFO ] [MAIN ] BIRK: Starting full primary refresh. [2025-09-16T17:29:03.029638] [INFO ] [REMOTE ] BIPI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:03.142681] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-02. [2025-09-16T17:29:03.173052] [INFO ] [REMOTE ] BIPJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:03.208516] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-02. [2025-09-16T17:29:03.472113] [INFO ] [REMOTE ] BIT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:03.482775] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-02. [2025-09-16T17:29:03.592595] [INFO ] [REMOTE ] BIRK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:03.592922] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-02. [2025-09-16T17:29:03.727438] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-09-16T17:29:03.728687] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-03. [2025-09-16T17:29:03.833717] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-09-16T17:29:03.834900] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-03. [2025-09-16T17:29:04.132807] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-03 (0.30 seconds). [2025-09-16T17:29:04.134014] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-04. [2025-09-16T17:29:04.350489] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-09-16T17:29:04.352398] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-04. [2025-09-16T17:29:04.501323] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-09-16T17:29:04.503466] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-05. [2025-09-16T17:29:04.525803] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T17:29:04.799228] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-09-16T17:29:04.831522] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-09-16T17:29:05.180421] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-06. [2025-09-16T17:29:05.183415] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T17:29:05.727281] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-09-16T17:29:05.974916] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-05. [2025-09-16T17:29:06.273602] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-09-16T17:29:06.774946] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-06. [2025-09-16T17:29:07.248744] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-09-16T17:29:07.574949] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-03. [2025-09-16T17:29:08.374899] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-03. [2025-09-16T17:29:08.798199] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-09-16T17:29:09.174918] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-04. [2025-09-16T17:29:09.456730] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T17:29:09.974912] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-07. [2025-09-16T17:29:10.443730] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-09-16T17:29:10.509558] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-09-16T17:29:10.774906] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-04. [2025-09-16T17:29:11.574885] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-07. [2025-09-16T17:29:11.999326] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-09-16T17:29:12.029110] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-09-16T17:29:12.374901] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-05. [2025-09-16T17:29:13.174879] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-08. [2025-09-16T17:29:13.369037] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T17:29:13.544020] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-09-16T17:29:13.974904] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-05. [2025-09-16T17:29:14.774903] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-08. [2025-09-16T17:29:15.447748] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-09-16T17:29:15.574884] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-09. [2025-09-16T17:29:15.769240] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T17:29:16.144858] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-09-16T17:29:16.146008] [INFO ] [REMOTE ] BIPI: Full fetch complete (3619 records). [2025-09-16T17:29:16.146078] [DEBUG] [VALIDATOR ] BIPI: Validating primary data. [2025-09-16T17:29:16.385178] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-06. [2025-09-16T17:29:16.512067] [DEBUG] [VALIDATOR ] BIPI: All expected trading days are present. [2025-09-16T17:29:16.512141] [DEBUG] [VALIDATOR ] BIPI: Primary data validation passed. [2025-09-16T17:29:16.512263] [INFO ] [MAIN ] BIPI: Fetched primary data is valid. [2025-09-16T17:29:16.512301] [DEBUG] [LOCAL ] BIPI: Saving primary data. [2025-09-16T17:29:16.512666] [DEBUG] [LOCAL ] BIPI: Trimmed 3619 records to 2076 (keeping 150 days of data) [2025-09-16T17:29:16.526032] [DEBUG] [LOCAL ] BIPI: Primary data saved (2076 records). [2025-09-16T17:29:16.526093] [INFO ] [MAIN ] BIPI: Full history saved. [2025-09-16T17:29:16.526164] [INFO ] [MAIN ] Update Progress: 281/2369 symbols completed. BIPI took 16.0s. Estimated remaining: 3h 37m 9s. Rate: 9.6 symbols/min [2025-09-16T17:29:16.693721] [INFO ] [MAIN ] BJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:16.741389] [DEBUG] [LOCAL ] BJ: Loaded 37884 existing records from primary storage. [2025-09-16T17:29:16.741534] [INFO ] [MAIN ] BJ: Loaded 37884 existing records. [2025-09-16T17:29:16.929899] [WARN ] [MAIN ] BJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:16.930250] [INFO ] [MAIN ] BJ: Wiping existing primary data before full refresh. [2025-09-16T17:29:16.930321] [INFO ] [LOCAL ] BJ: Starting deletion of primary data. [2025-09-16T17:29:16.931726] [DEBUG] [LOCAL ] BJ: Deleted primary data file. [2025-09-16T17:29:16.931992] [INFO ] [MAIN ] BJ: Starting full primary refresh. [2025-09-16T17:29:17.175552] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-09. [2025-09-16T17:29:17.244119] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-09-16T17:29:17.500672] [INFO ] [REMOTE ] BJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:17.946614] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-09-16T17:29:17.948427] [INFO ] [REMOTE ] BIPJ: Full fetch complete (3106 records). [2025-09-16T17:29:17.948551] [DEBUG] [VALIDATOR ] BIPJ: Validating primary data. [2025-09-16T17:29:17.982208] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-06. [2025-09-16T17:29:18.333290] [DEBUG] [VALIDATOR ] BIPJ: All expected trading days are present. [2025-09-16T17:29:18.333374] [DEBUG] [VALIDATOR ] BIPJ: Primary data validation passed. [2025-09-16T17:29:18.333520] [INFO ] [MAIN ] BIPJ: Fetched primary data is valid. [2025-09-16T17:29:18.333563] [DEBUG] [LOCAL ] BIPJ: Saving primary data. [2025-09-16T17:29:18.333970] [DEBUG] [LOCAL ] BIPJ: Trimmed 3106 records to 2171 (keeping 150 days of data) [2025-09-16T17:29:18.348344] [DEBUG] [LOCAL ] BIPJ: Primary data saved (2171 records). [2025-09-16T17:29:18.348441] [INFO ] [MAIN ] BIPJ: Full history saved. [2025-09-16T17:29:18.348531] [INFO ] [MAIN ] Update Progress: 282/2369 symbols completed. BIPJ took 17.8s. Estimated remaining: 3h 36m 30s. Rate: 9.6 symbols/min [2025-09-16T17:29:18.525549] [INFO ] [MAIN ] BK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:18.566871] [DEBUG] [LOCAL ] BK: Loaded 39555 existing records from primary storage. [2025-09-16T17:29:18.567047] [INFO ] [MAIN ] BK: Loaded 39555 existing records. [2025-09-16T17:29:18.747000] [WARN ] [MAIN ] BK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:18.747228] [INFO ] [MAIN ] BK: Wiping existing primary data before full refresh. [2025-09-16T17:29:18.747273] [INFO ] [LOCAL ] BK: Starting deletion of primary data. [2025-09-16T17:29:18.748213] [DEBUG] [LOCAL ] BK: Deleted primary data file. [2025-09-16T17:29:18.748262] [INFO ] [MAIN ] BK: Starting full primary refresh. [2025-09-16T17:29:18.780034] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-02. [2025-09-16T17:29:19.093835] [INFO ] [REMOTE ] BK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:19.561143] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T17:29:19.574861] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-07. [2025-09-16T17:29:20.374869] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-02. [2025-09-16T17:29:20.670079] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-09-16T17:29:21.021451] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T17:29:21.174903] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-03. [2025-09-16T17:29:21.974865] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-08. [2025-09-16T17:29:22.030510] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T17:29:22.599666] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T17:29:22.774912] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-04. [2025-09-16T17:29:23.317772] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T17:29:23.574906] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-07. [2025-09-16T17:29:24.351834] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T17:29:24.374850] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-05. [2025-09-16T17:29:24.669081] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T17:29:25.174902] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-03. [2025-09-16T17:29:25.927166] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-09-16T17:29:25.974844] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-06. [2025-09-16T17:29:26.774908] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-09. [2025-09-16T17:29:27.045007] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-09-16T17:29:27.320158] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T17:29:27.574879] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-07. [2025-09-16T17:29:28.370202] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T17:29:28.376198] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-04. [2025-09-16T17:29:28.378224] [INFO ] [REMOTE ] BIRK: Full fetch complete (55266 records). [2025-09-16T17:29:28.378327] [DEBUG] [VALIDATOR ] BIRK: Validating primary data. [2025-09-16T17:29:28.770246] [DEBUG] [VALIDATOR ] BIRK: All expected trading days are present. [2025-09-16T17:29:28.770328] [DEBUG] [VALIDATOR ] BIRK: Primary data validation passed. [2025-09-16T17:29:28.771051] [INFO ] [MAIN ] BIRK: Fetched primary data is valid. [2025-09-16T17:29:28.771097] [DEBUG] [LOCAL ] BIRK: Saving primary data. [2025-09-16T17:29:28.772252] [DEBUG] [LOCAL ] BIRK: Trimmed 55266 records to 37975 (keeping 150 days of data) [2025-09-16T17:29:28.995914] [DEBUG] [LOCAL ] BIRK: Primary data saved (37975 records). [2025-09-16T17:29:28.996072] [INFO ] [MAIN ] BIRK: Full history saved. [2025-09-16T17:29:28.996162] [INFO ] [MAIN ] Update Progress: 283/2369 symbols completed. BIRK took 28.5s. Estimated remaining: 3h 36m 57s. Rate: 9.6 symbols/min [2025-09-16T17:29:29.167242] [INFO ] [MAIN ] BKD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:29.174874] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-08. [2025-09-16T17:29:29.202741] [DEBUG] [LOCAL ] BKD: Loaded 37053 existing records from primary storage. [2025-09-16T17:29:29.202966] [INFO ] [MAIN ] BKD: Loaded 37053 existing records. [2025-09-16T17:29:29.346748] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-09-16T17:29:29.423985] [WARN ] [MAIN ] BKD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:29.424220] [INFO ] [MAIN ] BKD: Wiping existing primary data before full refresh. [2025-09-16T17:29:29.424265] [INFO ] [LOCAL ] BKD: Starting deletion of primary data. [2025-09-16T17:29:29.425123] [DEBUG] [LOCAL ] BKD: Deleted primary data file. [2025-09-16T17:29:29.425173] [INFO ] [MAIN ] BKD: Starting full primary refresh. [2025-09-16T17:29:29.777892] [INFO ] [REMOTE ] BKD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:29.974968] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-08. [2025-09-16T17:29:30.088834] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-09-16T17:29:30.402222] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-09-16T17:29:30.774940] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-09. [2025-09-16T17:29:31.563546] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T17:29:31.574986] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-05. [2025-09-16T17:29:31.737274] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T17:29:31.741589] [INFO ] [REMOTE ] BIT: Full fetch complete (23240 records). [2025-09-16T17:29:31.741690] [DEBUG] [VALIDATOR ] BIT: Validating primary data. [2025-09-16T17:29:32.106069] [DEBUG] [VALIDATOR ] BIT: All expected trading days are present. [2025-09-16T17:29:32.106148] [DEBUG] [VALIDATOR ] BIT: Primary data validation passed. [2025-09-16T17:29:32.106518] [INFO ] [MAIN ] BIT: Fetched primary data is valid. [2025-09-16T17:29:32.106560] [DEBUG] [LOCAL ] BIT: Saving primary data. [2025-09-16T17:29:32.107194] [DEBUG] [LOCAL ] BIT: Trimmed 23240 records to 16575 (keeping 150 days of data) [2025-09-16T17:29:32.212905] [DEBUG] [LOCAL ] BIT: Primary data saved (16575 records). [2025-09-16T17:29:32.213052] [INFO ] [MAIN ] BIT: Full history saved. [2025-09-16T17:29:32.213133] [INFO ] [MAIN ] Update Progress: 284/2369 symbols completed. BIT took 31.7s. Estimated remaining: 3h 36m 28s. Rate: 9.6 symbols/min [2025-09-16T17:29:32.384686] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-09. [2025-09-16T17:29:32.387638] [INFO ] [MAIN ] BKE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:32.412074] [DEBUG] [LOCAL ] BKE: Loaded 24727 existing records from primary storage. [2025-09-16T17:29:32.412222] [INFO ] [MAIN ] BKE: Loaded 24727 existing records. [2025-09-16T17:29:32.589954] [WARN ] [MAIN ] BKE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:32.590241] [INFO ] [MAIN ] BKE: Wiping existing primary data before full refresh. [2025-09-16T17:29:32.590303] [INFO ] [LOCAL ] BKE: Starting deletion of primary data. [2025-09-16T17:29:32.591033] [DEBUG] [LOCAL ] BKE: Deleted primary data file. [2025-09-16T17:29:32.591116] [INFO ] [MAIN ] BKE: Starting full primary refresh. [2025-09-16T17:29:32.942010] [INFO ] [REMOTE ] BKE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:33.174895] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-02. [2025-09-16T17:29:33.445155] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-09-16T17:29:33.974896] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-06. [2025-09-16T17:29:34.176709] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-09-16T17:29:34.182839] [INFO ] [REMOTE ] BJ: Full fetch complete (55947 records). [2025-09-16T17:29:34.182945] [DEBUG] [VALIDATOR ] BJ: Validating primary data. [2025-09-16T17:29:34.561575] [DEBUG] [VALIDATOR ] BJ: All expected trading days are present. [2025-09-16T17:29:34.561698] [DEBUG] [VALIDATOR ] BJ: Primary data validation passed. [2025-09-16T17:29:34.562348] [INFO ] [MAIN ] BJ: Fetched primary data is valid. [2025-09-16T17:29:34.562403] [DEBUG] [LOCAL ] BJ: Saving primary data. [2025-09-16T17:29:34.563964] [DEBUG] [LOCAL ] BJ: Trimmed 55947 records to 38269 (keeping 150 days of data) [2025-09-16T17:29:34.689212] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-09-16T17:29:34.791793] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-03. [2025-09-16T17:29:34.824059] [DEBUG] [LOCAL ] BJ: Primary data saved (38269 records). [2025-09-16T17:29:34.824215] [INFO ] [MAIN ] BJ: Full history saved. [2025-09-16T17:29:34.824304] [INFO ] [MAIN ] Update Progress: 285/2369 symbols completed. BJ took 18.3s. Estimated remaining: 3h 35m 55s. Rate: 9.7 symbols/min [2025-09-16T17:29:34.991528] [INFO ] [MAIN ] BKH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:35.017378] [DEBUG] [LOCAL ] BKH: Loaded 27296 existing records from primary storage. [2025-09-16T17:29:35.017524] [INFO ] [MAIN ] BKH: Loaded 27296 existing records. [2025-09-16T17:29:35.192467] [WARN ] [MAIN ] BKH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:35.192736] [INFO ] [MAIN ] BKH: Wiping existing primary data before full refresh. [2025-09-16T17:29:35.192782] [INFO ] [LOCAL ] BKH: Starting deletion of primary data. [2025-09-16T17:29:35.193518] [DEBUG] [LOCAL ] BKH: Deleted primary data file. [2025-09-16T17:29:35.193566] [INFO ] [MAIN ] BKH: Starting full primary refresh. [2025-09-16T17:29:35.547610] [INFO ] [REMOTE ] BKH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:35.574853] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-02. [2025-09-16T17:29:35.640368] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-09-16T17:29:36.081565] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-09-16T17:29:36.374900] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-04. [2025-09-16T17:29:36.691138] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-09-16T17:29:37.174911] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-02. [2025-09-16T17:29:37.802175] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:29:37.974929] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-05. [2025-09-16T17:29:38.360883] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T17:29:38.774916] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-07. [2025-09-16T17:29:39.319977] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T17:29:39.574899] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-03. [2025-09-16T17:29:40.374889] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-03. [2025-09-16T17:29:40.442349] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-09-16T17:29:40.909353] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T17:29:41.174901] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-08. [2025-09-16T17:29:41.334001] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-09-16T17:29:41.974908] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-04. [2025-09-16T17:29:42.774907] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-06. [2025-09-16T17:29:42.990153] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T17:29:43.288792] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T17:29:43.574885] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-09. [2025-09-16T17:29:44.056908] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T17:29:44.374917] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-07. [2025-09-16T17:29:45.174893] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-05. [2025-09-16T17:29:45.323378] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-09-16T17:29:45.329265] [INFO ] [REMOTE ] BK: Full fetch complete (57858 records). [2025-09-16T17:29:45.329374] [DEBUG] [VALIDATOR ] BK: Validating primary data. [2025-09-16T17:29:45.330083] [FATAL] [VALIDATOR ] BK: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:29:45.330169] [FATAL] [MAIN ] BK: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:29:45.330211] [WARN ] [MAIN ] BK: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:29:45.330271] [INFO ] [LOCAL ] BK: Starting deletion of primary data. [2025-09-16T17:29:45.330360] [DEBUG] [LOCAL ] BK: No primary data file found to delete. [2025-09-16T17:29:45.330415] [INFO ] [LOCAL ] BK: Starting deletion of derivative data. [2025-09-16T17:29:45.330870] [DEBUG] [LOCAL ] BK: Deleted derivative data file. [2025-09-16T17:29:45.330982] [INFO ] [MAIN ] Update Progress: 286/2369 symbols completed. BK took 27.0s. Estimated remaining: 3h 36m 20s. Rate: 9.6 symbols/min [2025-09-16T17:29:45.514335] [INFO ] [MAIN ] BKKT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:45.542609] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T17:29:45.570916] [DEBUG] [LOCAL ] BKKT: Loaded 25845 existing records from primary storage. [2025-09-16T17:29:45.571041] [INFO ] [MAIN ] BKKT: Loaded 25845 existing records. [2025-09-16T17:29:45.753156] [WARN ] [MAIN ] BKKT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:45.753454] [INFO ] [MAIN ] BKKT: Wiping existing primary data before full refresh. [2025-09-16T17:29:45.753522] [INFO ] [LOCAL ] BKKT: Starting deletion of primary data. [2025-09-16T17:29:45.754355] [DEBUG] [LOCAL ] BKKT: Deleted primary data file. [2025-09-16T17:29:45.754428] [INFO ] [MAIN ] BKKT: Starting full primary refresh. [2025-09-16T17:29:45.980008] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-08. [2025-09-16T17:29:46.114035] [INFO ] [REMOTE ] BKKT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:46.668132] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T17:29:46.774881] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-06. [2025-09-16T17:29:47.089587] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-09-16T17:29:47.574941] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-04. [2025-09-16T17:29:48.374909] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-09. [2025-09-16T17:29:48.432189] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T17:29:48.694470] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T17:29:49.174933] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-05. [2025-09-16T17:29:49.350483] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T17:29:49.354964] [INFO ] [REMOTE ] BKE: Full fetch complete (38047 records). [2025-09-16T17:29:49.355065] [DEBUG] [VALIDATOR ] BKE: Validating primary data. [2025-09-16T17:29:49.753098] [DEBUG] [VALIDATOR ] BKE: All expected trading days are present. [2025-09-16T17:29:49.753327] [DEBUG] [VALIDATOR ] BKE: Primary data validation passed. [2025-09-16T17:29:49.753901] [INFO ] [MAIN ] BKE: Fetched primary data is valid. [2025-09-16T17:29:49.753953] [DEBUG] [LOCAL ] BKE: Saving primary data. [2025-09-16T17:29:49.755787] [DEBUG] [LOCAL ] BKE: Trimmed 38047 records to 25000 (keeping 150 days of data) [2025-09-16T17:29:49.911559] [DEBUG] [LOCAL ] BKE: Primary data saved (25000 records). [2025-09-16T17:29:49.911726] [INFO ] [MAIN ] BKE: Full history saved. [2025-09-16T17:29:49.911860] [INFO ] [MAIN ] Update Progress: 287/2369 symbols completed. BKE took 17.7s. Estimated remaining: 3h 36m 2s. Rate: 9.6 symbols/min [2025-09-16T17:29:49.980021] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-07. [2025-09-16T17:29:50.087574] [INFO ] [MAIN ] BKN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:50.094327] [DEBUG] [LOCAL ] BKN: Loaded 5415 existing records from primary storage. [2025-09-16T17:29:50.094417] [INFO ] [MAIN ] BKN: Loaded 5415 existing records. [2025-09-16T17:29:50.270312] [WARN ] [MAIN ] BKN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:50.270552] [INFO ] [MAIN ] BKN: Wiping existing primary data before full refresh. [2025-09-16T17:29:50.270598] [INFO ] [LOCAL ] BKN: Starting deletion of primary data. [2025-09-16T17:29:50.270893] [DEBUG] [LOCAL ] BKN: Deleted primary data file. [2025-09-16T17:29:50.270940] [INFO ] [MAIN ] BKN: Starting full primary refresh. [2025-09-16T17:29:50.632436] [INFO ] [REMOTE ] BKN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:50.653471] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T17:29:50.774866] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-02. [2025-09-16T17:29:51.385283] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-09-16T17:29:51.470972] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-09-16T17:29:51.574910] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-03. [2025-09-16T17:29:52.253343] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-09-16T17:29:52.374903] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-06. [2025-09-16T17:29:53.174896] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-08. [2025-09-16T17:29:53.732936] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-09-16T17:29:53.974918] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-07. [2025-09-16T17:29:54.686499] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T17:29:54.774880] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-09. [2025-09-16T17:29:55.489758] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T17:29:55.574900] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-08. [2025-09-16T17:29:56.271595] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T17:29:56.277595] [INFO ] [REMOTE ] BKD: Full fetch complete (54361 records). [2025-09-16T17:29:56.277742] [DEBUG] [VALIDATOR ] BKD: Validating primary data. [2025-09-16T17:29:56.381259] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-04. [2025-09-16T17:29:56.658622] [DEBUG] [VALIDATOR ] BKD: All expected trading days are present. [2025-09-16T17:29:56.658724] [DEBUG] [VALIDATOR ] BKD: Primary data validation passed. [2025-09-16T17:29:56.659330] [INFO ] [MAIN ] BKD: Fetched primary data is valid. [2025-09-16T17:29:56.659372] [DEBUG] [LOCAL ] BKD: Saving primary data. [2025-09-16T17:29:56.660477] [DEBUG] [LOCAL ] BKD: Trimmed 54361 records to 37399 (keeping 150 days of data) [2025-09-16T17:29:56.874289] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-09-16T17:29:56.907585] [DEBUG] [LOCAL ] BKD: Primary data saved (37399 records). [2025-09-16T17:29:56.907725] [INFO ] [MAIN ] BKD: Full history saved. [2025-09-16T17:29:56.908737] [INFO ] [MAIN ] Update Progress: 288/2369 symbols completed. BKD took 27.9s. Estimated remaining: 3h 36m 2s. Rate: 9.6 symbols/min [2025-09-16T17:29:57.076468] [INFO ] [MAIN ] BKSY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:57.114913] [DEBUG] [LOCAL ] BKSY: Loaded 35899 existing records from primary storage. [2025-09-16T17:29:57.115050] [INFO ] [MAIN ] BKSY: Loaded 35899 existing records. [2025-09-16T17:29:57.190765] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-09. [2025-09-16T17:29:57.259497] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-09-16T17:29:57.341980] [WARN ] [MAIN ] BKSY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:57.342269] [INFO ] [MAIN ] BKSY: Wiping existing primary data before full refresh. [2025-09-16T17:29:57.342315] [INFO ] [LOCAL ] BKSY: Starting deletion of primary data. [2025-09-16T17:29:57.343344] [DEBUG] [LOCAL ] BKSY: Deleted primary data file. [2025-09-16T17:29:57.343392] [INFO ] [MAIN ] BKSY: Starting full primary refresh. [2025-09-16T17:29:57.692268] [INFO ] [REMOTE ] BKSY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:57.974892] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-05. [2025-09-16T17:29:58.552051] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T17:29:58.556561] [INFO ] [REMOTE ] BKH: Full fetch complete (39524 records). [2025-09-16T17:29:58.556717] [DEBUG] [VALIDATOR ] BKH: Validating primary data. [2025-09-16T17:29:58.783841] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-02. [2025-09-16T17:29:58.817012] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T17:29:58.956458] [DEBUG] [VALIDATOR ] BKH: All expected trading days are present. [2025-09-16T17:29:58.956536] [DEBUG] [VALIDATOR ] BKH: Primary data validation passed. [2025-09-16T17:29:58.957082] [INFO ] [MAIN ] BKH: Fetched primary data is valid. [2025-09-16T17:29:58.957127] [DEBUG] [LOCAL ] BKH: Saving primary data. [2025-09-16T17:29:58.958030] [DEBUG] [LOCAL ] BKH: Trimmed 39524 records to 27582 (keeping 150 days of data) [2025-09-16T17:29:59.124295] [DEBUG] [LOCAL ] BKH: Primary data saved (27582 records). [2025-09-16T17:29:59.124431] [INFO ] [MAIN ] BKH: Full history saved. [2025-09-16T17:29:59.124515] [INFO ] [MAIN ] Update Progress: 289/2369 symbols completed. BKH took 24.3s. Estimated remaining: 3h 35m 26s. Rate: 9.7 symbols/min [2025-09-16T17:29:59.306101] [INFO ] [MAIN ] BKT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:29:59.315390] [DEBUG] [LOCAL ] BKT: Loaded 7700 existing records from primary storage. [2025-09-16T17:29:59.315514] [INFO ] [MAIN ] BKT: Loaded 7700 existing records. [2025-09-16T17:29:59.496799] [WARN ] [MAIN ] BKT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:29:59.497047] [INFO ] [MAIN ] BKT: Wiping existing primary data before full refresh. [2025-09-16T17:29:59.497095] [INFO ] [LOCAL ] BKT: Starting deletion of primary data. [2025-09-16T17:29:59.497413] [DEBUG] [LOCAL ] BKT: Deleted primary data file. [2025-09-16T17:29:59.497459] [INFO ] [MAIN ] BKT: Starting full primary refresh. [2025-09-16T17:29:59.579964] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-06. [2025-09-16T17:29:59.870438] [INFO ] [REMOTE ] BKT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:29:59.893780] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-09-16T17:30:00.359728] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T17:30:00.374854] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-07. [2025-09-16T17:30:01.174907] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-02. [2025-09-16T17:30:01.974892] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-02. [2025-09-16T17:30:02.774955] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-03. [2025-09-16T17:30:02.880565] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-07 (2.51 seconds). [2025-09-16T17:30:03.574889] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-08. [2025-09-16T17:30:03.862976] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-02 (2.69 seconds). [2025-09-16T17:30:04.019477] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-09-16T17:30:04.166470] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T17:30:04.374908] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-04. [2025-09-16T17:30:04.547758] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-02 (2.57 seconds). [2025-09-16T17:30:05.174893] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-09. [2025-09-16T17:30:05.704908] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T17:30:05.974901] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-03. [2025-09-16T17:30:06.142995] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T17:30:06.144487] [INFO ] [REMOTE ] BKN: Full fetch complete (8083 records). [2025-09-16T17:30:06.144566] [DEBUG] [VALIDATOR ] BKN: Validating primary data. [2025-09-16T17:30:06.145115] [FATAL] [VALIDATOR ] BKN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:30:06.145212] [FATAL] [MAIN ] BKN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:30:06.145271] [WARN ] [MAIN ] BKN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:30:06.145340] [INFO ] [LOCAL ] BKN: Starting deletion of primary data. [2025-09-16T17:30:06.145406] [DEBUG] [LOCAL ] BKN: No primary data file found to delete. [2025-09-16T17:30:06.145443] [INFO ] [LOCAL ] BKN: Starting deletion of derivative data. [2025-09-16T17:30:06.145796] [DEBUG] [LOCAL ] BKN: Deleted derivative data file. [2025-09-16T17:30:06.145888] [INFO ] [MAIN ] Update Progress: 290/2369 symbols completed. BKN took 16.2s. Estimated remaining: 3h 35m 26s. Rate: 9.7 symbols/min [2025-09-16T17:30:06.319337] [INFO ] [MAIN ] BKU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:30:06.353482] [DEBUG] [LOCAL ] BKU: Loaded 30685 existing records from primary storage. [2025-09-16T17:30:06.353619] [INFO ] [MAIN ] BKU: Loaded 30685 existing records. [2025-09-16T17:30:06.528506] [WARN ] [MAIN ] BKU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:30:06.528818] [INFO ] [MAIN ] BKU: Wiping existing primary data before full refresh. [2025-09-16T17:30:06.528865] [INFO ] [LOCAL ] BKU: Starting deletion of primary data. [2025-09-16T17:30:06.529632] [DEBUG] [LOCAL ] BKU: Deleted primary data file. [2025-09-16T17:30:06.529712] [INFO ] [MAIN ] BKU: Starting full primary refresh. [2025-09-16T17:30:06.774950] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-03. [2025-09-16T17:30:06.914267] [INFO ] [REMOTE ] BKU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:30:07.363331] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-09-16T17:30:07.504686] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T17:30:07.574912] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-05. [2025-09-16T17:30:08.374932] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-04. [2025-09-16T17:30:08.639792] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-09-16T17:30:09.067904] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-09-16T17:30:09.174944] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-02. [2025-09-16T17:30:09.974958] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-05. [2025-09-16T17:30:10.616611] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-09-16T17:30:10.774980] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-06. [2025-09-16T17:30:10.905543] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T17:30:11.574904] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-04. [2025-09-16T17:30:12.374977] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-03. [2025-09-16T17:30:12.434787] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T17:30:13.061722] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T17:30:13.174949] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-06. [2025-09-16T17:30:13.794575] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-09-16T17:30:13.954700] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-09-16T17:30:13.974860] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-04. [2025-09-16T17:30:14.774899] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-07. [2025-09-16T17:30:15.574925] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-07. [2025-09-16T17:30:16.374936] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-05. [2025-09-16T17:30:16.570743] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-04 (2.60 seconds). [2025-09-16T17:30:17.023662] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-09-16T17:30:17.174911] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-08. [2025-09-16T17:30:17.715533] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-09-16T17:30:17.980033] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-08. [2025-09-16T17:30:17.982983] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T17:30:18.197473] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-09-16T17:30:18.774903] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-05. [2025-09-16T17:30:19.574931] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-09. [2025-09-16T17:30:19.662991] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T17:30:19.875023] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-09-16T17:30:20.250461] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-09-16T17:30:20.253142] [INFO ] [REMOTE ] BKT: Full fetch complete (11286 records). [2025-09-16T17:30:20.253247] [DEBUG] [VALIDATOR ] BKT: Validating primary data. [2025-09-16T17:30:20.379980] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-09. [2025-09-16T17:30:20.638194] [DEBUG] [VALIDATOR ] BKT: All expected trading days are present. [2025-09-16T17:30:20.638473] [DEBUG] [VALIDATOR ] BKT: Primary data validation passed. [2025-09-16T17:30:20.638812] [INFO ] [MAIN ] BKT: Fetched primary data is valid. [2025-09-16T17:30:20.638879] [DEBUG] [LOCAL ] BKT: Saving primary data. [2025-09-16T17:30:20.639481] [DEBUG] [LOCAL ] BKT: Trimmed 11286 records to 8064 (keeping 150 days of data) [2025-09-16T17:30:20.692545] [DEBUG] [LOCAL ] BKT: Primary data saved (8064 records). [2025-09-16T17:30:20.692691] [INFO ] [MAIN ] BKT: Full history saved. [2025-09-16T17:30:20.692814] [INFO ] [MAIN ] Update Progress: 291/2369 symbols completed. BKT took 21.6s. Estimated remaining: 3h 36m 19s. Rate: 9.6 symbols/min [2025-09-16T17:30:20.870809] [INFO ] [MAIN ] BKV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:30:20.892535] [DEBUG] [LOCAL ] BKV: Loaded 20925 existing records from primary storage. [2025-09-16T17:30:20.892697] [INFO ] [MAIN ] BKV: Loaded 20925 existing records. [2025-09-16T17:30:21.086299] [WARN ] [MAIN ] BKV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:30:21.086557] [INFO ] [MAIN ] BKV: Wiping existing primary data before full refresh. [2025-09-16T17:30:21.086603] [INFO ] [LOCAL ] BKV: Starting deletion of primary data. [2025-09-16T17:30:21.087153] [DEBUG] [LOCAL ] BKV: Deleted primary data file. [2025-09-16T17:30:21.087211] [INFO ] [MAIN ] BKV: Starting full primary refresh. [2025-09-16T17:30:21.180724] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-06. [2025-09-16T17:30:21.457540] [INFO ] [REMOTE ] BKV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:30:21.974951] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-02. [2025-09-16T17:30:22.222279] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-09-16T17:30:22.227531] [INFO ] [REMOTE ] BKKT: Full fetch complete (36144 records). [2025-09-16T17:30:22.227626] [DEBUG] [VALIDATOR ] BKKT: Validating primary data. [2025-09-16T17:30:22.606118] [DEBUG] [VALIDATOR ] BKKT: All expected trading days are present. [2025-09-16T17:30:22.606205] [DEBUG] [VALIDATOR ] BKKT: Primary data validation passed. [2025-09-16T17:30:22.606750] [INFO ] [MAIN ] BKKT: Fetched primary data is valid. [2025-09-16T17:30:22.606798] [DEBUG] [LOCAL ] BKKT: Saving primary data. [2025-09-16T17:30:22.607778] [DEBUG] [LOCAL ] BKKT: Trimmed 36144 records to 26158 (keeping 150 days of data) [2025-09-16T17:30:22.768314] [DEBUG] [LOCAL ] BKKT: Primary data saved (26158 records). [2025-09-16T17:30:22.768460] [INFO ] [MAIN ] BKKT: Full history saved. [2025-09-16T17:30:22.768557] [INFO ] [MAIN ] Update Progress: 292/2369 symbols completed. BKKT took 37.4s. Estimated remaining: 3h 35m 43s. Rate: 9.6 symbols/min [2025-09-16T17:30:22.802754] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-06. [2025-09-16T17:30:22.952800] [INFO ] [MAIN ] BLCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:30:22.953034] [DEBUG] [LOCAL ] BLCO: No existing data file found in primary storage. [2025-09-16T17:30:22.953074] [WARN ] [MAIN ] BLCO: No existing data found, performing full refresh instead. [2025-09-16T17:30:22.953196] [INFO ] [MAIN ] BLCO: Wiping existing primary data before full refresh. [2025-09-16T17:30:22.953242] [INFO ] [LOCAL ] BLCO: Starting deletion of primary data. [2025-09-16T17:30:22.953291] [DEBUG] [LOCAL ] BLCO: No primary data file found to delete. [2025-09-16T17:30:22.953329] [INFO ] [MAIN ] BLCO: Starting full primary refresh. [2025-09-16T17:30:23.310881] [INFO ] [REMOTE ] BLCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:30:23.329865] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-09-16T17:30:23.364507] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-06 (2.18 seconds). [2025-09-16T17:30:23.574905] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-03. [2025-09-16T17:30:24.374901] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-07. [2025-09-16T17:30:24.529561] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-09-16T17:30:24.731535] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T17:30:25.174906] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-04. [2025-09-16T17:30:25.974906] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-07. [2025-09-16T17:30:26.373523] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-09-16T17:30:26.425747] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-09-16T17:30:26.774910] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-02. [2025-09-16T17:30:27.464896] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T17:30:27.574945] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-08. [2025-09-16T17:30:28.151686] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T17:30:28.374941] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-08. [2025-09-16T17:30:29.174978] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-03. [2025-09-16T17:30:29.550966] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-09-16T17:30:29.611378] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T17:30:29.974924] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-09. [2025-09-16T17:30:30.593442] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T17:30:30.774917] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-04. [2025-09-16T17:30:31.154716] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T17:30:31.159349] [INFO ] [REMOTE ] BKU: Full fetch complete (45293 records). [2025-09-16T17:30:31.159474] [DEBUG] [VALIDATOR ] BKU: Validating primary data. [2025-09-16T17:30:31.528346] [DEBUG] [VALIDATOR ] BKU: All expected trading days are present. [2025-09-16T17:30:31.528425] [DEBUG] [VALIDATOR ] BKU: Primary data validation passed. [2025-09-16T17:30:31.528955] [INFO ] [MAIN ] BKU: Fetched primary data is valid. [2025-09-16T17:30:31.528999] [DEBUG] [LOCAL ] BKU: Saving primary data. [2025-09-16T17:30:31.529954] [DEBUG] [LOCAL ] BKU: Trimmed 45293 records to 30997 (keeping 150 days of data) [2025-09-16T17:30:31.586005] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-05. [2025-09-16T17:30:31.722314] [DEBUG] [LOCAL ] BKU: Primary data saved (30997 records). [2025-09-16T17:30:31.722451] [INFO ] [MAIN ] BKU: Full history saved. [2025-09-16T17:30:31.722542] [INFO ] [MAIN ] Update Progress: 293/2369 symbols completed. BKU took 25.6s. Estimated remaining: 3h 35m 56s. Rate: 9.6 symbols/min [2025-09-16T17:30:31.891428] [INFO ] [MAIN ] BLD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:30:31.910338] [DEBUG] [LOCAL ] BLD: Loaded 20372 existing records from primary storage. [2025-09-16T17:30:31.910458] [INFO ] [MAIN ] BLD: Loaded 20372 existing records. [2025-09-16T17:30:32.083469] [WARN ] [MAIN ] BLD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:30:32.083761] [INFO ] [MAIN ] BLD: Wiping existing primary data before full refresh. [2025-09-16T17:30:32.083812] [INFO ] [LOCAL ] BLD: Starting deletion of primary data. [2025-09-16T17:30:32.084382] [DEBUG] [LOCAL ] BLD: Deleted primary data file. [2025-09-16T17:30:32.084428] [INFO ] [MAIN ] BLD: Starting full primary refresh. [2025-09-16T17:30:32.380045] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-09. [2025-09-16T17:30:32.447119] [INFO ] [REMOTE ] BLD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:30:32.469348] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T17:30:32.928463] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T17:30:33.174898] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-06. [2025-09-16T17:30:33.977081] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-05. [2025-09-16T17:30:33.978909] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T17:30:33.988201] [INFO ] [REMOTE ] BKSY: Full fetch complete (52754 records). [2025-09-16T17:30:33.988443] [DEBUG] [VALIDATOR ] BKSY: Validating primary data. [2025-09-16T17:30:34.374051] [DEBUG] [VALIDATOR ] BKSY: All expected trading days are present. [2025-09-16T17:30:34.374158] [DEBUG] [VALIDATOR ] BKSY: Primary data validation passed. [2025-09-16T17:30:34.374804] [INFO ] [MAIN ] BKSY: Fetched primary data is valid. [2025-09-16T17:30:34.374859] [DEBUG] [LOCAL ] BKSY: Saving primary data. [2025-09-16T17:30:34.392244] [DEBUG] [LOCAL ] BKSY: Trimmed 52754 records to 36256 (keeping 150 days of data) [2025-09-16T17:30:34.396849] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-09-16T17:30:34.627967] [DEBUG] [LOCAL ] BKSY: Primary data saved (36256 records). [2025-09-16T17:30:34.628172] [INFO ] [MAIN ] BKSY: Full history saved. [2025-09-16T17:30:34.628848] [INFO ] [MAIN ] Update Progress: 294/2369 symbols completed. BKSY took 37.7s. Estimated remaining: 3h 35m 27s. Rate: 9.6 symbols/min [2025-09-16T17:30:34.779833] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-02. [2025-09-16T17:30:34.799010] [INFO ] [MAIN ] BLDR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:30:34.799882] [DEBUG] [LOCAL ] BLDR: No existing data file found in primary storage. [2025-09-16T17:30:34.799921] [WARN ] [MAIN ] BLDR: No existing data found, performing full refresh instead. [2025-09-16T17:30:34.800055] [INFO ] [MAIN ] BLDR: Wiping existing primary data before full refresh. [2025-09-16T17:30:34.800102] [INFO ] [LOCAL ] BLDR: Starting deletion of primary data. [2025-09-16T17:30:34.800146] [DEBUG] [LOCAL ] BLDR: No primary data file found to delete. [2025-09-16T17:30:34.800190] [INFO ] [MAIN ] BLDR: Starting full primary refresh. [2025-09-16T17:30:35.162772] [INFO ] [REMOTE ] BLDR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:30:35.186202] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-09-16T17:30:35.574888] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-07. [2025-09-16T17:30:36.374938] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-02. [2025-09-16T17:30:36.411037] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T17:30:36.493066] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-09-16T17:30:37.174927] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-03. [2025-09-16T17:30:37.974907] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-08. [2025-09-16T17:30:38.017364] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T17:30:38.387412] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-09-16T17:30:38.774944] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-06. [2025-09-16T17:30:39.029222] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T17:30:39.574896] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-04. [2025-09-16T17:30:40.275798] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T17:30:40.374919] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-09. [2025-09-16T17:30:40.441475] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-09-16T17:30:41.174902] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-07. [2025-09-16T17:30:41.561612] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-09-16T17:30:41.564966] [INFO ] [REMOTE ] BKV: Full fetch complete (33315 records). [2025-09-16T17:30:41.565086] [DEBUG] [VALIDATOR ] BKV: Validating primary data. [2025-09-16T17:30:41.953354] [DEBUG] [VALIDATOR ] BKV: All expected trading days are present. [2025-09-16T17:30:41.953441] [DEBUG] [VALIDATOR ] BKV: Primary data validation passed. [2025-09-16T17:30:41.953971] [INFO ] [MAIN ] BKV: Fetched primary data is valid. [2025-09-16T17:30:41.954042] [DEBUG] [LOCAL ] BKV: Saving primary data. [2025-09-16T17:30:41.954930] [DEBUG] [LOCAL ] BKV: Trimmed 33315 records to 21066 (keeping 150 days of data) [2025-09-16T17:30:42.009490] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-05. [2025-09-16T17:30:42.086353] [DEBUG] [LOCAL ] BKV: Primary data saved (21066 records). [2025-09-16T17:30:42.086486] [INFO ] [MAIN ] BKV: Full history saved. [2025-09-16T17:30:42.086589] [INFO ] [MAIN ] Update Progress: 295/2369 symbols completed. BKV took 21.4s. Estimated remaining: 3h 35m 29s. Rate: 9.6 symbols/min [2025-09-16T17:30:42.263356] [INFO ] [MAIN ] BLE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:30:42.274861] [DEBUG] [LOCAL ] BLE: Loaded 10139 existing records from primary storage. [2025-09-16T17:30:42.275017] [INFO ] [MAIN ] BLE: Loaded 10139 existing records. [2025-09-16T17:30:42.462423] [WARN ] [MAIN ] BLE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:30:42.462768] [INFO ] [MAIN ] BLE: Wiping existing primary data before full refresh. [2025-09-16T17:30:42.462828] [INFO ] [LOCAL ] BLE: Starting deletion of primary data. [2025-09-16T17:30:42.463443] [DEBUG] [LOCAL ] BLE: Deleted primary data file. [2025-09-16T17:30:42.463507] [INFO ] [MAIN ] BLE: Starting full primary refresh. [2025-09-16T17:30:42.778995] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-03. [2025-09-16T17:30:42.841863] [INFO ] [REMOTE ] BLE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:30:42.867689] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T17:30:42.898210] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-09-16T17:30:43.574898] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-02. [2025-09-16T17:30:44.177663] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T17:30:44.317361] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-09-16T17:30:44.374937] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-06. [2025-09-16T17:30:45.174903] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-04. [2025-09-16T17:30:45.268328] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T17:30:45.974948] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-07. [2025-09-16T17:30:46.774905] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-08. [2025-09-16T17:30:47.059506] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T17:30:47.428148] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T17:30:47.574894] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-05. [2025-09-16T17:30:47.841438] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T17:30:48.374917] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-08. [2025-09-16T17:30:49.175146] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-09. [2025-09-16T17:30:49.422941] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-09-16T17:30:49.482171] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-09-16T17:30:49.974922] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-06. [2025-09-16T17:30:50.159351] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T17:30:50.163186] [INFO ] [REMOTE ] BLCO: Full fetch complete (40073 records). [2025-09-16T17:30:50.163280] [DEBUG] [VALIDATOR ] BLCO: Validating primary data. [2025-09-16T17:30:50.587214] [DEBUG] [VALIDATOR ] BLCO: All expected trading days are present. [2025-09-16T17:30:50.587308] [DEBUG] [VALIDATOR ] BLCO: Primary data validation passed. [2025-09-16T17:30:50.587873] [INFO ] [MAIN ] BLCO: Fetched primary data is valid. [2025-09-16T17:30:50.587933] [DEBUG] [LOCAL ] BLCO: Saving primary data. [2025-09-16T17:30:50.589163] [DEBUG] [LOCAL ] BLCO: Trimmed 40073 records to 26549 (keeping 150 days of data) [2025-09-16T17:30:50.776025] [DEBUG] [LOCAL ] BLCO: Primary data saved (26549 records). [2025-09-16T17:30:50.776160] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-03. [2025-09-16T17:30:50.776497] [INFO ] [MAIN ] BLCO: Full history saved. [2025-09-16T17:30:50.777530] [INFO ] [MAIN ] Update Progress: 296/2369 symbols completed. BLCO took 28.0s. Estimated remaining: 3h 35m 40s. Rate: 9.6 symbols/min [2025-09-16T17:30:50.961615] [INFO ] [MAIN ] BLK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:30:50.986617] [DEBUG] [LOCAL ] BLK: Loaded 24344 existing records from primary storage. [2025-09-16T17:30:50.986781] [INFO ] [MAIN ] BLK: Loaded 24344 existing records. [2025-09-16T17:30:51.184410] [WARN ] [MAIN ] BLK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:30:51.184677] [INFO ] [MAIN ] BLK: Wiping existing primary data before full refresh. [2025-09-16T17:30:51.184728] [INFO ] [LOCAL ] BLK: Starting deletion of primary data. [2025-09-16T17:30:51.185452] [DEBUG] [LOCAL ] BLK: Deleted primary data file. [2025-09-16T17:30:51.185500] [INFO ] [MAIN ] BLK: Starting full primary refresh. [2025-09-16T17:30:51.551404] [INFO ] [REMOTE ] BLK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:30:51.574897] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-02. [2025-09-16T17:30:51.680566] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T17:30:51.707319] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-09-16T17:30:52.374912] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-04. [2025-09-16T17:30:53.174998] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-07. [2025-09-16T17:30:53.358580] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T17:30:53.396699] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-09-16T17:30:53.974906] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-05. [2025-09-16T17:30:54.774927] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-03. [2025-09-16T17:30:54.824805] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T17:30:54.845704] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-09-16T17:30:55.574882] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-06. [2025-09-16T17:30:56.374937] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-09. [2025-09-16T17:30:56.602449] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T17:30:56.645976] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T17:30:57.174917] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-04. [2025-09-16T17:30:57.632936] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T17:30:57.636536] [INFO ] [REMOTE ] BLD: Full fetch complete (30878 records). [2025-09-16T17:30:57.636692] [DEBUG] [VALIDATOR ] BLD: Validating primary data. [2025-09-16T17:30:57.977490] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-08. [2025-09-16T17:30:58.013342] [DEBUG] [VALIDATOR ] BLD: All expected trading days are present. [2025-09-16T17:30:58.013433] [DEBUG] [VALIDATOR ] BLD: Primary data validation passed. [2025-09-16T17:30:58.013931] [INFO ] [MAIN ] BLD: Fetched primary data is valid. [2025-09-16T17:30:58.013981] [DEBUG] [LOCAL ] BLD: Saving primary data. [2025-09-16T17:30:58.015024] [DEBUG] [LOCAL ] BLD: Trimmed 30878 records to 20558 (keeping 150 days of data) [2025-09-16T17:30:58.160415] [DEBUG] [LOCAL ] BLD: Primary data saved (20558 records). [2025-09-16T17:30:58.160563] [INFO ] [MAIN ] BLD: Full history saved. [2025-09-16T17:30:58.160697] [INFO ] [MAIN ] Update Progress: 297/2369 symbols completed. BLD took 26.4s. Estimated remaining: 3h 35m 42s. Rate: 9.6 symbols/min [2025-09-16T17:30:58.333475] [INFO ] [MAIN ] BLND: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:30:58.368758] [DEBUG] [LOCAL ] BLND: Loaded 36059 existing records from primary storage. [2025-09-16T17:30:58.368909] [INFO ] [MAIN ] BLND: Loaded 36059 existing records. [2025-09-16T17:30:58.553207] [WARN ] [MAIN ] BLND: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:30:58.553512] [INFO ] [MAIN ] BLND: Wiping existing primary data before full refresh. [2025-09-16T17:30:58.553570] [INFO ] [LOCAL ] BLND: Starting deletion of primary data. [2025-09-16T17:30:58.554357] [DEBUG] [LOCAL ] BLND: Deleted primary data file. [2025-09-16T17:30:58.554422] [INFO ] [MAIN ] BLND: Starting full primary refresh. [2025-09-16T17:30:58.780636] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-07. [2025-09-16T17:30:58.925075] [INFO ] [REMOTE ] BLND: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:30:59.006560] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T17:30:59.574907] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-05. [2025-09-16T17:30:59.630737] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-09-16T17:31:00.015155] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-08 (2.04 seconds). [2025-09-16T17:31:00.374920] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-02. [2025-09-16T17:31:01.174927] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-08. [2025-09-16T17:31:01.590708] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-05 (2.02 seconds). [2025-09-16T17:31:01.974906] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-09. [2025-09-16T17:31:02.664398] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-02 (2.29 seconds). [2025-09-16T17:31:02.774939] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-03. [2025-09-16T17:31:03.574992] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-06. [2025-09-16T17:31:03.683674] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-08 (2.51 seconds). [2025-09-16T17:31:04.374952] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-09. [2025-09-16T17:31:04.462430] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-09 (2.49 seconds). [2025-09-16T17:31:04.469711] [INFO ] [REMOTE ] BLDR: Full fetch complete (56103 records). [2025-09-16T17:31:04.469939] [DEBUG] [VALIDATOR ] BLDR: Validating primary data. [2025-09-16T17:31:04.710546] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-09-16T17:31:04.955033] [DEBUG] [VALIDATOR ] BLDR: All expected trading days are present. [2025-09-16T17:31:04.955145] [DEBUG] [VALIDATOR ] BLDR: Primary data validation passed. [2025-09-16T17:31:04.955812] [INFO ] [MAIN ] BLDR: Fetched primary data is valid. [2025-09-16T17:31:04.955861] [DEBUG] [LOCAL ] BLDR: Saving primary data. [2025-09-16T17:31:04.957196] [DEBUG] [LOCAL ] BLDR: Trimmed 56103 records to 38691 (keeping 150 days of data) [2025-09-16T17:31:05.175221] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-04. [2025-09-16T17:31:05.213411] [DEBUG] [LOCAL ] BLDR: Primary data saved (38691 records). [2025-09-16T17:31:05.213581] [INFO ] [MAIN ] BLDR: Full history saved. [2025-09-16T17:31:05.213690] [INFO ] [MAIN ] Update Progress: 298/2369 symbols completed. BLDR took 30.6s. Estimated remaining: 3h 35m 41s. Rate: 9.6 symbols/min [2025-09-16T17:31:05.340346] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T17:31:05.398167] [INFO ] [REMOTE ] BLE: Full fetch complete (15618 records). [2025-09-16T17:31:05.408008] [INFO ] [MAIN ] BLSH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:31:05.408279] [DEBUG] [LOCAL ] BLSH: No existing data file found in primary storage. [2025-09-16T17:31:05.408097] [DEBUG] [VALIDATOR ] BLE: Validating primary data. [2025-09-16T17:31:05.408387] [WARN ] [MAIN ] BLSH: No existing data found, performing full refresh instead. [2025-09-16T17:31:05.429656] [INFO ] [MAIN ] BLSH: Wiping existing primary data before full refresh. [2025-09-16T17:31:05.444960] [INFO ] [LOCAL ] BLSH: Starting deletion of primary data. [2025-09-16T17:31:05.451727] [DEBUG] [LOCAL ] BLSH: No primary data file found to delete. [2025-09-16T17:31:05.451928] [INFO ] [MAIN ] BLSH: Starting full primary refresh. [2025-09-16T17:31:05.622610] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-06 (2.05 seconds). [2025-09-16T17:31:05.994200] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-07. [2025-09-16T17:31:06.140213] [INFO ] [REMOTE ] BLSH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:31:06.181841] [DEBUG] [VALIDATOR ] BLE: All expected trading days are present. [2025-09-16T17:31:06.181920] [DEBUG] [VALIDATOR ] BLE: Primary data validation passed. [2025-09-16T17:31:06.182333] [INFO ] [MAIN ] BLE: Fetched primary data is valid. [2025-09-16T17:31:06.182400] [DEBUG] [LOCAL ] BLE: Saving primary data. [2025-09-16T17:31:06.183057] [DEBUG] [LOCAL ] BLE: Trimmed 15618 records to 10217 (keeping 150 days of data) [2025-09-16T17:31:06.249383] [DEBUG] [LOCAL ] BLE: Primary data saved (10217 records). [2025-09-16T17:31:06.249532] [INFO ] [MAIN ] BLE: Full history saved. [2025-09-16T17:31:06.249629] [INFO ] [MAIN ] Update Progress: 299/2369 symbols completed. BLE took 24.2s. Estimated remaining: 3h 34m 59s. Rate: 9.6 symbols/min [2025-09-16T17:31:06.418981] [INFO ] [MAIN ] BLW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:31:06.429044] [DEBUG] [LOCAL ] BLW: Loaded 8953 existing records from primary storage. [2025-09-16T17:31:06.429166] [INFO ] [MAIN ] BLW: Loaded 8953 existing records. [2025-09-16T17:31:06.609723] [WARN ] [MAIN ] BLW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:31:06.610151] [INFO ] [MAIN ] BLW: Wiping existing primary data before full refresh. [2025-09-16T17:31:06.610276] [INFO ] [LOCAL ] BLW: Starting deletion of primary data. [2025-09-16T17:31:06.610769] [DEBUG] [LOCAL ] BLW: Deleted primary data file. [2025-09-16T17:31:06.610837] [INFO ] [MAIN ] BLW: Starting full primary refresh. [2025-09-16T17:31:06.803156] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-02. [2025-09-16T17:31:06.890081] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-09-16T17:31:07.029736] [INFO ] [REMOTE ] BLW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:31:07.121792] [DEBUG] [REMOTE ] BLSH: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:31:07.122102] [WARN ] [REMOTE ] BLSH: No data returned for month 2025-02 [2025-09-16T17:31:07.574858] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-03. [2025-09-16T17:31:07.798945] [DEBUG] [REMOTE ] BLSH: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:31:07.799313] [WARN ] [REMOTE ] BLSH: No data returned for month 2025-03 [2025-09-16T17:31:08.006629] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-09-16T17:31:08.374922] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-02. [2025-09-16T17:31:09.174936] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-04. [2025-09-16T17:31:09.224395] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-09-16T17:31:09.381983] [DEBUG] [REMOTE ] BLSH: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:31:09.382438] [WARN ] [REMOTE ] BLSH: No data returned for month 2025-04 [2025-09-16T17:31:09.974951] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-03. [2025-09-16T17:31:10.708269] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-09-16T17:31:10.774917] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-08. [2025-09-16T17:31:11.574916] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-05. [2025-09-16T17:31:11.763927] [DEBUG] [REMOTE ] BLSH: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:31:11.764265] [WARN ] [REMOTE ] BLSH: No data returned for month 2025-05 [2025-09-16T17:31:12.338168] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T17:31:12.374903] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-09. [2025-09-16T17:31:13.174914] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-05. [2025-09-16T17:31:13.762150] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T17:31:13.765938] [INFO ] [REMOTE ] BLK: Full fetch complete (38881 records). [2025-09-16T17:31:13.766054] [DEBUG] [VALIDATOR ] BLK: Validating primary data. [2025-09-16T17:31:13.766616] [FATAL] [VALIDATOR ] BLK: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:31:13.766720] [FATAL] [MAIN ] BLK: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:31:13.766777] [WARN ] [MAIN ] BLK: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:31:13.766818] [INFO ] [LOCAL ] BLK: Starting deletion of primary data. [2025-09-16T17:31:13.766884] [DEBUG] [LOCAL ] BLK: No primary data file found to delete. [2025-09-16T17:31:13.766938] [INFO ] [LOCAL ] BLK: Starting deletion of derivative data. [2025-09-16T17:31:13.767293] [DEBUG] [LOCAL ] BLK: Deleted derivative data file. [2025-09-16T17:31:13.767384] [INFO ] [MAIN ] Update Progress: 300/2369 symbols completed. BLK took 23.0s. Estimated remaining: 3h 35m 1s. Rate: 9.6 symbols/min [2025-09-16T17:31:13.940073] [INFO ] [MAIN ] BLX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:31:13.940244] [DEBUG] [LOCAL ] BLX: No existing data file found in primary storage. [2025-09-16T17:31:13.940282] [WARN ] [MAIN ] BLX: No existing data found, performing full refresh instead. [2025-09-16T17:31:13.940403] [INFO ] [MAIN ] BLX: Wiping existing primary data before full refresh. [2025-09-16T17:31:13.940441] [INFO ] [LOCAL ] BLX: Starting deletion of primary data. [2025-09-16T17:31:13.940489] [DEBUG] [LOCAL ] BLX: No primary data file found to delete. [2025-09-16T17:31:13.940521] [INFO ] [MAIN ] BLX: Starting full primary refresh. [2025-09-16T17:31:13.983244] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-04. [2025-09-16T17:31:14.299199] [INFO ] [REMOTE ] BLX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:31:14.772456] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-09-16T17:31:14.776281] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-02. [2025-09-16T17:31:15.574894] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-06. [2025-09-16T17:31:15.597264] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T17:31:15.818587] [DEBUG] [REMOTE ] BLSH: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:31:15.818938] [WARN ] [REMOTE ] BLSH: No data returned for month 2025-06 [2025-09-16T17:31:16.186567] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-05 (3.01 seconds). [2025-09-16T17:31:16.374914] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-07. [2025-09-16T17:31:16.575293] [DEBUG] [REMOTE ] BLSH: Error fetching primary data for 2025-07: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:31:16.575595] [WARN ] [REMOTE ] BLSH: No data returned for month 2025-07 [2025-09-16T17:31:17.174936] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-05. [2025-09-16T17:31:17.974910] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-06. [2025-09-16T17:31:18.009802] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T17:31:18.774915] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-06. [2025-09-16T17:31:19.519969] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-09-16T17:31:19.556486] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T17:31:19.574906] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-07. [2025-09-16T17:31:20.161496] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-09-16T17:31:20.374935] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-08. [2025-09-16T17:31:21.174940] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-03. [2025-09-16T17:31:21.903450] [INFO ] [REMOTE ] BLSH: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T17:31:21.974925] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-09. [2025-09-16T17:31:22.130981] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-09-16T17:31:22.774924] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-07. [2025-09-16T17:31:23.575048] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-04. [2025-09-16T17:31:24.039572] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T17:31:24.269925] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-09-16T17:31:24.374911] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-05. [2025-09-16T17:31:24.450456] [INFO ] [REMOTE ] BLSH: Incremental fetch complete for 2025-09 (2.48 seconds). [2025-09-16T17:31:24.457920] [INFO ] [REMOTE ] BLSH: Full fetch complete (9102 records). [2025-09-16T17:31:24.458151] [DEBUG] [VALIDATOR ] BLSH: Validating primary data. [2025-09-16T17:31:24.649077] [WARN ] [VALIDATOR ] BLSH: Validation failed - insufficient historical data. Have 24 unique days, need 150 days [2025-09-16T17:31:24.649367] [FATAL] [MAIN ] BLSH: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:31:24.649421] [WARN ] [MAIN ] BLSH: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:31:24.649467] [INFO ] [LOCAL ] BLSH: Starting deletion of primary data. [2025-09-16T17:31:24.649555] [DEBUG] [LOCAL ] BLSH: No primary data file found to delete. [2025-09-16T17:31:24.649596] [INFO ] [LOCAL ] BLSH: Starting deletion of derivative data. [2025-09-16T17:31:24.649638] [DEBUG] [LOCAL ] BLSH: No derivative data file found to delete. [2025-09-16T17:31:24.649724] [INFO ] [MAIN ] Update Progress: 301/2369 symbols completed. BLSH took 19.4s. Estimated remaining: 3h 35m 27s. Rate: 9.6 symbols/min [2025-09-16T17:31:24.829271] [INFO ] [MAIN ] BMA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:31:24.854627] [DEBUG] [LOCAL ] BMA: Loaded 25268 existing records from primary storage. [2025-09-16T17:31:24.854793] [INFO ] [MAIN ] BMA: Loaded 25268 existing records. [2025-09-16T17:31:25.035554] [WARN ] [MAIN ] BMA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:31:25.035833] [INFO ] [MAIN ] BMA: Wiping existing primary data before full refresh. [2025-09-16T17:31:25.035879] [INFO ] [LOCAL ] BMA: Starting deletion of primary data. [2025-09-16T17:31:25.036589] [DEBUG] [LOCAL ] BMA: Deleted primary data file. [2025-09-16T17:31:25.036638] [INFO ] [MAIN ] BMA: Starting full primary refresh. [2025-09-16T17:31:25.195199] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-08. [2025-09-16T17:31:25.327865] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-09-16T17:31:25.419401] [INFO ] [REMOTE ] BMA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:31:25.974945] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-02. [2025-09-16T17:31:26.641558] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T17:31:26.774909] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-09. [2025-09-16T17:31:27.548896] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T17:31:27.574892] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-03. [2025-09-16T17:31:28.374923] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-06. [2025-09-16T17:31:28.451299] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T17:31:28.457402] [INFO ] [REMOTE ] BLND: Full fetch complete (53942 records). [2025-09-16T17:31:28.457500] [DEBUG] [VALIDATOR ] BLND: Validating primary data. [2025-09-16T17:31:28.839023] [DEBUG] [VALIDATOR ] BLND: All expected trading days are present. [2025-09-16T17:31:28.839152] [DEBUG] [VALIDATOR ] BLND: Primary data validation passed. [2025-09-16T17:31:28.839863] [INFO ] [MAIN ] BLND: Fetched primary data is valid. [2025-09-16T17:31:28.840004] [DEBUG] [LOCAL ] BLND: Saving primary data. [2025-09-16T17:31:28.841299] [DEBUG] [LOCAL ] BLND: Trimmed 53942 records to 36409 (keeping 150 days of data) [2025-09-16T17:31:28.954579] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-09-16T17:31:29.082942] [DEBUG] [LOCAL ] BLND: Primary data saved (36409 records). [2025-09-16T17:31:29.083093] [INFO ] [MAIN ] BLND: Full history saved. [2025-09-16T17:31:29.083188] [INFO ] [MAIN ] Update Progress: 302/2369 symbols completed. BLND took 30.9s. Estimated remaining: 3h 35m 8s. Rate: 9.6 symbols/min [2025-09-16T17:31:29.180066] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-04. [2025-09-16T17:31:29.256871] [INFO ] [MAIN ] BME: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:31:29.257069] [DEBUG] [LOCAL ] BME: No existing data file found in primary storage. [2025-09-16T17:31:29.257108] [WARN ] [MAIN ] BME: No existing data found, performing full refresh instead. [2025-09-16T17:31:29.257232] [INFO ] [MAIN ] BME: Wiping existing primary data before full refresh. [2025-09-16T17:31:29.257272] [INFO ] [LOCAL ] BME: Starting deletion of primary data. [2025-09-16T17:31:29.257314] [DEBUG] [LOCAL ] BME: No primary data file found to delete. [2025-09-16T17:31:29.257346] [INFO ] [MAIN ] BME: Starting full primary refresh. [2025-09-16T17:31:29.603796] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T17:31:29.625577] [INFO ] [REMOTE ] BME: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:31:29.974904] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-07. [2025-09-16T17:31:30.616714] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-09-16T17:31:30.640685] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T17:31:30.774898] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-08. [2025-09-16T17:31:31.574923] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-08. [2025-09-16T17:31:31.631761] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T17:31:32.374950] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-05. [2025-09-16T17:31:32.508853] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T17:31:33.174918] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-09. [2025-09-16T17:31:33.539873] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-09-16T17:31:33.942254] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-09-16T17:31:33.944894] [INFO ] [REMOTE ] BLW: Full fetch complete (14098 records). [2025-09-16T17:31:33.944988] [DEBUG] [VALIDATOR ] BLW: Validating primary data. [2025-09-16T17:31:33.980948] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-06. [2025-09-16T17:31:34.348258] [DEBUG] [VALIDATOR ] BLW: All expected trading days are present. [2025-09-16T17:31:34.348340] [DEBUG] [VALIDATOR ] BLW: Primary data validation passed. [2025-09-16T17:31:34.348627] [INFO ] [MAIN ] BLW: Fetched primary data is valid. [2025-09-16T17:31:34.348686] [DEBUG] [LOCAL ] BLW: Saving primary data. [2025-09-16T17:31:34.349177] [DEBUG] [LOCAL ] BLW: Trimmed 14098 records to 9090 (keeping 150 days of data) [2025-09-16T17:31:34.406774] [DEBUG] [LOCAL ] BLW: Primary data saved (9090 records). [2025-09-16T17:31:34.406937] [INFO ] [MAIN ] BLW: Full history saved. [2025-09-16T17:31:34.407030] [INFO ] [MAIN ] Update Progress: 303/2369 symbols completed. BLW took 28.2s. Estimated remaining: 3h 34m 56s. Rate: 9.6 symbols/min [2025-09-16T17:31:34.575725] [INFO ] [MAIN ] BMEZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:31:34.594327] [DEBUG] [LOCAL ] BMEZ: Loaded 17633 existing records from primary storage. [2025-09-16T17:31:34.594517] [INFO ] [MAIN ] BMEZ: Loaded 17633 existing records. [2025-09-16T17:31:34.769333] [WARN ] [MAIN ] BMEZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:31:34.769573] [INFO ] [MAIN ] BMEZ: Wiping existing primary data before full refresh. [2025-09-16T17:31:34.769622] [INFO ] [LOCAL ] BMEZ: Starting deletion of primary data. [2025-09-16T17:31:34.770174] [DEBUG] [LOCAL ] BMEZ: Deleted primary data file. [2025-09-16T17:31:34.770221] [INFO ] [MAIN ] BMEZ: Starting full primary refresh. [2025-09-16T17:31:34.781436] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-09. [2025-09-16T17:31:35.167581] [INFO ] [REMOTE ] BMEZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:31:35.511573] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-09-16T17:31:35.574901] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-07. [2025-09-16T17:31:35.824209] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-09-16T17:31:35.826246] [INFO ] [REMOTE ] BLX: Full fetch complete (17012 records). [2025-09-16T17:31:35.826337] [DEBUG] [VALIDATOR ] BLX: Validating primary data. [2025-09-16T17:31:35.826769] [FATAL] [VALIDATOR ] BLX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:31:35.826841] [FATAL] [MAIN ] BLX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:31:35.826878] [WARN ] [MAIN ] BLX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:31:35.826919] [INFO ] [LOCAL ] BLX: Starting deletion of primary data. [2025-09-16T17:31:35.826975] [DEBUG] [LOCAL ] BLX: No primary data file found to delete. [2025-09-16T17:31:35.827010] [INFO ] [LOCAL ] BLX: Starting deletion of derivative data. [2025-09-16T17:31:35.827060] [DEBUG] [LOCAL ] BLX: No derivative data file found to delete. [2025-09-16T17:31:35.827115] [INFO ] [MAIN ] Update Progress: 304/2369 symbols completed. BLX took 22.1s. Estimated remaining: 3h 34m 17s. Rate: 9.6 symbols/min [2025-09-16T17:31:35.998099] [INFO ] [MAIN ] BMI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:31:36.016254] [DEBUG] [LOCAL ] BMI: Loaded 17207 existing records from primary storage. [2025-09-16T17:31:36.016391] [INFO ] [MAIN ] BMI: Loaded 17207 existing records. [2025-09-16T17:31:36.203821] [WARN ] [MAIN ] BMI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:31:36.204073] [INFO ] [MAIN ] BMI: Wiping existing primary data before full refresh. [2025-09-16T17:31:36.204120] [INFO ] [LOCAL ] BMI: Starting deletion of primary data. [2025-09-16T17:31:36.204633] [DEBUG] [LOCAL ] BMI: Deleted primary data file. [2025-09-16T17:31:36.204712] [INFO ] [MAIN ] BMI: Starting full primary refresh. [2025-09-16T17:31:36.383571] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-02. [2025-09-16T17:31:36.589603] [INFO ] [REMOTE ] BMI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:31:37.057463] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T17:31:37.174927] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-08. [2025-09-16T17:31:37.525675] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-09-16T17:31:37.974928] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-02. [2025-09-16T17:31:38.758482] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T17:31:38.774874] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-09. [2025-09-16T17:31:39.000038] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-09-16T17:31:39.574916] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-02. [2025-09-16T17:31:40.374911] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-03. [2025-09-16T17:31:40.417978] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T17:31:40.424634] [INFO ] [REMOTE ] BMA: Full fetch complete (37712 records). [2025-09-16T17:31:40.424814] [DEBUG] [VALIDATOR ] BMA: Validating primary data. [2025-09-16T17:31:40.876476] [DEBUG] [VALIDATOR ] BMA: All expected trading days are present. [2025-09-16T17:31:40.876581] [DEBUG] [VALIDATOR ] BMA: Primary data validation passed. [2025-09-16T17:31:40.877113] [INFO ] [MAIN ] BMA: Fetched primary data is valid. [2025-09-16T17:31:40.877161] [DEBUG] [LOCAL ] BMA: Saving primary data. [2025-09-16T17:31:40.878070] [DEBUG] [LOCAL ] BMA: Trimmed 37712 records to 25533 (keeping 150 days of data) [2025-09-16T17:31:41.060542] [DEBUG] [LOCAL ] BMA: Primary data saved (25533 records). [2025-09-16T17:31:41.060748] [INFO ] [MAIN ] BMA: Full history saved. [2025-09-16T17:31:41.060840] [INFO ] [MAIN ] Update Progress: 305/2369 symbols completed. BMA took 16.4s. Estimated remaining: 3h 34m 4s. Rate: 9.6 symbols/min [2025-09-16T17:31:41.179997] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-03. [2025-09-16T17:31:41.241184] [INFO ] [MAIN ] BMN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:31:41.241614] [DEBUG] [LOCAL ] BMN: No existing data file found in primary storage. [2025-09-16T17:31:41.241732] [WARN ] [MAIN ] BMN: No existing data found, performing full refresh instead. [2025-09-16T17:31:41.248989] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-09-16T17:31:41.248741] [INFO ] [MAIN ] BMN: Wiping existing primary data before full refresh. [2025-09-16T17:31:41.249176] [INFO ] [LOCAL ] BMN: Starting deletion of primary data. [2025-09-16T17:31:41.251876] [DEBUG] [LOCAL ] BMN: No primary data file found to delete. [2025-09-16T17:31:41.251977] [INFO ] [MAIN ] BMN: Starting full primary refresh. [2025-09-16T17:31:41.384793] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-09-16T17:31:41.637260] [INFO ] [REMOTE ] BMN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:31:41.974894] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-04. [2025-09-16T17:31:42.496897] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-09-16T17:31:42.774910] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-04. [2025-09-16T17:31:42.985887] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-09-16T17:31:43.574918] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-02. [2025-09-16T17:31:43.780901] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-09-16T17:31:44.148466] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-09-16T17:31:44.374901] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-05. [2025-09-16T17:31:45.174869] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-03. [2025-09-16T17:31:45.479878] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-09-16T17:31:45.494658] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T17:31:45.974897] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-06. [2025-09-16T17:31:46.774905] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-05. [2025-09-16T17:31:46.837462] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-09-16T17:31:47.574902] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-07. [2025-09-16T17:31:47.736275] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-09-16T17:31:48.375777] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-06. [2025-09-16T17:31:48.378372] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T17:31:49.131036] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-09-16T17:31:49.174899] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-04. [2025-09-16T17:31:49.760057] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-09-16T17:31:49.974893] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-07. [2025-09-16T17:31:50.774908] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-08. [2025-09-16T17:31:50.807785] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-09-16T17:31:51.574889] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-08. [2025-09-16T17:31:51.671937] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-09-16T17:31:52.374903] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-03. [2025-09-16T17:31:52.423920] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-09-16T17:31:53.174918] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-09. [2025-09-16T17:31:53.347030] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-09-16T17:31:53.951256] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-09-16T17:31:53.953071] [INFO ] [REMOTE ] BME: Full fetch complete (11453 records). [2025-09-16T17:31:53.953167] [DEBUG] [VALIDATOR ] BME: Validating primary data. [2025-09-16T17:31:53.983894] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-05. [2025-09-16T17:31:54.342757] [DEBUG] [VALIDATOR ] BME: All expected trading days are present. [2025-09-16T17:31:54.342889] [DEBUG] [VALIDATOR ] BME: Primary data validation passed. [2025-09-16T17:31:54.343183] [INFO ] [MAIN ] BME: Fetched primary data is valid. [2025-09-16T17:31:54.343225] [DEBUG] [LOCAL ] BME: Saving primary data. [2025-09-16T17:31:54.343785] [DEBUG] [LOCAL ] BME: Trimmed 11453 records to 8162 (keeping 150 days of data) [2025-09-16T17:31:54.395483] [DEBUG] [LOCAL ] BME: Primary data saved (8162 records). [2025-09-16T17:31:54.395623] [INFO ] [MAIN ] BME: Full history saved. [2025-09-16T17:31:54.395740] [INFO ] [MAIN ] Update Progress: 306/2369 symbols completed. BME took 25.3s. Estimated remaining: 3h 34m 46s. Rate: 9.6 symbols/min [2025-09-16T17:31:54.582018] [INFO ] [MAIN ] BMO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:31:54.582242] [DEBUG] [LOCAL ] BMO: No existing data file found in primary storage. [2025-09-16T17:31:54.582312] [WARN ] [MAIN ] BMO: No existing data found, performing full refresh instead. [2025-09-16T17:31:54.582446] [INFO ] [MAIN ] BMO: Wiping existing primary data before full refresh. [2025-09-16T17:31:54.582488] [INFO ] [LOCAL ] BMO: Starting deletion of primary data. [2025-09-16T17:31:54.582531] [DEBUG] [LOCAL ] BMO: No primary data file found to delete. [2025-09-16T17:31:54.582566] [INFO ] [MAIN ] BMO: Starting full primary refresh. [2025-09-16T17:31:54.709003] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T17:31:54.790385] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-04. [2025-09-16T17:31:55.045390] [INFO ] [REMOTE ] BMO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:31:55.574967] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-02. [2025-09-16T17:31:56.313013] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T17:31:56.374870] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-05. [2025-09-16T17:31:57.174910] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-06. [2025-09-16T17:31:57.427896] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-09-16T17:31:57.537275] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-09-16T17:31:57.628530] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-09-16T17:31:57.974898] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-07. [2025-09-16T17:31:58.432393] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-09-16T17:31:58.774908] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-09. [2025-09-16T17:31:59.574884] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-03. [2025-09-16T17:31:59.789490] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T17:31:59.792526] [INFO ] [REMOTE ] BMEZ: Full fetch complete (28458 records). [2025-09-16T17:31:59.792601] [DEBUG] [VALIDATOR ] BMEZ: Validating primary data. [2025-09-16T17:32:00.161740] [DEBUG] [VALIDATOR ] BMEZ: All expected trading days are present. [2025-09-16T17:32:00.161947] [DEBUG] [VALIDATOR ] BMEZ: Primary data validation passed. [2025-09-16T17:32:00.162408] [INFO ] [MAIN ] BMEZ: Fetched primary data is valid. [2025-09-16T17:32:00.162455] [DEBUG] [LOCAL ] BMEZ: Saving primary data. [2025-09-16T17:32:00.163229] [DEBUG] [LOCAL ] BMEZ: Trimmed 28458 records to 17832 (keeping 150 days of data) [2025-09-16T17:32:00.274120] [DEBUG] [LOCAL ] BMEZ: Primary data saved (17832 records). [2025-09-16T17:32:00.274274] [INFO ] [MAIN ] BMEZ: Full history saved. [2025-09-16T17:32:00.274360] [INFO ] [MAIN ] Update Progress: 307/2369 symbols completed. BMEZ took 25.9s. Estimated remaining: 3h 34m 37s. Rate: 9.6 symbols/min [2025-09-16T17:32:00.375206] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-08. [2025-09-16T17:32:00.449104] [INFO ] [MAIN ] BMY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:00.490414] [DEBUG] [LOCAL ] BMY: Loaded 39717 existing records from primary storage. [2025-09-16T17:32:00.490581] [INFO ] [MAIN ] BMY: Loaded 39717 existing records. [2025-09-16T17:32:00.670522] [WARN ] [MAIN ] BMY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:32:00.670842] [INFO ] [MAIN ] BMY: Wiping existing primary data before full refresh. [2025-09-16T17:32:00.670891] [INFO ] [LOCAL ] BMY: Starting deletion of primary data. [2025-09-16T17:32:00.672080] [DEBUG] [LOCAL ] BMY: Deleted primary data file. [2025-09-16T17:32:00.672133] [INFO ] [MAIN ] BMY: Starting full primary refresh. [2025-09-16T17:32:01.034609] [INFO ] [REMOTE ] BMY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:01.174890] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-02. [2025-09-16T17:32:01.299964] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-09-16T17:32:01.828701] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-03 (2.25 seconds). [2025-09-16T17:32:01.974890] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-09. [2025-09-16T17:32:02.785514] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-06. [2025-09-16T17:32:02.795407] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T17:32:02.826802] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T17:32:02.828235] [INFO ] [REMOTE ] BMN: Full fetch complete (2749 records). [2025-09-16T17:32:02.828321] [DEBUG] [VALIDATOR ] BMN: Validating primary data. [2025-09-16T17:32:02.828887] [FATAL] [VALIDATOR ] BMN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:32:02.828989] [FATAL] [MAIN ] BMN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:32:02.829041] [WARN ] [MAIN ] BMN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:32:02.829107] [INFO ] [LOCAL ] BMN: Starting deletion of primary data. [2025-09-16T17:32:02.829194] [DEBUG] [LOCAL ] BMN: No primary data file found to delete. [2025-09-16T17:32:02.829250] [INFO ] [LOCAL ] BMN: Starting deletion of derivative data. [2025-09-16T17:32:02.829308] [DEBUG] [LOCAL ] BMN: No derivative data file found to delete. [2025-09-16T17:32:02.829916] [INFO ] [MAIN ] Update Progress: 308/2369 symbols completed. BMN took 21.8s. Estimated remaining: 3h 34m 6s. Rate: 9.6 symbols/min [2025-09-16T17:32:02.997863] [INFO ] [MAIN ] BN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:03.037495] [DEBUG] [LOCAL ] BN: Loaded 39006 existing records from primary storage. [2025-09-16T17:32:03.037680] [INFO ] [MAIN ] BN: Loaded 39006 existing records. [2025-09-16T17:32:03.216823] [WARN ] [MAIN ] BN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:32:03.217087] [INFO ] [MAIN ] BN: Wiping existing primary data before full refresh. [2025-09-16T17:32:03.217133] [INFO ] [LOCAL ] BN: Starting deletion of primary data. [2025-09-16T17:32:03.218103] [DEBUG] [LOCAL ] BN: Deleted primary data file. [2025-09-16T17:32:03.218153] [INFO ] [MAIN ] BN: Starting full primary refresh. [2025-09-16T17:32:03.572634] [INFO ] [REMOTE ] BN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:03.574853] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-02. [2025-09-16T17:32:04.011964] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T17:32:04.374964] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-07. [2025-09-16T17:32:05.174923] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-03. [2025-09-16T17:32:05.374616] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-09-16T17:32:05.503545] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-09-16T17:32:05.974904] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-04. [2025-09-16T17:32:06.774907] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-03. [2025-09-16T17:32:07.180552] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-09-16T17:32:07.406407] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:32:07.574863] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-05. [2025-09-16T17:32:08.374905] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-04. [2025-09-16T17:32:08.702507] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-09-16T17:32:08.772141] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T17:32:09.174902] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-04. [2025-09-16T17:32:09.974870] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-08. [2025-09-16T17:32:10.140486] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T17:32:10.742033] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-09-16T17:32:10.760329] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T17:32:10.774850] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-09. [2025-09-16T17:32:11.574905] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-05. [2025-09-16T17:32:12.207897] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-09-16T17:32:12.211248] [INFO ] [REMOTE ] BMI: Full fetch complete (25135 records). [2025-09-16T17:32:12.211335] [DEBUG] [VALIDATOR ] BMI: Validating primary data. [2025-09-16T17:32:12.211877] [FATAL] [VALIDATOR ] BMI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:32:12.211953] [FATAL] [MAIN ] BMI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:32:12.211989] [WARN ] [MAIN ] BMI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:32:12.212028] [INFO ] [LOCAL ] BMI: Starting deletion of primary data. [2025-09-16T17:32:12.212097] [DEBUG] [LOCAL ] BMI: No primary data file found to delete. [2025-09-16T17:32:12.212135] [INFO ] [LOCAL ] BMI: Starting deletion of derivative data. [2025-09-16T17:32:12.212482] [DEBUG] [LOCAL ] BMI: Deleted derivative data file. [2025-09-16T17:32:12.212565] [INFO ] [MAIN ] Update Progress: 309/2369 symbols completed. BMI took 36.4s. Estimated remaining: 3h 34m 21s. Rate: 9.6 symbols/min [2025-09-16T17:32:12.382614] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-05. [2025-09-16T17:32:12.385885] [INFO ] [MAIN ] BNED: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:12.407573] [DEBUG] [LOCAL ] BNED: Loaded 21697 existing records from primary storage. [2025-09-16T17:32:12.407729] [INFO ] [MAIN ] BNED: Loaded 21697 existing records. [2025-09-16T17:32:12.591310] [WARN ] [MAIN ] BNED: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:32:12.591559] [INFO ] [MAIN ] BNED: Wiping existing primary data before full refresh. [2025-09-16T17:32:12.591607] [INFO ] [LOCAL ] BNED: Starting deletion of primary data. [2025-09-16T17:32:12.592180] [DEBUG] [LOCAL ] BNED: Deleted primary data file. [2025-09-16T17:32:12.592253] [INFO ] [MAIN ] BNED: Starting full primary refresh. [2025-09-16T17:32:12.949476] [INFO ] [REMOTE ] BNED: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:13.174905] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-02. [2025-09-16T17:32:13.392149] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-09-16T17:32:13.974910] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-06. [2025-09-16T17:32:14.443147] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T17:32:14.478037] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-05 (2.10 seconds). [2025-09-16T17:32:14.774890] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-06. [2025-09-16T17:32:15.558691] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T17:32:15.574853] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-07. [2025-09-16T17:32:16.031444] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T17:32:16.374901] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-07. [2025-09-16T17:32:16.978403] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-09-16T17:32:17.174897] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-08. [2025-09-16T17:32:17.907350] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-09-16T17:32:17.974926] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-06. [2025-09-16T17:32:18.780428] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-08. [2025-09-16T17:32:18.786132] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-09-16T17:32:19.574929] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-09. [2025-09-16T17:32:19.932887] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-06 (1.96 seconds). [2025-09-16T17:32:20.223628] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T17:32:20.374905] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-09. [2025-09-16T17:32:21.174898] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-03. [2025-09-16T17:32:21.280258] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T17:32:21.286774] [INFO ] [REMOTE ] BN: Full fetch complete (57648 records). [2025-09-16T17:32:21.286900] [DEBUG] [VALIDATOR ] BN: Validating primary data. [2025-09-16T17:32:21.681988] [DEBUG] [VALIDATOR ] BN: All expected trading days are present. [2025-09-16T17:32:21.682133] [DEBUG] [VALIDATOR ] BN: Primary data validation passed. [2025-09-16T17:32:21.683025] [INFO ] [MAIN ] BN: Fetched primary data is valid. [2025-09-16T17:32:21.683130] [DEBUG] [LOCAL ] BN: Saving primary data. [2025-09-16T17:32:21.685102] [DEBUG] [LOCAL ] BN: Trimmed 57648 records to 39392 (keeping 150 days of data) [2025-09-16T17:32:21.928027] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-09-16T17:32:21.934335] [INFO ] [REMOTE ] BMO: Full fetch complete (44685 records). [2025-09-16T17:32:21.934944] [DEBUG] [VALIDATOR ] BMO: Validating primary data. [2025-09-16T17:32:21.936421] [FATAL] [VALIDATOR ] BMO: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:32:21.937175] [FATAL] [MAIN ] BMO: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:32:21.937911] [WARN ] [MAIN ] BMO: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:32:21.938907] [INFO ] [LOCAL ] BMO: Starting deletion of primary data. [2025-09-16T17:32:21.939456] [DEBUG] [LOCAL ] BMO: No primary data file found to delete. [2025-09-16T17:32:21.939967] [INFO ] [LOCAL ] BMO: Starting deletion of derivative data. [2025-09-16T17:32:21.940769] [DEBUG] [LOCAL ] BMO: No derivative data file found to delete. [2025-09-16T17:32:21.941781] [INFO ] [MAIN ] Update Progress: 310/2369 symbols completed. BMO took 27.5s. Estimated remaining: 3h 34m 38s. Rate: 9.6 symbols/min [2025-09-16T17:32:21.975014] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-07. [2025-09-16T17:32:22.133324] [INFO ] [MAIN ] BNH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:22.134315] [DEBUG] [LOCAL ] BNH: No existing data file found in primary storage. [2025-09-16T17:32:22.134812] [WARN ] [MAIN ] BNH: No existing data found, performing full refresh instead. [2025-09-16T17:32:22.135993] [INFO ] [MAIN ] BNH: Wiping existing primary data before full refresh. [2025-09-16T17:32:22.136557] [INFO ] [LOCAL ] BNH: Starting deletion of primary data. [2025-09-16T17:32:22.137104] [DEBUG] [LOCAL ] BNH: No primary data file found to delete. [2025-09-16T17:32:22.137585] [INFO ] [MAIN ] BNH: Starting full primary refresh. [2025-09-16T17:32:22.540124] [DEBUG] [LOCAL ] BN: Primary data saved (39392 records). [2025-09-16T17:32:22.540029] [INFO ] [REMOTE ] BNH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:22.541381] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T17:32:22.541502] [INFO ] [MAIN ] BN: Full history saved. [2025-09-16T17:32:22.549870] [INFO ] [MAIN ] Update Progress: 311/2369 symbols completed. BN took 19.7s. Estimated remaining: 3h 33m 54s. Rate: 9.6 symbols/min [2025-09-16T17:32:22.721584] [INFO ] [MAIN ] BNJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:22.725792] [DEBUG] [LOCAL ] BNJ: Loaded 1727 existing records from primary storage. [2025-09-16T17:32:22.725916] [INFO ] [MAIN ] BNJ: Loaded 1727 existing records. [2025-09-16T17:32:22.781287] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-02. [2025-09-16T17:32:22.906015] [WARN ] [MAIN ] BNJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:32:22.906242] [INFO ] [MAIN ] BNJ: Wiping existing primary data before full refresh. [2025-09-16T17:32:22.906291] [INFO ] [LOCAL ] BNJ: Starting deletion of primary data. [2025-09-16T17:32:22.906530] [DEBUG] [LOCAL ] BNJ: Deleted primary data file. [2025-09-16T17:32:22.906594] [INFO ] [MAIN ] BNJ: Starting full primary refresh. [2025-09-16T17:32:23.263242] [INFO ] [REMOTE ] BNJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:23.383041] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-09-16T17:32:23.574911] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-02. [2025-09-16T17:32:24.031707] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-09-16T17:32:24.177714] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-09-16T17:32:24.374917] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-03. [2025-09-16T17:32:24.726837] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-09-16T17:32:25.174898] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-04. [2025-09-16T17:32:25.906761] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-09-16T17:32:25.974920] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-08. [2025-09-16T17:32:26.774899] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-03. [2025-09-16T17:32:27.252674] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-09-16T17:32:27.574856] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-04. [2025-09-16T17:32:27.615306] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T17:32:28.019771] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-09-16T17:32:28.374873] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-09. [2025-09-16T17:32:29.174935] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-04. [2025-09-16T17:32:29.193835] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-09-16T17:32:29.200193] [INFO ] [REMOTE ] BMY: Full fetch complete (58437 records). [2025-09-16T17:32:29.200335] [DEBUG] [VALIDATOR ] BMY: Validating primary data. [2025-09-16T17:32:29.589135] [DEBUG] [VALIDATOR ] BMY: All expected trading days are present. [2025-09-16T17:32:29.589221] [DEBUG] [VALIDATOR ] BMY: Primary data validation passed. [2025-09-16T17:32:29.589924] [INFO ] [MAIN ] BMY: Fetched primary data is valid. [2025-09-16T17:32:29.589971] [DEBUG] [LOCAL ] BMY: Saving primary data. [2025-09-16T17:32:29.591161] [DEBUG] [LOCAL ] BMY: Trimmed 58437 records to 40107 (keeping 150 days of data) [2025-09-16T17:32:29.837195] [DEBUG] [LOCAL ] BMY: Primary data saved (40107 records). [2025-09-16T17:32:29.837321] [INFO ] [MAIN ] BMY: Full history saved. [2025-09-16T17:32:29.837418] [INFO ] [MAIN ] Update Progress: 312/2369 symbols completed. BMY took 29.6s. Estimated remaining: 3h 33m 55s. Rate: 9.6 symbols/min [2025-09-16T17:32:29.981200] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-05. [2025-09-16T17:32:30.013964] [INFO ] [MAIN ] BNL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:30.048254] [DEBUG] [LOCAL ] BNL: Loaded 34424 existing records from primary storage. [2025-09-16T17:32:30.048392] [INFO ] [MAIN ] BNL: Loaded 34424 existing records. [2025-09-16T17:32:30.228271] [WARN ] [MAIN ] BNL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:32:30.228518] [INFO ] [MAIN ] BNL: Wiping existing primary data before full refresh. [2025-09-16T17:32:30.228565] [INFO ] [LOCAL ] BNL: Starting deletion of primary data. [2025-09-16T17:32:30.229371] [DEBUG] [LOCAL ] BNL: Deleted primary data file. [2025-09-16T17:32:30.229427] [INFO ] [MAIN ] BNL: Starting full primary refresh. [2025-09-16T17:32:30.611578] [INFO ] [REMOTE ] BNL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:30.612672] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-09-16T17:32:30.632208] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T17:32:30.774900] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-05. [2025-09-16T17:32:31.574907] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-02. [2025-09-16T17:32:32.120793] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T17:32:32.374901] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-06. [2025-09-16T17:32:32.930977] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T17:32:33.174873] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-03. [2025-09-16T17:32:33.587098] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T17:32:33.974907] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-05. [2025-09-16T17:32:34.447510] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-09-16T17:32:34.774876] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-07. [2025-09-16T17:32:34.844291] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T17:32:35.574908] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-06. [2025-09-16T17:32:35.856278] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-06 (0.28 seconds). [2025-09-16T17:32:36.171265] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-09-16T17:32:36.374904] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-06. [2025-09-16T17:32:36.884540] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-09-16T17:32:37.174914] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-07. [2025-09-16T17:32:37.723481] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-09-16T17:32:37.974900] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-08. [2025-09-16T17:32:38.423330] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-09-16T17:32:38.774905] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-09. [2025-09-16T17:32:39.226181] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-09-16T17:32:39.227191] [INFO ] [REMOTE ] BNH: Full fetch complete (5145 records). [2025-09-16T17:32:39.227258] [DEBUG] [VALIDATOR ] BNH: Validating primary data. [2025-09-16T17:32:39.582947] [DEBUG] [VALIDATOR ] BNH: All expected trading days are present. [2025-09-16T17:32:39.582586] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-08. [2025-09-16T17:32:39.584279] [DEBUG] [VALIDATOR ] BNH: Primary data validation passed. [2025-09-16T17:32:39.584487] [INFO ] [MAIN ] BNH: Fetched primary data is valid. [2025-09-16T17:32:39.584531] [DEBUG] [LOCAL ] BNH: Saving primary data. [2025-09-16T17:32:39.584931] [DEBUG] [LOCAL ] BNH: Trimmed 5145 records to 3128 (keeping 150 days of data) [2025-09-16T17:32:39.606481] [DEBUG] [LOCAL ] BNH: Primary data saved (3128 records). [2025-09-16T17:32:39.606614] [INFO ] [MAIN ] BNH: Full history saved. [2025-09-16T17:32:39.606720] [INFO ] [MAIN ] Update Progress: 313/2369 symbols completed. BNH took 17.7s. Estimated remaining: 3h 34m 12s. Rate: 9.6 symbols/min [2025-09-16T17:32:39.816088] [INFO ] [MAIN ] BNS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:39.852923] [DEBUG] [LOCAL ] BNS: Loaded 37795 existing records from primary storage. [2025-09-16T17:32:39.853077] [INFO ] [MAIN ] BNS: Loaded 37795 existing records. [2025-09-16T17:32:40.036397] [WARN ] [MAIN ] BNS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:32:40.036670] [INFO ] [MAIN ] BNS: Wiping existing primary data before full refresh. [2025-09-16T17:32:40.036719] [INFO ] [LOCAL ] BNS: Starting deletion of primary data. [2025-09-16T17:32:40.037564] [DEBUG] [LOCAL ] BNS: Deleted primary data file. [2025-09-16T17:32:40.037616] [INFO ] [MAIN ] BNS: Starting full primary refresh. [2025-09-16T17:32:40.377090] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-07. [2025-09-16T17:32:40.397805] [INFO ] [REMOTE ] BNS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:40.814679] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-09-16T17:32:41.026370] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T17:32:41.174877] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-09. [2025-09-16T17:32:41.974912] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-08. [2025-09-16T17:32:42.171572] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T17:32:42.176175] [INFO ] [REMOTE ] BNED: Full fetch complete (32286 records). [2025-09-16T17:32:42.176339] [DEBUG] [VALIDATOR ] BNED: Validating primary data. [2025-09-16T17:32:42.177141] [FATAL] [VALIDATOR ] BNED: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:32:42.177288] [FATAL] [MAIN ] BNED: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:32:42.177355] [WARN ] [MAIN ] BNED: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:32:42.177414] [INFO ] [LOCAL ] BNED: Starting deletion of primary data. [2025-09-16T17:32:42.177476] [DEBUG] [LOCAL ] BNED: No primary data file found to delete. [2025-09-16T17:32:42.177520] [INFO ] [LOCAL ] BNED: Starting deletion of derivative data. [2025-09-16T17:32:42.177923] [DEBUG] [LOCAL ] BNED: Deleted derivative data file. [2025-09-16T17:32:42.178049] [INFO ] [MAIN ] Update Progress: 314/2369 symbols completed. BNED took 30.0s. Estimated remaining: 3h 33m 41s. Rate: 9.6 symbols/min [2025-09-16T17:32:42.364097] [INFO ] [MAIN ] BNT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:42.368064] [DEBUG] [LOCAL ] BNT: Loaded 2060 existing records from primary storage. [2025-09-16T17:32:42.368150] [INFO ] [MAIN ] BNT: Loaded 2060 existing records. [2025-09-16T17:32:42.487276] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-09-16T17:32:42.566152] [WARN ] [MAIN ] BNT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:32:42.566410] [INFO ] [MAIN ] BNT: Wiping existing primary data before full refresh. [2025-09-16T17:32:42.566459] [INFO ] [LOCAL ] BNT: Starting deletion of primary data. [2025-09-16T17:32:42.566709] [DEBUG] [LOCAL ] BNT: Deleted primary data file. [2025-09-16T17:32:42.566758] [INFO ] [MAIN ] BNT: Starting full primary refresh. [2025-09-16T17:32:42.781197] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-04. [2025-09-16T17:32:42.941824] [INFO ] [REMOTE ] BNT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:43.574907] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-02. [2025-09-16T17:32:44.374904] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-09. [2025-09-16T17:32:44.578221] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-09-16T17:32:45.082740] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-09-16T17:32:45.083778] [INFO ] [REMOTE ] BNJ: Full fetch complete (2900 records). [2025-09-16T17:32:45.083854] [DEBUG] [VALIDATOR ] BNJ: Validating primary data. [2025-09-16T17:32:45.177357] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-05. [2025-09-16T17:32:45.402283] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T17:32:45.482748] [DEBUG] [VALIDATOR ] BNJ: All expected trading days are present. [2025-09-16T17:32:45.482820] [DEBUG] [VALIDATOR ] BNJ: Primary data validation passed. [2025-09-16T17:32:45.482930] [INFO ] [MAIN ] BNJ: Fetched primary data is valid. [2025-09-16T17:32:45.482969] [DEBUG] [LOCAL ] BNJ: Saving primary data. [2025-09-16T17:32:45.483306] [DEBUG] [LOCAL ] BNJ: Trimmed 2900 records to 1747 (keeping 150 days of data) [2025-09-16T17:32:45.494969] [DEBUG] [LOCAL ] BNJ: Primary data saved (1747 records). [2025-09-16T17:32:45.495042] [INFO ] [MAIN ] BNJ: Full history saved. [2025-09-16T17:32:45.495130] [INFO ] [MAIN ] Update Progress: 315/2369 symbols completed. BNJ took 22.9s. Estimated remaining: 3h 33m 16s. Rate: 9.6 symbols/min [2025-09-16T17:32:45.668820] [INFO ] [MAIN ] BNY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:45.678971] [DEBUG] [LOCAL ] BNY: Loaded 5301 existing records from primary storage. [2025-09-16T17:32:45.679083] [INFO ] [MAIN ] BNY: Loaded 5301 existing records. [2025-09-16T17:32:45.854705] [WARN ] [MAIN ] BNY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:32:45.854949] [INFO ] [MAIN ] BNY: Wiping existing primary data before full refresh. [2025-09-16T17:32:45.855002] [INFO ] [LOCAL ] BNY: Starting deletion of primary data. [2025-09-16T17:32:45.855324] [DEBUG] [LOCAL ] BNY: Deleted primary data file. [2025-09-16T17:32:45.855370] [INFO ] [MAIN ] BNY: Starting full primary refresh. [2025-09-16T17:32:45.980057] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-02. [2025-09-16T17:32:46.229634] [INFO ] [REMOTE ] BNY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:46.774909] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-03. [2025-09-16T17:32:46.907520] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-09-16T17:32:46.910300] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T17:32:47.574922] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-06. [2025-09-16T17:32:48.374908] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-03. [2025-09-16T17:32:48.501116] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T17:32:48.630544] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-09-16T17:32:48.944149] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-09-16T17:32:49.174898] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-07. [2025-09-16T17:32:49.974901] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-04. [2025-09-16T17:32:50.447666] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-09-16T17:32:50.738296] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-09-16T17:32:50.774874] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-08. [2025-09-16T17:32:51.574906] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-02. [2025-09-16T17:32:52.079101] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-09-16T17:32:52.374894] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-03. [2025-09-16T17:32:52.434632] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T17:32:53.082264] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-09-16T17:32:53.174940] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-05. [2025-09-16T17:32:53.520010] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-09-16T17:32:53.974916] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-09. [2025-09-16T17:32:54.774919] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-06. [2025-09-16T17:32:55.224114] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-09-16T17:32:55.574927] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-04. [2025-09-16T17:32:55.714885] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-09-16T17:32:55.720524] [INFO ] [REMOTE ] BNL: Full fetch complete (50837 records). [2025-09-16T17:32:55.720683] [DEBUG] [VALIDATOR ] BNL: Validating primary data. [2025-09-16T17:32:56.105539] [DEBUG] [VALIDATOR ] BNL: All expected trading days are present. [2025-09-16T17:32:56.105633] [DEBUG] [VALIDATOR ] BNL: Primary data validation passed. [2025-09-16T17:32:56.106703] [INFO ] [MAIN ] BNL: Fetched primary data is valid. [2025-09-16T17:32:56.106769] [DEBUG] [LOCAL ] BNL: Saving primary data. [2025-09-16T17:32:56.108561] [DEBUG] [LOCAL ] BNL: Trimmed 50837 records to 34783 (keeping 150 days of data) [2025-09-16T17:32:56.321718] [DEBUG] [LOCAL ] BNL: Primary data saved (34783 records). [2025-09-16T17:32:56.321885] [INFO ] [MAIN ] BNL: Full history saved. [2025-09-16T17:32:56.321986] [INFO ] [MAIN ] Update Progress: 316/2369 symbols completed. BNL took 26.5s. Estimated remaining: 3h 33m 40s. Rate: 9.6 symbols/min [2025-09-16T17:32:56.380998] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-07. [2025-09-16T17:32:56.504447] [INFO ] [MAIN ] BOC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:32:56.504628] [DEBUG] [LOCAL ] BOC: No existing data file found in primary storage. [2025-09-16T17:32:56.504714] [WARN ] [MAIN ] BOC: No existing data found, performing full refresh instead. [2025-09-16T17:32:56.504865] [INFO ] [MAIN ] BOC: Wiping existing primary data before full refresh. [2025-09-16T17:32:56.504915] [INFO ] [LOCAL ] BOC: Starting deletion of primary data. [2025-09-16T17:32:56.504961] [DEBUG] [LOCAL ] BOC: No primary data file found to delete. [2025-09-16T17:32:56.505002] [INFO ] [MAIN ] BOC: Starting full primary refresh. [2025-09-16T17:32:56.871999] [INFO ] [REMOTE ] BOC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:32:57.119178] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-09-16T17:32:57.174857] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-08. [2025-09-16T17:32:57.581081] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-09-16T17:32:57.582544] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-09-16T17:32:57.974902] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-05. [2025-09-16T17:32:58.774906] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-04. [2025-09-16T17:32:59.574877] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-09. [2025-09-16T17:32:59.658277] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T17:32:59.705809] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T17:33:00.159610] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-09-16T17:33:00.160747] [INFO ] [REMOTE ] BNT: Full fetch complete (3501 records). [2025-09-16T17:33:00.160816] [DEBUG] [VALIDATOR ] BNT: Validating primary data. [2025-09-16T17:33:00.381026] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-05. [2025-09-16T17:33:00.524080] [DEBUG] [VALIDATOR ] BNT: All expected trading days are present. [2025-09-16T17:33:00.524159] [DEBUG] [VALIDATOR ] BNT: Primary data validation passed. [2025-09-16T17:33:00.524311] [INFO ] [MAIN ] BNT: Fetched primary data is valid. [2025-09-16T17:33:00.524368] [DEBUG] [LOCAL ] BNT: Saving primary data. [2025-09-16T17:33:00.524776] [DEBUG] [LOCAL ] BNT: Trimmed 3501 records to 2089 (keeping 150 days of data) [2025-09-16T17:33:00.537903] [DEBUG] [LOCAL ] BNT: Primary data saved (2089 records). [2025-09-16T17:33:00.537962] [INFO ] [MAIN ] BNT: Full history saved. [2025-09-16T17:33:00.538052] [INFO ] [MAIN ] Update Progress: 317/2369 symbols completed. BNT took 18.4s. Estimated remaining: 3h 33m 20s. Rate: 9.6 symbols/min [2025-09-16T17:33:01.174886] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-06. [2025-09-16T17:33:01.228825] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-09-16T17:33:01.974902] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-06. [2025-09-16T17:33:02.708472] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-09-16T17:33:02.774876] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-07. [2025-09-16T17:33:02.880302] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T17:33:03.559429] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-09-16T17:33:03.574866] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-08. [2025-09-16T17:33:04.320497] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T17:33:04.374896] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-09. [2025-09-16T17:33:04.959154] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-09-16T17:33:04.960578] [INFO ] [REMOTE ] BNY: Full fetch complete (7706 records). [2025-09-16T17:33:04.960692] [DEBUG] [VALIDATOR ] BNY: Validating primary data. [2025-09-16T17:33:05.184270] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-07. [2025-09-16T17:33:05.325583] [DEBUG] [VALIDATOR ] BNY: All expected trading days are present. [2025-09-16T17:33:05.325670] [DEBUG] [VALIDATOR ] BNY: Primary data validation passed. [2025-09-16T17:33:05.325868] [INFO ] [MAIN ] BNY: Fetched primary data is valid. [2025-09-16T17:33:05.325926] [DEBUG] [LOCAL ] BNY: Saving primary data. [2025-09-16T17:33:05.326319] [DEBUG] [LOCAL ] BNY: Trimmed 7706 records to 5332 (keeping 150 days of data) [2025-09-16T17:33:05.359111] [DEBUG] [LOCAL ] BNY: Primary data saved (5332 records). [2025-09-16T17:33:05.359198] [INFO ] [MAIN ] BNY: Full history saved. [2025-09-16T17:33:05.359279] [INFO ] [MAIN ] Update Progress: 318/2369 symbols completed. BNY took 19.9s. Estimated remaining: 3h 33m 5s. Rate: 9.6 symbols/min [2025-09-16T17:33:05.974937] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-02. [2025-09-16T17:33:06.784771] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T17:33:06.800591] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-08. [2025-09-16T17:33:06.953484] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-09-16T17:33:07.574964] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-03. [2025-09-16T17:33:08.314983] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T17:33:08.374929] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-09. [2025-09-16T17:33:08.549713] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-09-16T17:33:09.174902] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-04. [2025-09-16T17:33:09.781166] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T17:33:09.787794] [INFO ] [REMOTE ] BNS: Full fetch complete (55949 records). [2025-09-16T17:33:09.787917] [DEBUG] [VALIDATOR ] BNS: Validating primary data. [2025-09-16T17:33:10.175305] [DEBUG] [VALIDATOR ] BNS: All expected trading days are present. [2025-09-16T17:33:10.175414] [DEBUG] [VALIDATOR ] BNS: Primary data validation passed. [2025-09-16T17:33:10.176145] [INFO ] [MAIN ] BNS: Fetched primary data is valid. [2025-09-16T17:33:10.176196] [DEBUG] [LOCAL ] BNS: Saving primary data. [2025-09-16T17:33:10.177458] [DEBUG] [LOCAL ] BNS: Trimmed 55949 records to 38179 (keeping 150 days of data) [2025-09-16T17:33:10.292955] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T17:33:10.300665] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-05. [2025-09-16T17:33:10.436485] [DEBUG] [LOCAL ] BNS: Primary data saved (38179 records). [2025-09-16T17:33:10.436635] [INFO ] [MAIN ] BNS: Full history saved. [2025-09-16T17:33:10.436773] [INFO ] [MAIN ] Update Progress: 319/2369 symbols completed. BNS took 30.8s. Estimated remaining: 3h 32m 51s. Rate: 9.6 symbols/min [2025-09-16T17:33:11.277627] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-09-16T17:33:11.281539] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-06. [2025-09-16T17:33:12.065942] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T17:33:12.069813] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-07. [2025-09-16T17:33:13.070343] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T17:33:13.074919] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-08. [2025-09-16T17:33:14.134963] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-09-16T17:33:14.140539] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-09. [2025-09-16T17:33:15.140476] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T17:33:15.144246] [INFO ] [REMOTE ] BOC: Full fetch complete (19671 records). [2025-09-16T17:33:15.144393] [DEBUG] [VALIDATOR ] BOC: Validating primary data. [2025-09-16T17:33:15.506144] [DEBUG] [VALIDATOR ] BOC: All expected trading days are present. [2025-09-16T17:33:15.506225] [DEBUG] [VALIDATOR ] BOC: Primary data validation passed. [2025-09-16T17:33:15.506595] [INFO ] [MAIN ] BOC: Fetched primary data is valid. [2025-09-16T17:33:15.506635] [DEBUG] [LOCAL ] BOC: Saving primary data. [2025-09-16T17:33:15.507264] [DEBUG] [LOCAL ] BOC: Trimmed 19671 records to 13393 (keeping 150 days of data) [2025-09-16T17:33:15.593230] [DEBUG] [LOCAL ] BOC: Primary data saved (13393 records). [2025-09-16T17:33:15.593410] [INFO ] [MAIN ] BOC: Full history saved. [2025-09-16T17:33:15.593489] [INFO ] [MAIN ] Update Progress: 320/2369 symbols completed. BOC took 19.3s. Estimated remaining: 3h 32m 38s. Rate: 9.6 symbols/min [2025-09-16T17:33:16.210976] [INFO ] [MAIN ] BOOT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:16.247931] [INFO ] [MAIN ] BOH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:16.296229] [INFO ] [MAIN ] BOE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:16.298696] [INFO ] [MAIN ] BOND: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:16.321912] [DEBUG] [LOCAL ] BOE: Loaded 11861 existing records from primary storage. [2025-09-16T17:33:16.333028] [INFO ] [MAIN ] BOE: Loaded 11861 existing records. [2025-09-16T17:33:16.444582] [DEBUG] [LOCAL ] BOH: Loaded 21842 existing records from primary storage. [2025-09-16T17:33:16.467739] [INFO ] [MAIN ] BOH: Loaded 21842 existing records. [2025-09-16T17:33:16.489367] [DEBUG] [LOCAL ] BOOT: Loaded 29291 existing records from primary storage. [2025-09-16T17:33:16.561380] [DEBUG] [LOCAL ] BOND: Loaded 23537 existing records from primary storage. [2025-09-16T17:33:16.598734] [WARN ] [MAIN ] BOE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:33:16.618370] [INFO ] [MAIN ] BOE: Wiping existing primary data before full refresh. [2025-09-16T17:33:16.598839] [INFO ] [MAIN ] BOOT: Loaded 29291 existing records. [2025-09-16T17:33:16.644473] [INFO ] [MAIN ] BOND: Loaded 23537 existing records. [2025-09-16T17:33:16.634217] [INFO ] [LOCAL ] BOE: Starting deletion of primary data. [2025-09-16T17:33:16.696842] [DEBUG] [LOCAL ] BOE: Deleted primary data file. [2025-09-16T17:33:16.712628] [INFO ] [MAIN ] BOE: Starting full primary refresh. [2025-09-16T17:33:17.013749] [WARN ] [MAIN ] BOH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:33:17.108979] [WARN ] [MAIN ] BOOT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:33:17.109092] [INFO ] [MAIN ] BOH: Wiping existing primary data before full refresh. [2025-09-16T17:33:17.140356] [INFO ] [LOCAL ] BOH: Starting deletion of primary data. [2025-09-16T17:33:17.145741] [INFO ] [MAIN ] BOOT: Wiping existing primary data before full refresh. [2025-09-16T17:33:17.161823] [INFO ] [LOCAL ] BOOT: Starting deletion of primary data. [2025-09-16T17:33:17.176244] [DEBUG] [LOCAL ] BOH: Deleted primary data file. [2025-09-16T17:33:17.204728] [DEBUG] [LOCAL ] BOOT: Deleted primary data file. [2025-09-16T17:33:17.210118] [INFO ] [MAIN ] BOOT: Starting full primary refresh. [2025-09-16T17:33:17.209945] [WARN ] [MAIN ] BOND: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:33:17.225573] [INFO ] [MAIN ] BOH: Starting full primary refresh. [2025-09-16T17:33:17.360395] [INFO ] [MAIN ] BOND: Wiping existing primary data before full refresh. [2025-09-16T17:33:17.447024] [INFO ] [LOCAL ] BOND: Starting deletion of primary data. [2025-09-16T17:33:17.580961] [DEBUG] [LOCAL ] BOND: Deleted primary data file. [2025-09-16T17:33:17.645707] [INFO ] [MAIN ] BOND: Starting full primary refresh. [2025-09-16T17:33:17.926908] [INFO ] [REMOTE ] BOE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:33:17.992573] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-02. [2025-09-16T17:33:18.423692] [INFO ] [REMOTE ] BOH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:33:18.382118] [INFO ] [REMOTE ] BOOT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:33:18.451834] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-02. [2025-09-16T17:33:18.452847] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-02. [2025-09-16T17:33:18.568591] [INFO ] [REMOTE ] BOND: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:33:18.568917] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-02. [2025-09-16T17:33:19.125493] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-09-16T17:33:19.129745] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-03. [2025-09-16T17:33:19.627049] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T17:33:19.633253] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-03. [2025-09-16T17:33:19.761873] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T17:33:19.770394] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-03. [2025-09-16T17:33:19.923138] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T17:33:19.927482] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-04. [2025-09-16T17:33:20.118242] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T17:33:20.374942] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-03. [2025-09-16T17:33:20.654843] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-09-16T17:33:20.795759] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-09-16T17:33:21.075133] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T17:33:21.174880] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-05. [2025-09-16T17:33:21.625117] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-09-16T17:33:21.908636] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T17:33:21.974908] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-04. [2025-09-16T17:33:22.774903] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-04. [2025-09-16T17:33:23.574906] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-04. [2025-09-16T17:33:23.618974] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T17:33:24.157555] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-09-16T17:33:24.374899] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-05. [2025-09-16T17:33:24.801082] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T17:33:25.174905] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-05. [2025-09-16T17:33:25.959515] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T17:33:25.974842] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-06. [2025-09-16T17:33:26.423185] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T17:33:26.774955] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-05. [2025-09-16T17:33:27.431173] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T17:33:27.574910] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-06. [2025-09-16T17:33:28.089359] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T17:33:28.374914] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-06. [2025-09-16T17:33:28.534081] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-09-16T17:33:29.174908] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-07. [2025-09-16T17:33:29.635503] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T17:33:29.974898] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-07. [2025-09-16T17:33:30.774934] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-06. [2025-09-16T17:33:30.814622] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-09-16T17:33:31.048568] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T17:33:31.574909] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-08. [2025-09-16T17:33:32.025314] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T17:33:32.374906] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-08. [2025-09-16T17:33:33.061408] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T17:33:33.110078] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-09-16T17:33:33.174914] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-07. [2025-09-16T17:33:33.974912] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-09. [2025-09-16T17:33:34.711189] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-09-16T17:33:34.774911] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-09. [2025-09-16T17:33:35.374770] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:33:35.379846] [INFO ] [REMOTE ] BOOT: Full fetch complete (45417 records). [2025-09-16T17:33:35.379981] [DEBUG] [VALIDATOR ] BOOT: Validating primary data. [2025-09-16T17:33:35.575284] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-08. [2025-09-16T17:33:35.649112] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-09-16T17:33:35.691620] [INFO ] [REMOTE ] BOE: Full fetch complete (18935 records). [2025-09-16T17:33:35.707207] [DEBUG] [VALIDATOR ] BOE: Validating primary data. [2025-09-16T17:33:35.855148] [DEBUG] [VALIDATOR ] BOOT: All expected trading days are present. [2025-09-16T17:33:35.873604] [DEBUG] [VALIDATOR ] BOOT: Primary data validation passed. [2025-09-16T17:33:35.876366] [INFO ] [MAIN ] BOOT: Fetched primary data is valid. [2025-09-16T17:33:35.876424] [DEBUG] [LOCAL ] BOOT: Saving primary data. [2025-09-16T17:33:35.934444] [DEBUG] [LOCAL ] BOOT: Trimmed 45417 records to 29529 (keeping 150 days of data) [2025-09-16T17:33:36.252020] [DEBUG] [VALIDATOR ] BOE: All expected trading days are present. [2025-09-16T17:33:36.252599] [DEBUG] [VALIDATOR ] BOE: Primary data validation passed. [2025-09-16T17:33:36.252992] [INFO ] [MAIN ] BOE: Fetched primary data is valid. [2025-09-16T17:33:36.253257] [DEBUG] [LOCAL ] BOE: Saving primary data. [2025-09-16T17:33:36.255592] [DEBUG] [LOCAL ] BOE: Trimmed 18935 records to 11979 (keeping 150 days of data) [2025-09-16T17:33:36.375668] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-07. [2025-09-16T17:33:36.377217] [DEBUG] [LOCAL ] BOE: Primary data saved (11979 records). [2025-09-16T17:33:36.382888] [INFO ] [MAIN ] BOE: Full history saved. [2025-09-16T17:33:36.396628] [INFO ] [MAIN ] Update Progress: 321/2369 symbols completed. BOE took 20.8s. Estimated remaining: 3h 34m 5s. Rate: 9.6 symbols/min [2025-09-16T17:33:36.588973] [INFO ] [MAIN ] BORR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:36.595673] [DEBUG] [LOCAL ] BOOT: Primary data saved (29529 records). [2025-09-16T17:33:36.595924] [INFO ] [MAIN ] BOOT: Full history saved. [2025-09-16T17:33:36.599790] [INFO ] [MAIN ] Update Progress: 322/2369 symbols completed. BOOT took 21.0s. Estimated remaining: 3h 33m 20s. Rate: 9.6 symbols/min [2025-09-16T17:33:36.780059] [INFO ] [MAIN ] BOW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:36.795919] [DEBUG] [LOCAL ] BORR: Loaded 37699 existing records from primary storage. [2025-09-16T17:33:36.796070] [INFO ] [MAIN ] BORR: Loaded 37699 existing records. [2025-09-16T17:33:36.986519] [DEBUG] [LOCAL ] BOW: Loaded 16164 existing records from primary storage. [2025-09-16T17:33:36.986100] [WARN ] [MAIN ] BORR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:33:36.986937] [INFO ] [MAIN ] BOW: Loaded 16164 existing records. [2025-09-16T17:33:37.018186] [INFO ] [MAIN ] BORR: Wiping existing primary data before full refresh. [2025-09-16T17:33:37.023467] [INFO ] [LOCAL ] BORR: Starting deletion of primary data. [2025-09-16T17:33:37.068122] [DEBUG] [LOCAL ] BORR: Deleted primary data file. [2025-09-16T17:33:37.068390] [INFO ] [MAIN ] BORR: Starting full primary refresh. [2025-09-16T17:33:37.275704] [WARN ] [MAIN ] BOW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:33:37.302230] [INFO ] [MAIN ] BOW: Wiping existing primary data before full refresh. [2025-09-16T17:33:37.331935] [INFO ] [LOCAL ] BOW: Starting deletion of primary data. [2025-09-16T17:33:37.334884] [DEBUG] [LOCAL ] BOW: Deleted primary data file. [2025-09-16T17:33:37.335737] [INFO ] [MAIN ] BOW: Starting full primary refresh. [2025-09-16T17:33:37.519735] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-09-16T17:33:37.649214] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-09. [2025-09-16T17:33:37.752633] [INFO ] [REMOTE ] BORR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:33:37.966826] [INFO ] [REMOTE ] BOW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:33:37.980172] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-02. [2025-09-16T17:33:37.990009] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-09-16T17:33:38.774886] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-02. [2025-09-16T17:33:38.923495] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-09-16T17:33:39.100845] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-09-16T17:33:39.105021] [INFO ] [REMOTE ] BOND: Full fetch complete (34774 records). [2025-09-16T17:33:39.105130] [DEBUG] [VALIDATOR ] BOND: Validating primary data. [2025-09-16T17:33:39.481964] [DEBUG] [VALIDATOR ] BOND: All expected trading days are present. [2025-09-16T17:33:39.482046] [DEBUG] [VALIDATOR ] BOND: Primary data validation passed. [2025-09-16T17:33:39.482488] [INFO ] [MAIN ] BOND: Fetched primary data is valid. [2025-09-16T17:33:39.482528] [DEBUG] [LOCAL ] BOND: Saving primary data. [2025-09-16T17:33:39.483302] [DEBUG] [LOCAL ] BOND: Trimmed 34774 records to 23774 (keeping 150 days of data) [2025-09-16T17:33:39.580250] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-08. [2025-09-16T17:33:39.639745] [DEBUG] [LOCAL ] BOND: Primary data saved (23774 records). [2025-09-16T17:33:39.639874] [INFO ] [MAIN ] BOND: Full history saved. [2025-09-16T17:33:39.639961] [INFO ] [MAIN ] Update Progress: 323/2369 symbols completed. BOND took 24.0s. Estimated remaining: 3h 32m 54s. Rate: 9.6 symbols/min [2025-09-16T17:33:39.815885] [INFO ] [MAIN ] BOX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:39.851182] [DEBUG] [LOCAL ] BOX: Loaded 37029 existing records from primary storage. [2025-09-16T17:33:39.851319] [INFO ] [MAIN ] BOX: Loaded 37029 existing records. [2025-09-16T17:33:40.036978] [WARN ] [MAIN ] BOX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:33:40.037243] [INFO ] [MAIN ] BOX: Wiping existing primary data before full refresh. [2025-09-16T17:33:40.037303] [INFO ] [LOCAL ] BOX: Starting deletion of primary data. [2025-09-16T17:33:40.038325] [DEBUG] [LOCAL ] BOX: Deleted primary data file. [2025-09-16T17:33:40.038386] [INFO ] [MAIN ] BOX: Starting full primary refresh. [2025-09-16T17:33:40.383219] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-03. [2025-09-16T17:33:40.405491] [INFO ] [REMOTE ] BOX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:33:40.568341] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-09-16T17:33:40.689236] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-09-16T17:33:41.174875] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-03. [2025-09-16T17:33:41.479596] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T17:33:41.974907] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-04. [2025-09-16T17:33:42.755181] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-09-16T17:33:42.774892] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-05. [2025-09-16T17:33:43.052559] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-09-16T17:33:43.574909] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-02. [2025-09-16T17:33:43.849034] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-09-16T17:33:44.374934] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-06. [2025-09-16T17:33:44.677550] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T17:33:45.174946] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-04. [2025-09-16T17:33:45.340791] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-09-16T17:33:45.974912] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-07. [2025-09-16T17:33:46.646320] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T17:33:46.774903] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-05. [2025-09-16T17:33:46.937296] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-09-16T17:33:47.574899] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-03. [2025-09-16T17:33:48.003784] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-09-16T17:33:48.374913] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-09. [2025-09-16T17:33:49.135860] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T17:33:49.174854] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-06. [2025-09-16T17:33:49.804544] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-09-16T17:33:49.808277] [INFO ] [REMOTE ] BOH: Full fetch complete (33814 records). [2025-09-16T17:33:49.808355] [DEBUG] [VALIDATOR ] BOH: Validating primary data. [2025-09-16T17:33:49.980068] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-08. [2025-09-16T17:33:50.179350] [DEBUG] [VALIDATOR ] BOH: All expected trading days are present. [2025-09-16T17:33:50.179426] [DEBUG] [VALIDATOR ] BOH: Primary data validation passed. [2025-09-16T17:33:50.179878] [INFO ] [MAIN ] BOH: Fetched primary data is valid. [2025-09-16T17:33:50.179925] [DEBUG] [LOCAL ] BOH: Saving primary data. [2025-09-16T17:33:50.180705] [DEBUG] [LOCAL ] BOH: Trimmed 33814 records to 22048 (keeping 150 days of data) [2025-09-16T17:33:50.320238] [DEBUG] [LOCAL ] BOH: Primary data saved (22048 records). [2025-09-16T17:33:50.320369] [INFO ] [MAIN ] BOH: Full history saved. [2025-09-16T17:33:50.320451] [INFO ] [MAIN ] Update Progress: 324/2369 symbols completed. BOH took 34.7s. Estimated remaining: 3h 33m 15s. Rate: 9.6 symbols/min [2025-09-16T17:33:50.494381] [INFO ] [MAIN ] BP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:50.532989] [DEBUG] [LOCAL ] BP: Loaded 39671 existing records from primary storage. [2025-09-16T17:33:50.533148] [INFO ] [MAIN ] BP: Loaded 39671 existing records. [2025-09-16T17:33:50.717793] [WARN ] [MAIN ] BP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:33:50.718264] [INFO ] [MAIN ] BP: Wiping existing primary data before full refresh. [2025-09-16T17:33:50.718422] [INFO ] [LOCAL ] BP: Starting deletion of primary data. [2025-09-16T17:33:50.719597] [DEBUG] [LOCAL ] BP: Deleted primary data file. [2025-09-16T17:33:50.729063] [INFO ] [MAIN ] BP: Starting full primary refresh. [2025-09-16T17:33:50.777991] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-04. [2025-09-16T17:33:50.827650] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T17:33:51.120604] [INFO ] [REMOTE ] BP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:33:51.137771] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T17:33:51.574912] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-02. [2025-09-16T17:33:52.374891] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-07. [2025-09-16T17:33:52.791946] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-09-16T17:33:53.174926] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-05. [2025-09-16T17:33:53.847179] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-02 (2.27 seconds). [2025-09-16T17:33:53.933886] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-09-16T17:33:53.974845] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-08. [2025-09-16T17:33:54.544030] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-09-16T17:33:54.774904] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-06. [2025-09-16T17:33:55.574922] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-03. [2025-09-16T17:33:55.743607] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T17:33:56.292825] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T17:33:56.374913] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-09. [2025-09-16T17:33:57.174901] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-09. [2025-09-16T17:33:57.665408] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-09-16T17:33:57.974866] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-07. [2025-09-16T17:33:58.124284] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-09-16T17:33:58.130813] [INFO ] [REMOTE ] BORR: Full fetch complete (55813 records). [2025-09-16T17:33:58.130940] [DEBUG] [VALIDATOR ] BORR: Validating primary data. [2025-09-16T17:33:58.451900] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-09-16T17:33:58.496364] [INFO ] [REMOTE ] BOW: Full fetch complete (25064 records). [2025-09-16T17:33:58.511513] [DEBUG] [VALIDATOR ] BOW: Validating primary data. [2025-09-16T17:33:58.573737] [DEBUG] [VALIDATOR ] BORR: All expected trading days are present. [2025-09-16T17:33:58.594464] [DEBUG] [VALIDATOR ] BORR: Primary data validation passed. [2025-09-16T17:33:58.606576] [INFO ] [MAIN ] BORR: Fetched primary data is valid. [2025-09-16T17:33:58.611773] [DEBUG] [LOCAL ] BORR: Saving primary data. [2025-09-16T17:33:58.674723] [DEBUG] [LOCAL ] BORR: Trimmed 55813 records to 38074 (keeping 150 days of data) [2025-09-16T17:33:58.795353] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-04. [2025-09-16T17:33:59.008737] [DEBUG] [VALIDATOR ] BOW: All expected trading days are present. [2025-09-16T17:33:59.009347] [DEBUG] [VALIDATOR ] BOW: Primary data validation passed. [2025-09-16T17:33:59.015336] [INFO ] [MAIN ] BOW: Fetched primary data is valid. [2025-09-16T17:33:59.015955] [DEBUG] [LOCAL ] BOW: Saving primary data. [2025-09-16T17:33:59.028258] [DEBUG] [LOCAL ] BOW: Trimmed 25064 records to 16305 (keeping 150 days of data) [2025-09-16T17:33:59.221435] [DEBUG] [LOCAL ] BOW: Primary data saved (16305 records). [2025-09-16T17:33:59.224555] [INFO ] [MAIN ] BOW: Full history saved. [2025-09-16T17:33:59.225573] [INFO ] [MAIN ] Update Progress: 325/2369 symbols completed. BOW took 22.6s. Estimated remaining: 3h 33m 26s. Rate: 9.6 symbols/min [2025-09-16T17:33:59.273939] [DEBUG] [LOCAL ] BORR: Primary data saved (38074 records). [2025-09-16T17:33:59.312514] [INFO ] [MAIN ] BORR: Full history saved. [2025-09-16T17:33:59.334345] [INFO ] [MAIN ] Update Progress: 326/2369 symbols completed. BORR took 22.9s. Estimated remaining: 3h 32m 41s. Rate: 9.6 symbols/min [2025-09-16T17:33:59.513342] [INFO ] [MAIN ] BR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:59.586073] [INFO ] [MAIN ] BRBR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:33:59.636821] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-09-16T17:33:59.648474] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-08. [2025-09-16T17:33:59.659117] [DEBUG] [LOCAL ] BR: Loaded 25002 existing records from primary storage. [2025-09-16T17:33:59.659272] [INFO ] [MAIN ] BR: Loaded 25002 existing records. [2025-09-16T17:33:59.841457] [DEBUG] [LOCAL ] BRBR: Loaded 38487 existing records from primary storage. [2025-09-16T17:33:59.871729] [INFO ] [MAIN ] BRBR: Loaded 38487 existing records. [2025-09-16T17:33:59.936722] [WARN ] [MAIN ] BR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:34:00.043822] [INFO ] [MAIN ] BR: Wiping existing primary data before full refresh. [2025-09-16T17:34:00.058780] [INFO ] [LOCAL ] BR: Starting deletion of primary data. [2025-09-16T17:34:00.066684] [DEBUG] [LOCAL ] BR: Deleted primary data file. [2025-09-16T17:34:00.100479] [INFO ] [MAIN ] BR: Starting full primary refresh. [2025-09-16T17:34:00.289488] [WARN ] [MAIN ] BRBR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:34:00.316298] [INFO ] [MAIN ] BRBR: Wiping existing primary data before full refresh. [2025-09-16T17:34:00.340609] [INFO ] [LOCAL ] BRBR: Starting deletion of primary data. [2025-09-16T17:34:00.407456] [DEBUG] [LOCAL ] BRBR: Deleted primary data file. [2025-09-16T17:34:00.407746] [INFO ] [MAIN ] BRBR: Starting full primary refresh. [2025-09-16T17:34:00.992752] [INFO ] [REMOTE ] BR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:01.045133] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-02. [2025-09-16T17:34:01.140515] [INFO ] [REMOTE ] BRBR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:01.183310] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-02. [2025-09-16T17:34:01.202210] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T17:34:01.404543] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-04 (2.61 seconds). [2025-09-16T17:34:01.974903] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-05. [2025-09-16T17:34:02.774899] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-09. [2025-09-16T17:34:02.792519] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T17:34:03.255796] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-09-16T17:34:03.574892] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-03. [2025-09-16T17:34:04.270977] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-05 (2.30 seconds). [2025-09-16T17:34:04.368909] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T17:34:04.374880] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-03. [2025-09-16T17:34:04.375147] [INFO ] [REMOTE ] BOX: Full fetch complete (54557 records). [2025-09-16T17:34:04.375257] [DEBUG] [VALIDATOR ] BOX: Validating primary data. [2025-09-16T17:34:04.764198] [DEBUG] [VALIDATOR ] BOX: All expected trading days are present. [2025-09-16T17:34:04.764278] [DEBUG] [VALIDATOR ] BOX: Primary data validation passed. [2025-09-16T17:34:04.764884] [INFO ] [MAIN ] BOX: Fetched primary data is valid. [2025-09-16T17:34:04.764929] [DEBUG] [LOCAL ] BOX: Saving primary data. [2025-09-16T17:34:04.766013] [DEBUG] [LOCAL ] BOX: Trimmed 54557 records to 37385 (keeping 150 days of data) [2025-09-16T17:34:04.982347] [DEBUG] [LOCAL ] BOX: Primary data saved (37385 records). [2025-09-16T17:34:04.982489] [INFO ] [MAIN ] BOX: Full history saved. [2025-09-16T17:34:04.983445] [INFO ] [MAIN ] Update Progress: 327/2369 symbols completed. BOX took 25.3s. Estimated remaining: 3h 32m 31s. Rate: 9.6 symbols/min [2025-09-16T17:34:05.154701] [INFO ] [MAIN ] BRC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:34:05.170274] [DEBUG] [LOCAL ] BRC: Loaded 13268 existing records from primary storage. [2025-09-16T17:34:05.170462] [INFO ] [MAIN ] BRC: Loaded 13268 existing records. [2025-09-16T17:34:05.181722] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-06. [2025-09-16T17:34:05.383494] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T17:34:05.387712] [WARN ] [MAIN ] BRC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:34:05.398762] [INFO ] [MAIN ] BRC: Wiping existing primary data before full refresh. [2025-09-16T17:34:05.399159] [INFO ] [LOCAL ] BRC: Starting deletion of primary data. [2025-09-16T17:34:05.399726] [DEBUG] [LOCAL ] BRC: Deleted primary data file. [2025-09-16T17:34:05.399778] [INFO ] [MAIN ] BRC: Starting full primary refresh. [2025-09-16T17:34:05.749675] [INFO ] [REMOTE ] BRC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:05.903928] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T17:34:05.974861] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-04. [2025-09-16T17:34:06.774931] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-04. [2025-09-16T17:34:07.107984] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-09-16T17:34:07.445755] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T17:34:07.574906] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-02. [2025-09-16T17:34:08.374935] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-05. [2025-09-16T17:34:08.458915] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-09-16T17:34:08.878294] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-09-16T17:34:09.174920] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-03. [2025-09-16T17:34:09.798117] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-09-16T17:34:09.974924] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-06. [2025-09-16T17:34:10.048622] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-09-16T17:34:10.774915] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-04. [2025-09-16T17:34:11.261364] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T17:34:11.574947] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-05. [2025-09-16T17:34:11.898950] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T17:34:12.374903] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-05. [2025-09-16T17:34:13.174919] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-07. [2025-09-16T17:34:13.384427] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T17:34:13.715237] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T17:34:13.974919] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-07. [2025-09-16T17:34:14.614819] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T17:34:14.774953] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-06. [2025-09-16T17:34:15.574906] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-06. [2025-09-16T17:34:15.828559] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T17:34:16.174665] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-09-16T17:34:16.374878] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-07. [2025-09-16T17:34:16.441616] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-09-16T17:34:17.174903] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-07. [2025-09-16T17:34:17.810016] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T17:34:17.833404] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-09-16T17:34:17.974909] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-08. [2025-09-16T17:34:18.774923] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-08. [2025-09-16T17:34:19.127407] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T17:34:19.575000] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-08. [2025-09-16T17:34:20.149589] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-09-16T17:34:20.374910] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-08. [2025-09-16T17:34:21.174886] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-09. [2025-09-16T17:34:21.288722] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-09-16T17:34:21.383231] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T17:34:21.974903] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-09. [2025-09-16T17:34:22.588167] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T17:34:22.594554] [INFO ] [REMOTE ] BRBR: Full fetch complete (55112 records). [2025-09-16T17:34:22.594709] [DEBUG] [VALIDATOR ] BRBR: Validating primary data. [2025-09-16T17:34:22.780030] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-09. [2025-09-16T17:34:22.994128] [DEBUG] [VALIDATOR ] BRBR: All expected trading days are present. [2025-09-16T17:34:22.994227] [DEBUG] [VALIDATOR ] BRBR: Primary data validation passed. [2025-09-16T17:34:22.994897] [INFO ] [MAIN ] BRBR: Fetched primary data is valid. [2025-09-16T17:34:22.994954] [DEBUG] [LOCAL ] BRBR: Saving primary data. [2025-09-16T17:34:22.996125] [DEBUG] [LOCAL ] BRBR: Trimmed 55112 records to 38872 (keeping 150 days of data) [2025-09-16T17:34:23.218160] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T17:34:23.224585] [INFO ] [REMOTE ] BR: Full fetch complete (37263 records). [2025-09-16T17:34:23.225015] [DEBUG] [VALIDATOR ] BR: Validating primary data. [2025-09-16T17:34:23.600932] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-09. [2025-09-16T17:34:23.633095] [DEBUG] [LOCAL ] BRBR: Primary data saved (38872 records). [2025-09-16T17:34:23.635369] [DEBUG] [VALIDATOR ] BR: All expected trading days are present. [2025-09-16T17:34:23.636328] [DEBUG] [VALIDATOR ] BR: Primary data validation passed. [2025-09-16T17:34:23.635236] [INFO ] [MAIN ] BRBR: Full history saved. [2025-09-16T17:34:23.636931] [INFO ] [MAIN ] BR: Fetched primary data is valid. [2025-09-16T17:34:23.639273] [INFO ] [MAIN ] Update Progress: 328/2369 symbols completed. BRBR took 24.3s. Estimated remaining: 3h 33m 42s. Rate: 9.6 symbols/min [2025-09-16T17:34:23.639599] [DEBUG] [LOCAL ] BR: Saving primary data. [2025-09-16T17:34:23.692666] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-09-16T17:34:23.744378] [INFO ] [REMOTE ] BRC: Full fetch complete (21890 records). [2025-09-16T17:34:23.755323] [DEBUG] [VALIDATOR ] BRC: Validating primary data. [2025-09-16T17:34:23.755227] [DEBUG] [LOCAL ] BR: Trimmed 37263 records to 25280 (keeping 150 days of data) [2025-09-16T17:34:23.882533] [INFO ] [MAIN ] BRCC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:34:24.257081] [DEBUG] [LOCAL ] BRCC: Loaded 30061 existing records from primary storage. [2025-09-16T17:34:24.285749] [INFO ] [MAIN ] BRCC: Loaded 30061 existing records. [2025-09-16T17:34:24.384170] [DEBUG] [VALIDATOR ] BRC: All expected trading days are present. [2025-09-16T17:34:24.396168] [DEBUG] [VALIDATOR ] BRC: Primary data validation passed. [2025-09-16T17:34:24.425045] [INFO ] [MAIN ] BRC: Fetched primary data is valid. [2025-09-16T17:34:24.426152] [DEBUG] [LOCAL ] BRC: Saving primary data. [2025-09-16T17:34:24.484550] [DEBUG] [LOCAL ] BRC: Trimmed 21890 records to 13406 (keeping 150 days of data) [2025-09-16T17:34:24.504335] [WARN ] [MAIN ] BRCC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:34:24.507947] [INFO ] [MAIN ] BRCC: Wiping existing primary data before full refresh. [2025-09-16T17:34:24.545436] [INFO ] [LOCAL ] BRCC: Starting deletion of primary data. [2025-09-16T17:34:24.570214] [DEBUG] [LOCAL ] BRCC: Deleted primary data file. [2025-09-16T17:34:24.570696] [INFO ] [MAIN ] BRCC: Starting full primary refresh. [2025-09-16T17:34:24.703595] [DEBUG] [LOCAL ] BR: Primary data saved (25280 records). [2025-09-16T17:34:24.711195] [INFO ] [MAIN ] BR: Full history saved. [2025-09-16T17:34:24.724231] [INFO ] [MAIN ] Update Progress: 329/2369 symbols completed. BR took 25.5s. Estimated remaining: 3h 33m 3s. Rate: 9.6 symbols/min [2025-09-16T17:34:25.099354] [INFO ] [MAIN ] BRFS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:34:25.115536] [DEBUG] [LOCAL ] BRFS: No existing data file found in primary storage. [2025-09-16T17:34:25.116141] [WARN ] [MAIN ] BRFS: No existing data found, performing full refresh instead. [2025-09-16T17:34:25.142474] [INFO ] [MAIN ] BRFS: Wiping existing primary data before full refresh. [2025-09-16T17:34:25.168837] [INFO ] [REMOTE ] BRCC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:25.168933] [INFO ] [LOCAL ] BRFS: Starting deletion of primary data. [2025-09-16T17:34:25.165049] [DEBUG] [LOCAL ] BRC: Primary data saved (13406 records). [2025-09-16T17:34:25.169312] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-02. [2025-09-16T17:34:25.169085] [INFO ] [MAIN ] BRC: Full history saved. [2025-09-16T17:34:25.169445] [INFO ] [MAIN ] Update Progress: 330/2369 symbols completed. BRC took 20.2s. Estimated remaining: 3h 32m 21s. Rate: 9.6 symbols/min [2025-09-16T17:34:25.169049] [DEBUG] [LOCAL ] BRFS: No primary data file found to delete. [2025-09-16T17:34:25.170298] [INFO ] [MAIN ] BRFS: Starting full primary refresh. [2025-09-16T17:34:25.481220] [INFO ] [MAIN ] BRO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:34:25.514162] [DEBUG] [LOCAL ] BRO: No existing data file found in primary storage. [2025-09-16T17:34:25.553215] [WARN ] [MAIN ] BRO: No existing data found, performing full refresh instead. [2025-09-16T17:34:25.620689] [INFO ] [MAIN ] BRO: Wiping existing primary data before full refresh. [2025-09-16T17:34:25.620916] [INFO ] [LOCAL ] BRO: Starting deletion of primary data. [2025-09-16T17:34:25.621239] [DEBUG] [LOCAL ] BRO: No primary data file found to delete. [2025-09-16T17:34:25.631674] [INFO ] [MAIN ] BRO: Starting full primary refresh. [2025-09-16T17:34:25.712946] [INFO ] [REMOTE ] BRFS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:25.751195] [INFO ] [REMOTE ] BRFS: Fetching primary data for 2025-02. [2025-09-16T17:34:25.831592] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-09 (2.23 seconds). [2025-09-16T17:34:25.891826] [INFO ] [REMOTE ] BP: Full fetch complete (58390 records). [2025-09-16T17:34:25.899727] [DEBUG] [VALIDATOR ] BP: Validating primary data. [2025-09-16T17:34:26.280275] [INFO ] [REMOTE ] BRO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:26.297224] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-02. [2025-09-16T17:34:26.515446] [DEBUG] [VALIDATOR ] BP: All expected trading days are present. [2025-09-16T17:34:26.515678] [DEBUG] [VALIDATOR ] BP: Primary data validation passed. [2025-09-16T17:34:26.516452] [INFO ] [MAIN ] BP: Fetched primary data is valid. [2025-09-16T17:34:26.533701] [DEBUG] [LOCAL ] BP: Saving primary data. [2025-09-16T17:34:26.535370] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T17:34:26.542835] [DEBUG] [LOCAL ] BP: Trimmed 58390 records to 40061 (keeping 150 days of data) [2025-09-16T17:34:26.774904] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-03. [2025-09-16T17:34:26.790138] [DEBUG] [LOCAL ] BP: Primary data saved (40061 records). [2025-09-16T17:34:26.790263] [INFO ] [MAIN ] BP: Full history saved. [2025-09-16T17:34:26.791735] [INFO ] [MAIN ] Update Progress: 331/2369 symbols completed. BP took 36.5s. Estimated remaining: 3h 31m 46s. Rate: 9.6 symbols/min [2025-09-16T17:34:26.963013] [INFO ] [MAIN ] BROS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:34:26.963173] [DEBUG] [LOCAL ] BROS: No existing data file found in primary storage. [2025-09-16T17:34:26.963210] [WARN ] [MAIN ] BROS: No existing data found, performing full refresh instead. [2025-09-16T17:34:26.963328] [INFO ] [MAIN ] BROS: Wiping existing primary data before full refresh. [2025-09-16T17:34:26.963386] [INFO ] [LOCAL ] BROS: Starting deletion of primary data. [2025-09-16T17:34:26.963429] [DEBUG] [LOCAL ] BROS: No primary data file found to delete. [2025-09-16T17:34:26.963462] [INFO ] [MAIN ] BROS: Starting full primary refresh. [2025-09-16T17:34:27.324956] [INFO ] [REMOTE ] BROS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:27.348217] [INFO ] [REMOTE ] BRFS: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T17:34:27.574933] [INFO ] [REMOTE ] BRFS: Fetching primary data for 2025-03. [2025-09-16T17:34:27.702786] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-09-16T17:34:27.809927] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-09-16T17:34:28.374900] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-03. [2025-09-16T17:34:28.632675] [INFO ] [REMOTE ] BRFS: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-09-16T17:34:29.174971] [INFO ] [REMOTE ] BRFS: Fetching primary data for 2025-04. [2025-09-16T17:34:29.744107] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T17:34:29.974910] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-04. [2025-09-16T17:34:30.365961] [INFO ] [REMOTE ] BRFS: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:34:30.774921] [INFO ] [REMOTE ] BRFS: Fetching primary data for 2025-05. [2025-09-16T17:34:31.409590] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:34:31.574905] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-05. [2025-09-16T17:34:31.912272] [INFO ] [REMOTE ] BRFS: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T17:34:32.374963] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-02. [2025-09-16T17:34:33.022888] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T17:34:33.174927] [INFO ] [REMOTE ] BRFS: Fetching primary data for 2025-06. [2025-09-16T17:34:33.974895] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-06. [2025-09-16T17:34:34.041015] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T17:34:34.388339] [INFO ] [REMOTE ] BRFS: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T17:34:34.774964] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-03. [2025-09-16T17:34:35.272685] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:34:35.574919] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-04. [2025-09-16T17:34:36.374900] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-07. [2025-09-16T17:34:36.502716] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T17:34:36.689986] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T17:34:37.174895] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-04. [2025-09-16T17:34:37.956282] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-09-16T17:34:37.974855] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-08. [2025-09-16T17:34:38.774928] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-05. [2025-09-16T17:34:39.166857] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-09-16T17:34:39.462392] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T17:34:39.574884] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-09. [2025-09-16T17:34:40.241438] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T17:34:40.374926] [INFO ] [REMOTE ] BRFS: Fetching primary data for 2025-07. [2025-09-16T17:34:41.174916] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-05. [2025-09-16T17:34:41.201404] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-09-16T17:34:41.208135] [INFO ] [REMOTE ] BRO: Full fetch complete (56239 records). [2025-09-16T17:34:41.208271] [DEBUG] [VALIDATOR ] BRO: Validating primary data. [2025-09-16T17:34:41.598913] [DEBUG] [VALIDATOR ] BRO: All expected trading days are present. [2025-09-16T17:34:41.599071] [DEBUG] [VALIDATOR ] BRO: Primary data validation passed. [2025-09-16T17:34:41.599766] [INFO ] [MAIN ] BRO: Fetched primary data is valid. [2025-09-16T17:34:41.599911] [DEBUG] [LOCAL ] BRO: Saving primary data. [2025-09-16T17:34:41.601271] [DEBUG] [LOCAL ] BRO: Trimmed 56239 records to 38727 (keeping 150 days of data) [2025-09-16T17:34:41.716003] [INFO ] [REMOTE ] BRFS: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-09-16T17:34:41.874610] [DEBUG] [LOCAL ] BRO: Primary data saved (38727 records). [2025-09-16T17:34:41.874784] [INFO ] [MAIN ] BRO: Full history saved. [2025-09-16T17:34:41.874875] [INFO ] [MAIN ] Update Progress: 332/2369 symbols completed. BRO took 16.7s. Estimated remaining: 3h 32m 34s. Rate: 9.6 symbols/min [2025-09-16T17:34:41.975603] [INFO ] [REMOTE ] BRFS: Fetching primary data for 2025-08. [2025-09-16T17:34:42.045037] [INFO ] [MAIN ] BRSL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:34:42.045190] [DEBUG] [LOCAL ] BRSL: No existing data file found in primary storage. [2025-09-16T17:34:42.045227] [WARN ] [MAIN ] BRSL: No existing data found, performing full refresh instead. [2025-09-16T17:34:42.045343] [INFO ] [MAIN ] BRSL: Wiping existing primary data before full refresh. [2025-09-16T17:34:42.045380] [INFO ] [LOCAL ] BRSL: Starting deletion of primary data. [2025-09-16T17:34:42.045421] [DEBUG] [LOCAL ] BRSL: No primary data file found to delete. [2025-09-16T17:34:42.045453] [INFO ] [MAIN ] BRSL: Starting full primary refresh. [2025-09-16T17:34:42.399433] [INFO ] [REMOTE ] BRSL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:42.774868] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-02. [2025-09-16T17:34:42.965714] [DEBUG] [REMOTE ] BRSL: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:34:42.967864] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T17:34:42.968326] [WARN ] [REMOTE ] BRSL: No data returned for month 2025-02 [2025-09-16T17:34:43.094445] [INFO ] [REMOTE ] BRFS: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T17:34:43.574911] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-06. [2025-09-16T17:34:44.374903] [INFO ] [REMOTE ] BRFS: Fetching primary data for 2025-09. [2025-09-16T17:34:45.174919] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-06. [2025-09-16T17:34:45.569551] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-09-16T17:34:45.592016] [INFO ] [REMOTE ] BRFS: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-09-16T17:34:45.596416] [INFO ] [REMOTE ] BRFS: Full fetch complete (44962 records). [2025-09-16T17:34:45.596538] [DEBUG] [VALIDATOR ] BRFS: Validating primary data. [2025-09-16T17:34:45.982154] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-07. [2025-09-16T17:34:45.994115] [DEBUG] [VALIDATOR ] BRFS: All expected trading days are present. [2025-09-16T17:34:45.994173] [DEBUG] [VALIDATOR ] BRFS: Primary data validation passed. [2025-09-16T17:34:45.994740] [INFO ] [MAIN ] BRFS: Fetched primary data is valid. [2025-09-16T17:34:45.994786] [DEBUG] [LOCAL ] BRFS: Saving primary data. [2025-09-16T17:34:45.995802] [DEBUG] [LOCAL ] BRFS: Trimmed 44962 records to 30222 (keeping 150 days of data) [2025-09-16T17:34:46.168910] [DEBUG] [LOCAL ] BRFS: Primary data saved (30222 records). [2025-09-16T17:34:46.169072] [INFO ] [MAIN ] BRFS: Full history saved. [2025-09-16T17:34:46.169153] [INFO ] [MAIN ] Update Progress: 333/2369 symbols completed. BRFS took 21.4s. Estimated remaining: 3h 32m 16s. Rate: 9.6 symbols/min [2025-09-16T17:34:46.351787] [INFO ] [MAIN ] BRSP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:34:46.351947] [DEBUG] [LOCAL ] BRSP: No existing data file found in primary storage. [2025-09-16T17:34:46.351986] [WARN ] [MAIN ] BRSP: No existing data found, performing full refresh instead. [2025-09-16T17:34:46.352120] [INFO ] [MAIN ] BRSP: Wiping existing primary data before full refresh. [2025-09-16T17:34:46.352159] [INFO ] [LOCAL ] BRSP: Starting deletion of primary data. [2025-09-16T17:34:46.352201] [DEBUG] [LOCAL ] BRSP: No primary data file found to delete. [2025-09-16T17:34:46.352234] [INFO ] [MAIN ] BRSP: Starting full primary refresh. [2025-09-16T17:34:46.719687] [INFO ] [REMOTE ] BRSP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:46.742539] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T17:34:46.774859] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-07. [2025-09-16T17:34:47.574903] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-02. [2025-09-16T17:34:48.078484] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-09-16T17:34:48.375004] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-03. [2025-09-16T17:34:48.374935] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T17:34:48.639156] [DEBUG] [REMOTE ] BRSL: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:34:48.639486] [WARN ] [REMOTE ] BRSL: No data returned for month 2025-03 [2025-09-16T17:34:49.174963] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-04. [2025-09-16T17:34:49.295485] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-09-16T17:34:49.361966] [DEBUG] [REMOTE ] BRSL: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:34:49.362323] [WARN ] [REMOTE ] BRSL: No data returned for month 2025-04 [2025-09-16T17:34:49.974931] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-08. [2025-09-16T17:34:50.774945] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-05. [2025-09-16T17:34:51.014333] [DEBUG] [REMOTE ] BRSL: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:34:51.014632] [WARN ] [REMOTE ] BRSL: No data returned for month 2025-05 [2025-09-16T17:34:51.253622] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-09-16T17:34:51.574915] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-08. [2025-09-16T17:34:52.374961] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-03. [2025-09-16T17:34:53.175002] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-06. [2025-09-16T17:34:53.375989] [DEBUG] [REMOTE ] BRSL: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:34:53.376408] [WARN ] [REMOTE ] BRSL: No data returned for month 2025-06 [2025-09-16T17:34:53.713816] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-09-16T17:34:53.852928] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T17:34:53.974896] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-09. [2025-09-16T17:34:54.774922] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-04. [2025-09-16T17:34:55.574933] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-09. [2025-09-16T17:34:56.298326] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-09 (2.32 seconds). [2025-09-16T17:34:56.305060] [INFO ] [REMOTE ] BROS: Full fetch complete (57952 records). [2025-09-16T17:34:56.305178] [DEBUG] [VALIDATOR ] BROS: Validating primary data. [2025-09-16T17:34:56.376231] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-07. [2025-09-16T17:34:56.716432] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-09-16T17:34:56.746218] [DEBUG] [VALIDATOR ] BROS: All expected trading days are present. [2025-09-16T17:34:56.746623] [DEBUG] [VALIDATOR ] BROS: Primary data validation passed. [2025-09-16T17:34:56.747374] [INFO ] [MAIN ] BROS: Fetched primary data is valid. [2025-09-16T17:34:56.747422] [DEBUG] [LOCAL ] BROS: Saving primary data. [2025-09-16T17:34:56.748697] [DEBUG] [LOCAL ] BROS: Trimmed 57952 records to 39721 (keeping 150 days of data) [2025-09-16T17:34:56.861148] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T17:34:56.868302] [INFO ] [REMOTE ] BRCC: Full fetch complete (43073 records). [2025-09-16T17:34:56.868928] [DEBUG] [VALIDATOR ] BRCC: Validating primary data. [2025-09-16T17:34:56.870156] [FATAL] [VALIDATOR ] BRCC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:34:56.871062] [FATAL] [MAIN ] BRCC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:34:56.871621] [WARN ] [MAIN ] BRCC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:34:56.872465] [INFO ] [LOCAL ] BRCC: Starting deletion of primary data. [2025-09-16T17:34:56.873054] [DEBUG] [LOCAL ] BRCC: No primary data file found to delete. [2025-09-16T17:34:56.873586] [INFO ] [LOCAL ] BRCC: Starting deletion of derivative data. [2025-09-16T17:34:56.875001] [DEBUG] [LOCAL ] BRCC: Deleted derivative data file. [2025-09-16T17:34:56.875376] [INFO ] [MAIN ] Update Progress: 334/2369 symbols completed. BRCC took 33.2s. Estimated remaining: 3h 32m 37s. Rate: 9.6 symbols/min [2025-09-16T17:34:57.083031] [INFO ] [MAIN ] BRT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:34:57.083910] [DEBUG] [LOCAL ] BRT: No existing data file found in primary storage. [2025-09-16T17:34:57.084511] [WARN ] [MAIN ] BRT: No existing data found, performing full refresh instead. [2025-09-16T17:34:57.085894] [INFO ] [MAIN ] BRT: Wiping existing primary data before full refresh. [2025-09-16T17:34:57.086489] [INFO ] [LOCAL ] BRT: Starting deletion of primary data. [2025-09-16T17:34:57.087553] [DEBUG] [LOCAL ] BRT: No primary data file found to delete. [2025-09-16T17:34:57.088934] [INFO ] [MAIN ] BRT: Starting full primary refresh. [2025-09-16T17:34:57.228070] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-05. [2025-09-16T17:34:57.566755] [INFO ] [REMOTE ] BRT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:57.587840] [DEBUG] [LOCAL ] BROS: Primary data saved (39721 records). [2025-09-16T17:34:57.588010] [INFO ] [MAIN ] BROS: Full history saved. [2025-09-16T17:34:57.588094] [INFO ] [MAIN ] Update Progress: 335/2369 symbols completed. BROS took 30.8s. Estimated remaining: 3h 31m 57s. Rate: 9.6 symbols/min [2025-09-16T17:34:57.760795] [INFO ] [MAIN ] BRW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:34:57.775217] [DEBUG] [LOCAL ] BRW: Loaded 14142 existing records from primary storage. [2025-09-16T17:34:57.775334] [INFO ] [MAIN ] BRW: Loaded 14142 existing records. [2025-09-16T17:34:57.955162] [WARN ] [MAIN ] BRW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:34:57.955444] [INFO ] [MAIN ] BRW: Wiping existing primary data before full refresh. [2025-09-16T17:34:57.955501] [INFO ] [LOCAL ] BRW: Starting deletion of primary data. [2025-09-16T17:34:57.955927] [DEBUG] [LOCAL ] BRW: Deleted primary data file. [2025-09-16T17:34:57.955976] [INFO ] [MAIN ] BRW: Starting full primary refresh. [2025-09-16T17:34:57.986973] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-02. [2025-09-16T17:34:58.310222] [INFO ] [REMOTE ] BRW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:34:58.349555] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T17:34:58.386672] [INFO ] [REMOTE ] BRSL: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-09-16T17:34:58.719437] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-09-16T17:34:58.774881] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-02. [2025-09-16T17:34:59.574900] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-03. [2025-09-16T17:34:59.846589] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-09-16T17:35:00.287876] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-09-16T17:35:00.374879] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-08. [2025-09-16T17:35:01.174928] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-03. [2025-09-16T17:35:01.974900] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-04. [2025-09-16T17:35:02.389886] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-09-16T17:35:02.496375] [INFO ] [REMOTE ] BRSL: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-09-16T17:35:02.564084] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-09-16T17:35:02.774920] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-09. [2025-09-16T17:35:03.574898] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-04. [2025-09-16T17:35:04.299610] [INFO ] [REMOTE ] BRSL: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T17:35:04.305441] [INFO ] [REMOTE ] BRSL: Full fetch complete (19399 records). [2025-09-16T17:35:04.305542] [DEBUG] [VALIDATOR ] BRSL: Validating primary data. [2025-09-16T17:35:04.384111] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-05. [2025-09-16T17:35:04.497964] [WARN ] [VALIDATOR ] BRSL: Validation failed - insufficient historical data. Have 53 unique days, need 150 days [2025-09-16T17:35:04.498302] [FATAL] [MAIN ] BRSL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:35:04.498368] [WARN ] [MAIN ] BRSL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:35:04.498424] [INFO ] [LOCAL ] BRSL: Starting deletion of primary data. [2025-09-16T17:35:04.498485] [DEBUG] [LOCAL ] BRSL: No primary data file found to delete. [2025-09-16T17:35:04.498520] [INFO ] [LOCAL ] BRSL: Starting deletion of derivative data. [2025-09-16T17:35:04.498560] [DEBUG] [LOCAL ] BRSL: No derivative data file found to delete. [2025-09-16T17:35:04.498630] [INFO ] [MAIN ] Update Progress: 336/2369 symbols completed. BRSL took 22.6s. Estimated remaining: 3h 31m 55s. Rate: 9.6 symbols/min [2025-09-16T17:35:04.683287] [INFO ] [MAIN ] BRX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:35:04.703482] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-09-16T17:35:04.752444] [DEBUG] [LOCAL ] BRX: Loaded 38344 existing records from primary storage. [2025-09-16T17:35:04.752598] [INFO ] [MAIN ] BRX: Loaded 38344 existing records. [2025-09-16T17:35:04.928497] [WARN ] [MAIN ] BRX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:35:04.928795] [INFO ] [MAIN ] BRX: Wiping existing primary data before full refresh. [2025-09-16T17:35:04.928843] [INFO ] [LOCAL ] BRX: Starting deletion of primary data. [2025-09-16T17:35:04.929804] [DEBUG] [LOCAL ] BRX: Deleted primary data file. [2025-09-16T17:35:04.929855] [INFO ] [MAIN ] BRX: Starting full primary refresh. [2025-09-16T17:35:05.180431] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-09-16T17:35:05.180535] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-05. [2025-09-16T17:35:05.300478] [INFO ] [REMOTE ] BRX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:35:05.974909] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-06. [2025-09-16T17:35:06.129101] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-09-16T17:35:06.359903] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-09-16T17:35:06.774894] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-06. [2025-09-16T17:35:07.479759] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-09-16T17:35:07.574919] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-07. [2025-09-16T17:35:08.374910] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-02. [2025-09-16T17:35:08.737032] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-09-16T17:35:09.174948] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-08. [2025-09-16T17:35:09.960702] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T17:35:09.974869] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-03. [2025-09-16T17:35:10.487083] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T17:35:10.774928] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-09. [2025-09-16T17:35:11.486210] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-09-16T17:35:11.574927] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-07. [2025-09-16T17:35:11.720308] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T17:35:11.723385] [INFO ] [REMOTE ] BRW: Full fetch complete (20416 records). [2025-09-16T17:35:11.723477] [DEBUG] [VALIDATOR ] BRW: Validating primary data. [2025-09-16T17:35:12.088630] [DEBUG] [VALIDATOR ] BRW: All expected trading days are present. [2025-09-16T17:35:12.088837] [DEBUG] [VALIDATOR ] BRW: Primary data validation passed. [2025-09-16T17:35:12.089164] [INFO ] [MAIN ] BRW: Fetched primary data is valid. [2025-09-16T17:35:12.089223] [DEBUG] [LOCAL ] BRW: Saving primary data. [2025-09-16T17:35:12.089887] [DEBUG] [LOCAL ] BRW: Trimmed 20416 records to 14443 (keeping 150 days of data) [2025-09-16T17:35:12.178750] [DEBUG] [LOCAL ] BRW: Primary data saved (14443 records). [2025-09-16T17:35:12.178885] [INFO ] [MAIN ] BRW: Full history saved. [2025-09-16T17:35:12.179057] [INFO ] [MAIN ] Update Progress: 337/2369 symbols completed. BRW took 14.6s. Estimated remaining: 3h 31m 57s. Rate: 9.6 symbols/min [2025-09-16T17:35:12.347084] [INFO ] [MAIN ] BSAC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:35:12.352350] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-09-16T17:35:12.374193] [DEBUG] [LOCAL ] BSAC: Loaded 24120 existing records from primary storage. [2025-09-16T17:35:12.374313] [INFO ] [MAIN ] BSAC: Loaded 24120 existing records. [2025-09-16T17:35:12.385033] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-04. [2025-09-16T17:35:12.550410] [WARN ] [MAIN ] BSAC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:35:12.550664] [INFO ] [MAIN ] BSAC: Wiping existing primary data before full refresh. [2025-09-16T17:35:12.550735] [INFO ] [LOCAL ] BSAC: Starting deletion of primary data. [2025-09-16T17:35:12.551423] [DEBUG] [LOCAL ] BSAC: Deleted primary data file. [2025-09-16T17:35:12.551483] [INFO ] [MAIN ] BSAC: Starting full primary refresh. [2025-09-16T17:35:12.904285] [INFO ] [REMOTE ] BSAC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:35:13.174884] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-08. [2025-09-16T17:35:13.707030] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-09-16T17:35:13.974892] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-09. [2025-09-16T17:35:14.141281] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T17:35:14.476113] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-09-16T17:35:14.477230] [INFO ] [REMOTE ] BRT: Full fetch complete (4986 records). [2025-09-16T17:35:14.477298] [DEBUG] [VALIDATOR ] BRT: Validating primary data. [2025-09-16T17:35:14.780001] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-06. [2025-09-16T17:35:14.851045] [DEBUG] [VALIDATOR ] BRT: All expected trading days are present. [2025-09-16T17:35:14.851120] [DEBUG] [VALIDATOR ] BRT: Primary data validation passed. [2025-09-16T17:35:14.851279] [INFO ] [MAIN ] BRT: Fetched primary data is valid. [2025-09-16T17:35:14.851319] [DEBUG] [LOCAL ] BRT: Saving primary data. [2025-09-16T17:35:14.851714] [DEBUG] [LOCAL ] BRT: Trimmed 4986 records to 3507 (keeping 150 days of data) [2025-09-16T17:35:14.873198] [DEBUG] [LOCAL ] BRT: Primary data saved (3507 records). [2025-09-16T17:35:14.873277] [INFO ] [MAIN ] BRT: Full history saved. [2025-09-16T17:35:14.873352] [INFO ] [MAIN ] Update Progress: 338/2369 symbols completed. BRT took 18.0s. Estimated remaining: 3h 31m 29s. Rate: 9.6 symbols/min [2025-09-16T17:35:15.044874] [INFO ] [MAIN ] BSBR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:35:15.073978] [DEBUG] [LOCAL ] BSBR: Loaded 26500 existing records from primary storage. [2025-09-16T17:35:15.074135] [INFO ] [MAIN ] BSBR: Loaded 26500 existing records. [2025-09-16T17:35:15.249408] [WARN ] [MAIN ] BSBR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:35:15.249690] [INFO ] [MAIN ] BSBR: Wiping existing primary data before full refresh. [2025-09-16T17:35:15.249754] [INFO ] [LOCAL ] BSBR: Starting deletion of primary data. [2025-09-16T17:35:15.250419] [DEBUG] [LOCAL ] BSBR: Deleted primary data file. [2025-09-16T17:35:15.250478] [INFO ] [MAIN ] BSBR: Starting full primary refresh. [2025-09-16T17:35:15.580208] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-02. [2025-09-16T17:35:15.608630] [INFO ] [REMOTE ] BSBR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:35:16.277409] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T17:35:16.374846] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-07. [2025-09-16T17:35:16.598930] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T17:35:17.174952] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-02. [2025-09-16T17:35:17.853862] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T17:35:17.974897] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-08. [2025-09-16T17:35:18.634157] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-09-16T17:35:18.774903] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-03. [2025-09-16T17:35:18.978725] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-09-16T17:35:19.574946] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-05. [2025-09-16T17:35:19.671453] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-09-16T17:35:20.374905] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-09. [2025-09-16T17:35:21.174954] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-04. [2025-09-16T17:35:21.330901] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T17:35:21.780974] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T17:35:21.786926] [INFO ] [REMOTE ] BRSP: Full fetch complete (40528 records). [2025-09-16T17:35:21.787123] [DEBUG] [VALIDATOR ] BRSP: Validating primary data. [2025-09-16T17:35:21.981081] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-06. [2025-09-16T17:35:22.212968] [DEBUG] [VALIDATOR ] BRSP: All expected trading days are present. [2025-09-16T17:35:22.213184] [DEBUG] [VALIDATOR ] BRSP: Primary data validation passed. [2025-09-16T17:35:22.213869] [INFO ] [MAIN ] BRSP: Fetched primary data is valid. [2025-09-16T17:35:22.213928] [DEBUG] [LOCAL ] BRSP: Saving primary data. [2025-09-16T17:35:22.215594] [DEBUG] [LOCAL ] BRSP: Trimmed 40528 records to 26482 (keeping 150 days of data) [2025-09-16T17:35:22.355967] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T17:35:22.430027] [DEBUG] [LOCAL ] BRSP: Primary data saved (26482 records). [2025-09-16T17:35:22.430173] [INFO ] [MAIN ] BRSP: Full history saved. [2025-09-16T17:35:22.430262] [INFO ] [MAIN ] Update Progress: 339/2369 symbols completed. BRSP took 36.3s. Estimated remaining: 3h 31m 31s. Rate: 9.6 symbols/min [2025-09-16T17:35:22.614022] [INFO ] [MAIN ] BSL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:35:22.621082] [DEBUG] [LOCAL ] BSL: Loaded 5094 existing records from primary storage. [2025-09-16T17:35:22.621182] [INFO ] [MAIN ] BSL: Loaded 5094 existing records. [2025-09-16T17:35:22.778367] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-05. [2025-09-16T17:35:22.801536] [WARN ] [MAIN ] BSL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:35:22.801913] [INFO ] [MAIN ] BSL: Wiping existing primary data before full refresh. [2025-09-16T17:35:22.801972] [INFO ] [LOCAL ] BSL: Starting deletion of primary data. [2025-09-16T17:35:22.802470] [DEBUG] [LOCAL ] BSL: Deleted primary data file. [2025-09-16T17:35:22.802517] [INFO ] [MAIN ] BSL: Starting full primary refresh. [2025-09-16T17:35:23.152137] [INFO ] [REMOTE ] BSL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:35:23.574951] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-03. [2025-09-16T17:35:23.848399] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-09-16T17:35:24.374904] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-06. [2025-09-16T17:35:24.551541] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T17:35:24.981536] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-06 (3.00 seconds). [2025-09-16T17:35:25.174917] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-07. [2025-09-16T17:35:25.974992] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-02. [2025-09-16T17:35:26.525693] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T17:35:26.602400] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-09-16T17:35:26.774943] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-04. [2025-09-16T17:35:27.580528] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-08. [2025-09-16T17:35:27.583036] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-06 (3.21 seconds). [2025-09-16T17:35:28.286884] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T17:35:28.374928] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-05. [2025-09-16T17:35:28.922508] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T17:35:29.174922] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-09. [2025-09-16T17:35:29.340224] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T17:35:29.974944] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-07. [2025-09-16T17:35:30.616963] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T17:35:30.622636] [INFO ] [REMOTE ] BRX: Full fetch complete (56068 records). [2025-09-16T17:35:30.622769] [DEBUG] [VALIDATOR ] BRX: Validating primary data. [2025-09-16T17:35:30.624970] [FATAL] [VALIDATOR ] BRX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:35:30.625054] [FATAL] [MAIN ] BRX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:35:30.625094] [WARN ] [MAIN ] BRX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:35:30.625134] [INFO ] [LOCAL ] BRX: Starting deletion of primary data. [2025-09-16T17:35:30.625204] [DEBUG] [LOCAL ] BRX: No primary data file found to delete. [2025-09-16T17:35:30.625243] [INFO ] [LOCAL ] BRX: Starting deletion of derivative data. [2025-09-16T17:35:30.625502] [DEBUG] [LOCAL ] BRX: Deleted derivative data file. [2025-09-16T17:35:30.627927] [INFO ] [MAIN ] Update Progress: 340/2369 symbols completed. BRX took 26.1s. Estimated remaining: 3h 31m 36s. Rate: 9.6 symbols/min [2025-09-16T17:35:30.779963] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-06. [2025-09-16T17:35:30.803965] [INFO ] [MAIN ] BSM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:35:30.846209] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T17:35:30.861796] [DEBUG] [LOCAL ] BSM: Loaded 23991 existing records from primary storage. [2025-09-16T17:35:30.862257] [INFO ] [MAIN ] BSM: Loaded 23991 existing records. [2025-09-16T17:35:31.038148] [WARN ] [MAIN ] BSM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:35:31.038381] [INFO ] [MAIN ] BSM: Wiping existing primary data before full refresh. [2025-09-16T17:35:31.038427] [INFO ] [LOCAL ] BSM: Starting deletion of primary data. [2025-09-16T17:35:31.039137] [DEBUG] [LOCAL ] BSM: Deleted primary data file. [2025-09-16T17:35:31.039187] [INFO ] [MAIN ] BSM: Starting full primary refresh. [2025-09-16T17:35:31.387689] [INFO ] [REMOTE ] BSM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:35:31.574908] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-08. [2025-09-16T17:35:31.713777] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-09-16T17:35:32.374894] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-07. [2025-09-16T17:35:32.459484] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-09-16T17:35:33.174904] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-09. [2025-09-16T17:35:33.366153] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-09-16T17:35:33.974913] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-08. [2025-09-16T17:35:34.310377] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T17:35:34.315170] [INFO ] [REMOTE ] BSAC: Full fetch complete (35579 records). [2025-09-16T17:35:34.315309] [DEBUG] [VALIDATOR ] BSAC: Validating primary data. [2025-09-16T17:35:34.709129] [DEBUG] [VALIDATOR ] BSAC: All expected trading days are present. [2025-09-16T17:35:34.709265] [DEBUG] [VALIDATOR ] BSAC: Primary data validation passed. [2025-09-16T17:35:34.709804] [INFO ] [MAIN ] BSAC: Fetched primary data is valid. [2025-09-16T17:35:34.709853] [DEBUG] [LOCAL ] BSAC: Saving primary data. [2025-09-16T17:35:34.710941] [DEBUG] [LOCAL ] BSAC: Trimmed 35579 records to 24383 (keeping 150 days of data) [2025-09-16T17:35:34.799431] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-03. [2025-09-16T17:35:34.858861] [DEBUG] [LOCAL ] BSAC: Primary data saved (24383 records). [2025-09-16T17:35:34.858979] [INFO ] [MAIN ] BSAC: Full history saved. [2025-09-16T17:35:34.859061] [INFO ] [MAIN ] Update Progress: 341/2369 symbols completed. BSAC took 22.7s. Estimated remaining: 3h 31m 18s. Rate: 9.6 symbols/min [2025-09-16T17:35:35.030982] [INFO ] [MAIN ] BST: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:35:35.031266] [DEBUG] [LOCAL ] BST: No existing data file found in primary storage. [2025-09-16T17:35:35.031364] [WARN ] [MAIN ] BST: No existing data found, performing full refresh instead. [2025-09-16T17:35:35.031763] [INFO ] [MAIN ] BST: Wiping existing primary data before full refresh. [2025-09-16T17:35:35.031834] [INFO ] [LOCAL ] BST: Starting deletion of primary data. [2025-09-16T17:35:35.031905] [DEBUG] [LOCAL ] BST: No primary data file found to delete. [2025-09-16T17:35:35.032089] [INFO ] [MAIN ] BST: Starting full primary refresh. [2025-09-16T17:35:35.160606] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-09-16T17:35:35.417824] [INFO ] [REMOTE ] BST: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:35:35.574900] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-02. [2025-09-16T17:35:35.601679] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T17:35:36.374861] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-04. [2025-09-16T17:35:36.405015] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-09-16T17:35:37.174909] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-03. [2025-09-16T17:35:37.240013] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-09-16T17:35:37.974879] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-05. [2025-09-16T17:35:37.993826] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-09-16T17:35:38.386053] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-09-16T17:35:38.774906] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-06. [2025-09-16T17:35:39.169316] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-09-16T17:35:39.574923] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-07. [2025-09-16T17:35:40.141952] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-09-16T17:35:40.374914] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-08. [2025-09-16T17:35:40.856994] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-09-16T17:35:41.174934] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-04. [2025-09-16T17:35:41.974869] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-02. [2025-09-16T17:35:42.072194] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-09-16T17:35:42.774907] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-09. [2025-09-16T17:35:43.221345] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T17:35:43.491844] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-09-16T17:35:43.493256] [INFO ] [REMOTE ] BSL: Full fetch complete (9481 records). [2025-09-16T17:35:43.493324] [DEBUG] [VALIDATOR ] BSL: Validating primary data. [2025-09-16T17:35:43.580017] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-05. [2025-09-16T17:35:43.858858] [DEBUG] [VALIDATOR ] BSL: All expected trading days are present. [2025-09-16T17:35:43.858939] [DEBUG] [VALIDATOR ] BSL: Primary data validation passed. [2025-09-16T17:35:43.859185] [INFO ] [MAIN ] BSL: Fetched primary data is valid. [2025-09-16T17:35:43.859226] [DEBUG] [LOCAL ] BSL: Saving primary data. [2025-09-16T17:35:43.859683] [DEBUG] [LOCAL ] BSL: Trimmed 9481 records to 5178 (keeping 150 days of data) [2025-09-16T17:35:43.891655] [DEBUG] [LOCAL ] BSL: Primary data saved (5178 records). [2025-09-16T17:35:43.891781] [INFO ] [MAIN ] BSL: Full history saved. [2025-09-16T17:35:43.891891] [INFO ] [MAIN ] Update Progress: 342/2369 symbols completed. BSL took 21.5s. Estimated remaining: 3h 31m 28s. Rate: 9.6 symbols/min [2025-09-16T17:35:44.106920] [INFO ] [MAIN ] BSTZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:35:44.124978] [DEBUG] [LOCAL ] BSTZ: Loaded 18005 existing records from primary storage. [2025-09-16T17:35:44.125103] [INFO ] [MAIN ] BSTZ: Loaded 18005 existing records. [2025-09-16T17:35:44.307018] [WARN ] [MAIN ] BSTZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:35:44.307286] [INFO ] [MAIN ] BSTZ: Wiping existing primary data before full refresh. [2025-09-16T17:35:44.307464] [INFO ] [LOCAL ] BSTZ: Starting deletion of primary data. [2025-09-16T17:35:44.313106] [DEBUG] [LOCAL ] BSTZ: Deleted primary data file. [2025-09-16T17:35:44.317989] [INFO ] [MAIN ] BSTZ: Starting full primary refresh. [2025-09-16T17:35:44.377084] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-03. [2025-09-16T17:35:44.392061] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-09-16T17:35:44.696380] [INFO ] [REMOTE ] BSTZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:35:45.174954] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-02. [2025-09-16T17:35:45.542551] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T17:35:45.974907] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-06. [2025-09-16T17:35:46.265789] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T17:35:46.774899] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-03. [2025-09-16T17:35:47.349130] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T17:35:47.574913] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-09. [2025-09-16T17:35:47.604609] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T17:35:48.374955] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-04. [2025-09-16T17:35:48.673999] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-09-16T17:35:48.679350] [INFO ] [REMOTE ] BSBR: Full fetch complete (38184 records). [2025-09-16T17:35:48.679482] [DEBUG] [VALIDATOR ] BSBR: Validating primary data. [2025-09-16T17:35:49.056433] [DEBUG] [VALIDATOR ] BSBR: All expected trading days are present. [2025-09-16T17:35:49.056582] [DEBUG] [VALIDATOR ] BSBR: Primary data validation passed. [2025-09-16T17:35:49.057055] [INFO ] [MAIN ] BSBR: Fetched primary data is valid. [2025-09-16T17:35:49.057110] [DEBUG] [LOCAL ] BSBR: Saving primary data. [2025-09-16T17:35:49.058312] [DEBUG] [LOCAL ] BSBR: Trimmed 38184 records to 26737 (keeping 150 days of data) [2025-09-16T17:35:49.133389] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-09-16T17:35:49.175066] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-07. [2025-09-16T17:35:49.236602] [DEBUG] [LOCAL ] BSBR: Primary data saved (26737 records). [2025-09-16T17:35:49.236766] [INFO ] [MAIN ] BSBR: Full history saved. [2025-09-16T17:35:49.236886] [INFO ] [MAIN ] Update Progress: 343/2369 symbols completed. BSBR took 34.4s. Estimated remaining: 3h 31m 17s. Rate: 9.6 symbols/min [2025-09-16T17:35:49.409524] [INFO ] [MAIN ] BSX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:35:49.449400] [DEBUG] [LOCAL ] BSX: Loaded 39648 existing records from primary storage. [2025-09-16T17:35:49.449541] [INFO ] [MAIN ] BSX: Loaded 39648 existing records. [2025-09-16T17:35:49.625490] [WARN ] [MAIN ] BSX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:35:49.625747] [INFO ] [MAIN ] BSX: Wiping existing primary data before full refresh. [2025-09-16T17:35:49.625803] [INFO ] [LOCAL ] BSX: Starting deletion of primary data. [2025-09-16T17:35:49.626930] [DEBUG] [LOCAL ] BSX: Deleted primary data file. [2025-09-16T17:35:49.626981] [INFO ] [MAIN ] BSX: Starting full primary refresh. [2025-09-16T17:35:49.978886] [INFO ] [REMOTE ] BSX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:35:49.978987] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-04. [2025-09-16T17:35:50.069682] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-09-16T17:35:50.774909] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-02. [2025-09-16T17:35:50.899172] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-09-16T17:35:51.574900] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-05. [2025-09-16T17:35:52.374870] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-08. [2025-09-16T17:35:52.493624] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-09-16T17:35:52.916607] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-02 (2.14 seconds). [2025-09-16T17:35:53.174915] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-06. [2025-09-16T17:35:53.262245] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-09-16T17:35:53.974912] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-09. [2025-09-16T17:35:54.039825] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-09-16T17:35:54.774915] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-03. [2025-09-16T17:35:55.143885] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-09-16T17:35:55.146910] [INFO ] [REMOTE ] BST: Full fetch complete (21237 records). [2025-09-16T17:35:55.146999] [DEBUG] [VALIDATOR ] BST: Validating primary data. [2025-09-16T17:35:55.517735] [DEBUG] [VALIDATOR ] BST: All expected trading days are present. [2025-09-16T17:35:55.517834] [DEBUG] [VALIDATOR ] BST: Primary data validation passed. [2025-09-16T17:35:55.518341] [INFO ] [MAIN ] BST: Fetched primary data is valid. [2025-09-16T17:35:55.518417] [DEBUG] [LOCAL ] BST: Saving primary data. [2025-09-16T17:35:55.519445] [DEBUG] [LOCAL ] BST: Trimmed 21237 records to 13342 (keeping 150 days of data) [2025-09-16T17:35:55.580245] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-07. [2025-09-16T17:35:55.604880] [DEBUG] [LOCAL ] BST: Primary data saved (13342 records). [2025-09-16T17:35:55.605030] [INFO ] [MAIN ] BST: Full history saved. [2025-09-16T17:35:55.605115] [INFO ] [MAIN ] Update Progress: 344/2369 symbols completed. BST took 20.7s. Estimated remaining: 3h 31m 11s. Rate: 9.6 symbols/min [2025-09-16T17:35:55.775402] [INFO ] [MAIN ] BTA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:35:55.780662] [DEBUG] [LOCAL ] BTA: Loaded 4014 existing records from primary storage. [2025-09-16T17:35:55.780748] [INFO ] [MAIN ] BTA: Loaded 4014 existing records. [2025-09-16T17:35:55.957079] [WARN ] [MAIN ] BTA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:35:55.957363] [INFO ] [MAIN ] BTA: Wiping existing primary data before full refresh. [2025-09-16T17:35:55.957411] [INFO ] [LOCAL ] BTA: Starting deletion of primary data. [2025-09-16T17:35:55.957613] [DEBUG] [LOCAL ] BTA: Deleted primary data file. [2025-09-16T17:35:55.957685] [INFO ] [MAIN ] BTA: Starting full primary refresh. [2025-09-16T17:35:56.314855] [INFO ] [REMOTE ] BTA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:35:56.374851] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-05. [2025-09-16T17:35:56.468388] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-09-16T17:35:56.992692] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-09-16T17:35:57.174907] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-02. [2025-09-16T17:35:57.627713] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T17:35:57.974909] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-06. [2025-09-16T17:35:58.030764] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T17:35:58.774913] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-03. [2025-09-16T17:35:59.004451] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-09-16T17:35:59.371740] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-09-16T17:35:59.574930] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-04. [2025-09-16T17:35:59.981628] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-09-16T17:36:00.374895] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-05. [2025-09-16T17:36:01.174907] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-08. [2025-09-16T17:36:01.223704] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-09-16T17:36:01.974893] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-06. [2025-09-16T17:36:02.466291] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T17:36:02.491244] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-09-16T17:36:02.774897] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-07. [2025-09-16T17:36:03.297632] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-09-16T17:36:03.574900] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-04. [2025-09-16T17:36:04.374901] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-08. [2025-09-16T17:36:04.954272] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-09-16T17:36:05.174918] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-09. [2025-09-16T17:36:05.511009] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-09-16T17:36:05.974911] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-09. [2025-09-16T17:36:06.120361] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T17:36:06.123316] [INFO ] [REMOTE ] BSTZ: Full fetch complete (26915 records). [2025-09-16T17:36:06.123387] [DEBUG] [VALIDATOR ] BSTZ: Validating primary data. [2025-09-16T17:36:06.123924] [FATAL] [VALIDATOR ] BSTZ: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:36:06.124012] [FATAL] [MAIN ] BSTZ: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:36:06.124076] [WARN ] [MAIN ] BSTZ: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:36:06.124134] [INFO ] [LOCAL ] BSTZ: Starting deletion of primary data. [2025-09-16T17:36:06.124206] [DEBUG] [LOCAL ] BSTZ: No primary data file found to delete. [2025-09-16T17:36:06.124310] [INFO ] [LOCAL ] BSTZ: Starting deletion of derivative data. [2025-09-16T17:36:06.124592] [DEBUG] [LOCAL ] BSTZ: Deleted derivative data file. [2025-09-16T17:36:06.124688] [INFO ] [MAIN ] Update Progress: 345/2369 symbols completed. BSTZ took 22.2s. Estimated remaining: 3h 31m 30s. Rate: 9.6 symbols/min [2025-09-16T17:36:06.299413] [INFO ] [MAIN ] BTCM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:36:06.312560] [DEBUG] [LOCAL ] BTCM: Loaded 12784 existing records from primary storage. [2025-09-16T17:36:06.312699] [INFO ] [MAIN ] BTCM: Loaded 12784 existing records. [2025-09-16T17:36:06.499378] [WARN ] [MAIN ] BTCM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:36:06.503486] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-09-16T17:36:06.505075] [INFO ] [MAIN ] BTCM: Wiping existing primary data before full refresh. [2025-09-16T17:36:06.505195] [INFO ] [REMOTE ] BTA: Full fetch complete (5875 records). [2025-09-16T17:36:06.505294] [INFO ] [LOCAL ] BTCM: Starting deletion of primary data. [2025-09-16T17:36:06.505362] [DEBUG] [VALIDATOR ] BTA: Validating primary data. [2025-09-16T17:36:06.505911] [FATAL] [VALIDATOR ] BTA: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:36:06.506020] [FATAL] [MAIN ] BTA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:36:06.506083] [WARN ] [MAIN ] BTA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:36:06.506143] [INFO ] [LOCAL ] BTA: Starting deletion of primary data. [2025-09-16T17:36:06.506220] [DEBUG] [LOCAL ] BTA: No primary data file found to delete. [2025-09-16T17:36:06.506282] [INFO ] [LOCAL ] BTA: Starting deletion of derivative data. [2025-09-16T17:36:06.506438] [DEBUG] [LOCAL ] BTCM: Deleted primary data file. [2025-09-16T17:36:06.506507] [INFO ] [MAIN ] BTCM: Starting full primary refresh. [2025-09-16T17:36:06.506551] [DEBUG] [LOCAL ] BTA: Deleted derivative data file. [2025-09-16T17:36:06.576439] [INFO ] [MAIN ] Update Progress: 346/2369 symbols completed. BTA took 11.0s. Estimated remaining: 3h 30m 50s. Rate: 9.6 symbols/min [2025-09-16T17:36:06.778903] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-05. [2025-09-16T17:36:06.991534] [INFO ] [MAIN ] BTE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:36:07.098129] [INFO ] [REMOTE ] BTCM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:36:07.127044] [DEBUG] [LOCAL ] BTE: Loaded 38887 existing records from primary storage. [2025-09-16T17:36:07.127193] [INFO ] [MAIN ] BTE: Loaded 38887 existing records. [2025-09-16T17:36:07.320956] [WARN ] [MAIN ] BTE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:36:07.321351] [INFO ] [MAIN ] BTE: Wiping existing primary data before full refresh. [2025-09-16T17:36:07.321402] [INFO ] [LOCAL ] BTE: Starting deletion of primary data. [2025-09-16T17:36:07.322304] [DEBUG] [LOCAL ] BTE: Deleted primary data file. [2025-09-16T17:36:07.322355] [INFO ] [MAIN ] BTE: Starting full primary refresh. [2025-09-16T17:36:07.580197] [INFO ] [REMOTE ] BTCM: Fetching primary data for 2025-02. [2025-09-16T17:36:07.685509] [INFO ] [REMOTE ] BTE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:36:08.374869] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-02. [2025-09-16T17:36:08.535659] [INFO ] [REMOTE ] BTCM: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-09-16T17:36:08.749221] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-09-16T17:36:09.174914] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-07. [2025-09-16T17:36:09.974917] [INFO ] [REMOTE ] BTCM: Fetching primary data for 2025-03. [2025-09-16T17:36:10.196669] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-09-16T17:36:10.630874] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T17:36:10.774922] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-08. [2025-09-16T17:36:10.815683] [INFO ] [REMOTE ] BTCM: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-09-16T17:36:11.574948] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-06. [2025-09-16T17:36:12.171065] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-09-16T17:36:12.374911] [INFO ] [REMOTE ] BTCM: Fetching primary data for 2025-04. [2025-09-16T17:36:12.974437] [INFO ] [REMOTE ] BTCM: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-09-16T17:36:13.174906] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-03. [2025-09-16T17:36:13.427864] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-09-16T17:36:13.974918] [INFO ] [REMOTE ] BTCM: Fetching primary data for 2025-05. [2025-09-16T17:36:14.556341] [INFO ] [REMOTE ] BTCM: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-09-16T17:36:14.691099] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T17:36:14.774914] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-07. [2025-09-16T17:36:15.574920] [INFO ] [REMOTE ] BTCM: Fetching primary data for 2025-06. [2025-09-16T17:36:15.979174] [INFO ] [REMOTE ] BTCM: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-09-16T17:36:16.374923] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-04. [2025-09-16T17:36:16.687852] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T17:36:17.174921] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-09. [2025-09-16T17:36:17.974916] [INFO ] [REMOTE ] BTCM: Fetching primary data for 2025-07. [2025-09-16T17:36:18.185254] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T17:36:18.382606] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-09-16T17:36:18.387150] [INFO ] [REMOTE ] BSM: Full fetch complete (35939 records). [2025-09-16T17:36:18.387232] [DEBUG] [VALIDATOR ] BSM: Validating primary data. [2025-09-16T17:36:18.767782] [DEBUG] [VALIDATOR ] BSM: All expected trading days are present. [2025-09-16T17:36:18.767868] [DEBUG] [VALIDATOR ] BSM: Primary data validation passed. [2025-09-16T17:36:18.768356] [INFO ] [MAIN ] BSM: Fetched primary data is valid. [2025-09-16T17:36:18.768399] [DEBUG] [LOCAL ] BSM: Saving primary data. [2025-09-16T17:36:18.769245] [DEBUG] [LOCAL ] BSM: Trimmed 35939 records to 24221 (keeping 150 days of data) [2025-09-16T17:36:18.825019] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-08. [2025-09-16T17:36:18.919822] [DEBUG] [LOCAL ] BSM: Primary data saved (24221 records). [2025-09-16T17:36:18.919975] [INFO ] [MAIN ] BSM: Full history saved. [2025-09-16T17:36:18.920064] [INFO ] [MAIN ] Update Progress: 347/2369 symbols completed. BSM took 48.3s. Estimated remaining: 3h 31m 19s. Rate: 9.6 symbols/min [2025-09-16T17:36:19.090385] [INFO ] [MAIN ] BTI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:36:19.131011] [DEBUG] [LOCAL ] BTI: Loaded 39570 existing records from primary storage. [2025-09-16T17:36:19.131152] [INFO ] [MAIN ] BTI: Loaded 39570 existing records. [2025-09-16T17:36:19.309309] [WARN ] [MAIN ] BTI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:36:19.309546] [INFO ] [MAIN ] BTI: Wiping existing primary data before full refresh. [2025-09-16T17:36:19.309613] [INFO ] [LOCAL ] BTI: Starting deletion of primary data. [2025-09-16T17:36:19.310691] [DEBUG] [LOCAL ] BTI: Deleted primary data file. [2025-09-16T17:36:19.310743] [INFO ] [MAIN ] BTI: Starting full primary refresh. [2025-09-16T17:36:19.589222] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-05. [2025-09-16T17:36:19.678604] [INFO ] [REMOTE ] BTI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:36:19.885592] [INFO ] [REMOTE ] BTCM: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T17:36:20.374869] [INFO ] [REMOTE ] BTCM: Fetching primary data for 2025-08. [2025-09-16T17:36:20.927071] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-08 (2.10 seconds). [2025-09-16T17:36:21.174933] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-02. [2025-09-16T17:36:21.621135] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-09-16T17:36:21.784904] [INFO ] [REMOTE ] BTCM: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-09-16T17:36:21.974903] [INFO ] [REMOTE ] BTCM: Fetching primary data for 2025-09. [2025-09-16T17:36:22.774904] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-09. [2025-09-16T17:36:22.945240] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-09-16T17:36:23.273244] [INFO ] [REMOTE ] BTCM: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T17:36:23.276343] [INFO ] [REMOTE ] BTCM: Full fetch complete (14683 records). [2025-09-16T17:36:23.276434] [DEBUG] [VALIDATOR ] BTCM: Validating primary data. [2025-09-16T17:36:23.580000] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-03. [2025-09-16T17:36:23.647042] [DEBUG] [VALIDATOR ] BTCM: All expected trading days are present. [2025-09-16T17:36:23.647120] [DEBUG] [VALIDATOR ] BTCM: Primary data validation passed. [2025-09-16T17:36:23.647422] [INFO ] [MAIN ] BTCM: Fetched primary data is valid. [2025-09-16T17:36:23.647462] [DEBUG] [LOCAL ] BTCM: Saving primary data. [2025-09-16T17:36:23.648046] [DEBUG] [LOCAL ] BTCM: Trimmed 14683 records to 12997 (keeping 150 days of data) [2025-09-16T17:36:23.721172] [DEBUG] [LOCAL ] BTCM: Primary data saved (12997 records). [2025-09-16T17:36:23.721313] [INFO ] [MAIN ] BTCM: Full history saved. [2025-09-16T17:36:23.721399] [INFO ] [MAIN ] Update Progress: 348/2369 symbols completed. BTCM took 17.6s. Estimated remaining: 3h 31m 4s. Rate: 9.6 symbols/min [2025-09-16T17:36:23.892161] [INFO ] [MAIN ] BTO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:36:23.899566] [DEBUG] [LOCAL ] BTO: Loaded 6142 existing records from primary storage. [2025-09-16T17:36:23.899680] [INFO ] [MAIN ] BTO: Loaded 6142 existing records. [2025-09-16T17:36:24.076796] [WARN ] [MAIN ] BTO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:36:24.077046] [INFO ] [MAIN ] BTO: Wiping existing primary data before full refresh. [2025-09-16T17:36:24.077090] [INFO ] [LOCAL ] BTO: Starting deletion of primary data. [2025-09-16T17:36:24.077445] [DEBUG] [LOCAL ] BTO: Deleted primary data file. [2025-09-16T17:36:24.077495] [INFO ] [MAIN ] BTO: Starting full primary refresh. [2025-09-16T17:36:24.380020] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-06. [2025-09-16T17:36:24.434024] [INFO ] [REMOTE ] BTO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:36:24.913597] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-09-16T17:36:24.921820] [INFO ] [REMOTE ] BSX: Full fetch complete (58365 records). [2025-09-16T17:36:24.922019] [DEBUG] [VALIDATOR ] BSX: Validating primary data. [2025-09-16T17:36:25.189835] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-02. [2025-09-16T17:36:25.337665] [DEBUG] [VALIDATOR ] BSX: All expected trading days are present. [2025-09-16T17:36:25.337761] [DEBUG] [VALIDATOR ] BSX: Primary data validation passed. [2025-09-16T17:36:25.338533] [INFO ] [MAIN ] BSX: Fetched primary data is valid. [2025-09-16T17:36:25.338591] [DEBUG] [LOCAL ] BSX: Saving primary data. [2025-09-16T17:36:25.339967] [DEBUG] [LOCAL ] BSX: Trimmed 58365 records to 40038 (keeping 150 days of data) [2025-09-16T17:36:25.604082] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-09-16T17:36:25.634448] [DEBUG] [LOCAL ] BSX: Primary data saved (40038 records). [2025-09-16T17:36:25.634579] [INFO ] [MAIN ] BSX: Full history saved. [2025-09-16T17:36:25.634689] [INFO ] [MAIN ] Update Progress: 349/2369 symbols completed. BSX took 36.4s. Estimated remaining: 3h 30m 33s. Rate: 9.6 symbols/min [2025-09-16T17:36:25.810904] [INFO ] [MAIN ] BTT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:36:25.811075] [DEBUG] [LOCAL ] BTT: No existing data file found in primary storage. [2025-09-16T17:36:25.811114] [WARN ] [MAIN ] BTT: No existing data found, performing full refresh instead. [2025-09-16T17:36:25.811249] [INFO ] [MAIN ] BTT: Wiping existing primary data before full refresh. [2025-09-16T17:36:25.811303] [INFO ] [LOCAL ] BTT: Starting deletion of primary data. [2025-09-16T17:36:25.811359] [DEBUG] [LOCAL ] BTT: No primary data file found to delete. [2025-09-16T17:36:25.811405] [INFO ] [MAIN ] BTT: Starting full primary refresh. [2025-09-16T17:36:25.986739] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-04. [2025-09-16T17:36:26.068736] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-09-16T17:36:26.184464] [INFO ] [REMOTE ] BTT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:36:26.218259] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-09-16T17:36:26.774907] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-07. [2025-09-16T17:36:27.574965] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-03. [2025-09-16T17:36:28.105265] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-09-16T17:36:28.156555] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T17:36:28.255850] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T17:36:28.374926] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-08. [2025-09-16T17:36:29.174929] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-02. [2025-09-16T17:36:29.974947] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-05. [2025-09-16T17:36:30.085252] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-09-16T17:36:30.131038] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-09-16T17:36:30.774911] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-09. [2025-09-16T17:36:31.574889] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-03. [2025-09-16T17:36:31.767917] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T17:36:32.121420] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T17:36:32.127209] [INFO ] [REMOTE ] BTE: Full fetch complete (56910 records). [2025-09-16T17:36:32.127320] [DEBUG] [VALIDATOR ] BTE: Validating primary data. [2025-09-16T17:36:32.127961] [FATAL] [VALIDATOR ] BTE: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:36:32.128041] [FATAL] [MAIN ] BTE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:36:32.128080] [WARN ] [MAIN ] BTE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:36:32.128156] [INFO ] [LOCAL ] BTE: Starting deletion of primary data. [2025-09-16T17:36:32.128216] [DEBUG] [LOCAL ] BTE: No primary data file found to delete. [2025-09-16T17:36:32.128253] [INFO ] [LOCAL ] BTE: Starting deletion of derivative data. [2025-09-16T17:36:32.128616] [DEBUG] [LOCAL ] BTE: Deleted derivative data file. [2025-09-16T17:36:32.128723] [INFO ] [MAIN ] Update Progress: 350/2369 symbols completed. BTE took 25.5s. Estimated remaining: 3h 30m 28s. Rate: 9.6 symbols/min [2025-09-16T17:36:32.310002] [INFO ] [MAIN ] BTU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:36:32.311036] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-09-16T17:36:32.356324] [DEBUG] [LOCAL ] BTU: Loaded 39314 existing records from primary storage. [2025-09-16T17:36:32.356471] [INFO ] [MAIN ] BTU: Loaded 39314 existing records. [2025-09-16T17:36:32.380050] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-04. [2025-09-16T17:36:32.534416] [WARN ] [MAIN ] BTU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:36:32.534670] [INFO ] [MAIN ] BTU: Wiping existing primary data before full refresh. [2025-09-16T17:36:32.534718] [INFO ] [LOCAL ] BTU: Starting deletion of primary data. [2025-09-16T17:36:32.535614] [DEBUG] [LOCAL ] BTU: Deleted primary data file. [2025-09-16T17:36:32.535690] [INFO ] [MAIN ] BTU: Starting full primary refresh. [2025-09-16T17:36:32.893024] [INFO ] [REMOTE ] BTU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:36:33.174874] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-02. [2025-09-16T17:36:33.575187] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-09-16T17:36:33.974916] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-06. [2025-09-16T17:36:34.774921] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-04. [2025-09-16T17:36:34.905711] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T17:36:35.540993] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T17:36:35.574845] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-07. [2025-09-16T17:36:35.661829] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-09-16T17:36:36.374901] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-05. [2025-09-16T17:36:37.176936] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-03. [2025-09-16T17:36:37.180436] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-09-16T17:36:37.704306] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-09-16T17:36:37.974910] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-06. [2025-09-16T17:36:38.561352] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T17:36:38.774859] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-07. [2025-09-16T17:36:38.908843] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T17:36:39.509177] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-09-16T17:36:39.574909] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-05. [2025-09-16T17:36:40.374876] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-08. [2025-09-16T17:36:40.672221] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-09-16T17:36:41.174904] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-06. [2025-09-16T17:36:41.918417] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T17:36:41.942454] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T17:36:41.974918] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-07. [2025-09-16T17:36:42.774889] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-04. [2025-09-16T17:36:42.886253] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-09-16T17:36:43.574923] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-09. [2025-09-16T17:36:44.374971] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-08. [2025-09-16T17:36:44.933197] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-09-16T17:36:44.987169] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-04 (2.21 seconds). [2025-09-16T17:36:45.174896] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-05. [2025-09-16T17:36:45.555190] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-09-16T17:36:45.562037] [INFO ] [REMOTE ] BTI: Full fetch complete (58222 records). [2025-09-16T17:36:45.562251] [DEBUG] [VALIDATOR ] BTI: Validating primary data. [2025-09-16T17:36:45.966626] [DEBUG] [VALIDATOR ] BTI: All expected trading days are present. [2025-09-16T17:36:45.966779] [DEBUG] [VALIDATOR ] BTI: Primary data validation passed. [2025-09-16T17:36:45.967772] [INFO ] [MAIN ] BTI: Fetched primary data is valid. [2025-09-16T17:36:45.967833] [DEBUG] [LOCAL ] BTI: Saving primary data. [2025-09-16T17:36:45.969807] [DEBUG] [LOCAL ] BTI: Trimmed 58222 records to 39958 (keeping 150 days of data) [2025-09-16T17:36:45.974939] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-08. [2025-09-16T17:36:46.235701] [DEBUG] [LOCAL ] BTI: Primary data saved (39958 records). [2025-09-16T17:36:46.235938] [INFO ] [MAIN ] BTI: Full history saved. [2025-09-16T17:36:46.237171] [INFO ] [MAIN ] Update Progress: 351/2369 symbols completed. BTI took 27.3s. Estimated remaining: 3h 31m 7s. Rate: 9.6 symbols/min [2025-09-16T17:36:46.416578] [INFO ] [MAIN ] BTX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:36:46.416819] [DEBUG] [LOCAL ] BTX: No existing data file found in primary storage. [2025-09-16T17:36:46.416868] [WARN ] [MAIN ] BTX: No existing data found, performing full refresh instead. [2025-09-16T17:36:46.417000] [INFO ] [MAIN ] BTX: Wiping existing primary data before full refresh. [2025-09-16T17:36:46.417040] [INFO ] [LOCAL ] BTX: Starting deletion of primary data. [2025-09-16T17:36:46.417081] [DEBUG] [LOCAL ] BTX: No primary data file found to delete. [2025-09-16T17:36:46.417113] [INFO ] [MAIN ] BTX: Starting full primary refresh. [2025-09-16T17:36:46.788673] [INFO ] [REMOTE ] BTX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:36:46.788736] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-09. [2025-09-16T17:36:46.791382] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-09-16T17:36:46.849737] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-09-16T17:36:47.574881] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-06. [2025-09-16T17:36:47.779087] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T17:36:47.780712] [INFO ] [REMOTE ] BTO: Full fetch complete (9820 records). [2025-09-16T17:36:47.780795] [DEBUG] [VALIDATOR ] BTO: Validating primary data. [2025-09-16T17:36:48.156187] [DEBUG] [VALIDATOR ] BTO: All expected trading days are present. [2025-09-16T17:36:48.156267] [DEBUG] [VALIDATOR ] BTO: Primary data validation passed. [2025-09-16T17:36:48.156511] [INFO ] [MAIN ] BTO: Fetched primary data is valid. [2025-09-16T17:36:48.156553] [DEBUG] [LOCAL ] BTO: Saving primary data. [2025-09-16T17:36:48.157034] [DEBUG] [LOCAL ] BTO: Trimmed 9820 records to 6236 (keeping 150 days of data) [2025-09-16T17:36:48.194297] [DEBUG] [LOCAL ] BTO: Primary data saved (6236 records). [2025-09-16T17:36:48.194382] [INFO ] [MAIN ] BTO: Full history saved. [2025-09-16T17:36:48.194477] [INFO ] [MAIN ] Update Progress: 352/2369 symbols completed. BTO took 24.5s. Estimated remaining: 3h 30m 36s. Rate: 9.6 symbols/min [2025-09-16T17:36:48.373952] [INFO ] [MAIN ] BTZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:36:48.374871] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-09. [2025-09-16T17:36:48.391766] [DEBUG] [LOCAL ] BTZ: Loaded 16047 existing records from primary storage. [2025-09-16T17:36:48.392265] [INFO ] [MAIN ] BTZ: Loaded 16047 existing records. [2025-09-16T17:36:48.572550] [WARN ] [MAIN ] BTZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:36:48.572833] [INFO ] [MAIN ] BTZ: Wiping existing primary data before full refresh. [2025-09-16T17:36:48.572907] [INFO ] [LOCAL ] BTZ: Starting deletion of primary data. [2025-09-16T17:36:48.573471] [DEBUG] [LOCAL ] BTZ: Deleted primary data file. [2025-09-16T17:36:48.573531] [INFO ] [MAIN ] BTZ: Starting full primary refresh. [2025-09-16T17:36:48.930972] [INFO ] [REMOTE ] BTZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:36:49.174874] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-02. [2025-09-16T17:36:49.338965] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T17:36:49.373775] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T17:36:49.376249] [INFO ] [REMOTE ] BTT: Full fetch complete (17329 records). [2025-09-16T17:36:49.376319] [DEBUG] [VALIDATOR ] BTT: Validating primary data. [2025-09-16T17:36:49.733489] [DEBUG] [VALIDATOR ] BTT: All expected trading days are present. [2025-09-16T17:36:49.733561] [DEBUG] [VALIDATOR ] BTT: Primary data validation passed. [2025-09-16T17:36:49.733853] [INFO ] [MAIN ] BTT: Fetched primary data is valid. [2025-09-16T17:36:49.733897] [DEBUG] [LOCAL ] BTT: Saving primary data. [2025-09-16T17:36:49.734419] [DEBUG] [LOCAL ] BTT: Trimmed 17329 records to 11485 (keeping 150 days of data) [2025-09-16T17:36:49.808030] [DEBUG] [LOCAL ] BTT: Primary data saved (11485 records). [2025-09-16T17:36:49.808164] [INFO ] [MAIN ] BTT: Full history saved. [2025-09-16T17:36:49.808252] [INFO ] [MAIN ] Update Progress: 353/2369 symbols completed. BTT took 24.2s. Estimated remaining: 3h 30m 3s. Rate: 9.6 symbols/min [2025-09-16T17:36:49.976569] [INFO ] [MAIN ] BUD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:36:49.975337] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-02. [2025-09-16T17:36:50.013083] [DEBUG] [LOCAL ] BUD: Loaded 38809 existing records from primary storage. [2025-09-16T17:36:50.013225] [INFO ] [MAIN ] BUD: Loaded 38809 existing records. [2025-09-16T17:36:50.129370] [INFO ] [REMOTE ] BTX: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-09-16T17:36:50.200968] [WARN ] [MAIN ] BUD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:36:50.201154] [INFO ] [MAIN ] BUD: Wiping existing primary data before full refresh. [2025-09-16T17:36:50.201201] [INFO ] [LOCAL ] BUD: Starting deletion of primary data. [2025-09-16T17:36:50.202013] [DEBUG] [LOCAL ] BUD: Deleted primary data file. [2025-09-16T17:36:50.202079] [INFO ] [MAIN ] BUD: Starting full primary refresh. [2025-09-16T17:36:50.555243] [INFO ] [REMOTE ] BUD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:36:50.774846] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-02. [2025-09-16T17:36:51.056816] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-09-16T17:36:51.574861] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-07. [2025-09-16T17:36:52.374910] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-03. [2025-09-16T17:36:52.713603] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-02 (1.94 seconds). [2025-09-16T17:36:53.174920] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-03. [2025-09-16T17:36:53.498152] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T17:36:53.859722] [INFO ] [REMOTE ] BTX: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T17:36:53.974888] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-04. [2025-09-16T17:36:54.774922] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-08. [2025-09-16T17:36:54.889861] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T17:36:54.921845] [INFO ] [REMOTE ] BTX: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-09-16T17:36:55.574903] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-05. [2025-09-16T17:36:56.374932] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-03. [2025-09-16T17:36:56.516841] [INFO ] [REMOTE ] BTX: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-09-16T17:36:56.674023] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-09-16T17:36:57.119240] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-09-16T17:36:57.174993] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-04. [2025-09-16T17:36:57.974905] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-04. [2025-09-16T17:36:58.354714] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T17:36:58.774902] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-05. [2025-09-16T17:36:59.447614] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T17:36:59.574903] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-09. [2025-09-16T17:36:59.803769] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-09-16T17:37:00.374886] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-06. [2025-09-16T17:37:01.174911] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-05. [2025-09-16T17:37:01.356758] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T17:37:01.363429] [INFO ] [REMOTE ] BTU: Full fetch complete (57932 records). [2025-09-16T17:37:01.363609] [DEBUG] [VALIDATOR ] BTU: Validating primary data. [2025-09-16T17:37:01.748015] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T17:37:01.766472] [DEBUG] [VALIDATOR ] BTU: All expected trading days are present. [2025-09-16T17:37:01.766598] [DEBUG] [VALIDATOR ] BTU: Primary data validation passed. [2025-09-16T17:37:01.771959] [INFO ] [MAIN ] BTU: Fetched primary data is valid. [2025-09-16T17:37:01.772137] [DEBUG] [LOCAL ] BTU: Saving primary data. [2025-09-16T17:37:01.773520] [DEBUG] [LOCAL ] BTU: Trimmed 57932 records to 39703 (keeping 150 days of data) [2025-09-16T17:37:01.991065] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-06. [2025-09-16T17:37:02.025509] [DEBUG] [LOCAL ] BTU: Primary data saved (39703 records). [2025-09-16T17:37:02.025679] [INFO ] [MAIN ] BTU: Full history saved. [2025-09-16T17:37:02.026880] [INFO ] [MAIN ] Update Progress: 354/2369 symbols completed. BTU took 29.9s. Estimated remaining: 3h 30m 30s. Rate: 9.6 symbols/min [2025-09-16T17:37:02.196739] [INFO ] [MAIN ] BUI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:02.208002] [DEBUG] [LOCAL ] BUI: Loaded 8539 existing records from primary storage. [2025-09-16T17:37:02.208132] [INFO ] [MAIN ] BUI: Loaded 8539 existing records. [2025-09-16T17:37:02.389589] [WARN ] [MAIN ] BUI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:02.389888] [INFO ] [MAIN ] BUI: Wiping existing primary data before full refresh. [2025-09-16T17:37:02.389975] [INFO ] [LOCAL ] BUI: Starting deletion of primary data. [2025-09-16T17:37:02.390306] [DEBUG] [LOCAL ] BUI: Deleted primary data file. [2025-09-16T17:37:02.390356] [INFO ] [MAIN ] BUI: Starting full primary refresh. [2025-09-16T17:37:02.759262] [INFO ] [REMOTE ] BUI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:02.774865] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-02. [2025-09-16T17:37:03.035782] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-09-16T17:37:03.189619] [INFO ] [REMOTE ] BTX: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T17:37:03.574919] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-07. [2025-09-16T17:37:03.664398] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-09-16T17:37:04.374893] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-06. [2025-09-16T17:37:04.700250] [INFO ] [REMOTE ] BTX: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-09-16T17:37:05.174894] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-08. [2025-09-16T17:37:05.974905] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-07. [2025-09-16T17:37:06.104579] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-09-16T17:37:06.165804] [INFO ] [REMOTE ] BTX: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T17:37:06.774925] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-07. [2025-09-16T17:37:06.797369] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-09-16T17:37:07.574909] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-03. [2025-09-16T17:37:08.374905] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-09. [2025-09-16T17:37:08.409536] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T17:37:08.531614] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T17:37:09.174900] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-04. [2025-09-16T17:37:09.737351] [INFO ] [REMOTE ] BTX: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T17:37:09.741052] [INFO ] [REMOTE ] BTX: Full fetch complete (34419 records). [2025-09-16T17:37:09.741159] [DEBUG] [VALIDATOR ] BTX: Validating primary data. [2025-09-16T17:37:09.932874] [WARN ] [VALIDATOR ] BTX: Validation failed - insufficient historical data. Have 144 unique days, need 150 days [2025-09-16T17:37:09.933363] [FATAL] [MAIN ] BTX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:37:09.933416] [WARN ] [MAIN ] BTX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:37:09.933465] [INFO ] [LOCAL ] BTX: Starting deletion of primary data. [2025-09-16T17:37:09.933533] [DEBUG] [LOCAL ] BTX: No primary data file found to delete. [2025-09-16T17:37:09.933591] [INFO ] [LOCAL ] BTX: Starting deletion of derivative data. [2025-09-16T17:37:09.933632] [DEBUG] [LOCAL ] BTX: No derivative data file found to delete. [2025-09-16T17:37:09.933747] [INFO ] [MAIN ] Update Progress: 355/2369 symbols completed. BTX took 23.7s. Estimated remaining: 3h 30m 33s. Rate: 9.6 symbols/min [2025-09-16T17:37:09.979997] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-08. [2025-09-16T17:37:10.123128] [INFO ] [MAIN ] BUR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:10.131580] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-09-16T17:37:10.173268] [DEBUG] [LOCAL ] BUR: Loaded 33462 existing records from primary storage. [2025-09-16T17:37:10.173412] [INFO ] [MAIN ] BUR: Loaded 33462 existing records. [2025-09-16T17:37:10.360126] [WARN ] [MAIN ] BUR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:10.360413] [INFO ] [MAIN ] BUR: Wiping existing primary data before full refresh. [2025-09-16T17:37:10.360469] [INFO ] [LOCAL ] BUR: Starting deletion of primary data. [2025-09-16T17:37:10.361222] [DEBUG] [LOCAL ] BUR: Deleted primary data file. [2025-09-16T17:37:10.361272] [INFO ] [MAIN ] BUR: Starting full primary refresh. [2025-09-16T17:37:10.715089] [INFO ] [REMOTE ] BUR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:10.774930] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-05. [2025-09-16T17:37:11.532824] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-09-16T17:37:11.574867] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-06. [2025-09-16T17:37:11.835796] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-09-16T17:37:12.083626] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-09-16T17:37:12.374887] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-07. [2025-09-16T17:37:13.073906] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-09-16T17:37:13.174911] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-09. [2025-09-16T17:37:13.974938] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-08. [2025-09-16T17:37:14.576244] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:37:14.582775] [INFO ] [REMOTE ] BUD: Full fetch complete (56980 records). [2025-09-16T17:37:14.582881] [DEBUG] [VALIDATOR ] BUD: Validating primary data. [2025-09-16T17:37:14.583527] [FATAL] [VALIDATOR ] BUD: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:37:14.583603] [FATAL] [MAIN ] BUD: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:37:14.583666] [WARN ] [MAIN ] BUD: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:37:14.583709] [INFO ] [LOCAL ] BUD: Starting deletion of primary data. [2025-09-16T17:37:14.583789] [DEBUG] [LOCAL ] BUD: No primary data file found to delete. [2025-09-16T17:37:14.583830] [INFO ] [LOCAL ] BUD: Starting deletion of derivative data. [2025-09-16T17:37:14.584129] [DEBUG] [LOCAL ] BUD: Deleted derivative data file. [2025-09-16T17:37:14.584216] [INFO ] [MAIN ] Update Progress: 356/2369 symbols completed. BUD took 24.8s. Estimated remaining: 3h 30m 18s. Rate: 9.6 symbols/min [2025-09-16T17:37:14.755711] [INFO ] [MAIN ] BURL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:14.775143] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-08. [2025-09-16T17:37:14.790316] [DEBUG] [LOCAL ] BURL: Loaded 33645 existing records from primary storage. [2025-09-16T17:37:14.790440] [INFO ] [MAIN ] BURL: Loaded 33645 existing records. [2025-09-16T17:37:14.972633] [WARN ] [MAIN ] BURL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:14.972953] [INFO ] [MAIN ] BURL: Wiping existing primary data before full refresh. [2025-09-16T17:37:14.973003] [INFO ] [LOCAL ] BURL: Starting deletion of primary data. [2025-09-16T17:37:14.973845] [DEBUG] [LOCAL ] BURL: Deleted primary data file. [2025-09-16T17:37:14.973896] [INFO ] [MAIN ] BURL: Starting full primary refresh. [2025-09-16T17:37:15.337753] [INFO ] [REMOTE ] BURL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:15.350842] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T17:37:15.561370] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T17:37:15.574848] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-09. [2025-09-16T17:37:16.330434] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-09-16T17:37:16.332578] [INFO ] [REMOTE ] BUI: Full fetch complete (12250 records). [2025-09-16T17:37:16.332674] [DEBUG] [VALIDATOR ] BUI: Validating primary data. [2025-09-16T17:37:16.380116] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-02. [2025-09-16T17:37:16.706629] [DEBUG] [VALIDATOR ] BUI: All expected trading days are present. [2025-09-16T17:37:16.706729] [DEBUG] [VALIDATOR ] BUI: Primary data validation passed. [2025-09-16T17:37:16.707019] [INFO ] [MAIN ] BUI: Fetched primary data is valid. [2025-09-16T17:37:16.707079] [DEBUG] [LOCAL ] BUI: Saving primary data. [2025-09-16T17:37:16.707706] [DEBUG] [LOCAL ] BUI: Trimmed 12250 records to 8640 (keeping 150 days of data) [2025-09-16T17:37:16.763148] [DEBUG] [LOCAL ] BUI: Primary data saved (8640 records). [2025-09-16T17:37:16.763271] [INFO ] [MAIN ] BUI: Full history saved. [2025-09-16T17:37:16.763376] [INFO ] [MAIN ] Update Progress: 357/2369 symbols completed. BUI took 14.7s. Estimated remaining: 3h 29m 49s. Rate: 9.6 symbols/min [2025-09-16T17:37:17.174930] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-09. [2025-09-16T17:37:17.640068] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-09-16T17:37:17.974901] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-03. [2025-09-16T17:37:18.143527] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T17:37:18.147219] [INFO ] [REMOTE ] BTZ: Full fetch complete (24972 records). [2025-09-16T17:37:18.147294] [DEBUG] [VALIDATOR ] BTZ: Validating primary data. [2025-09-16T17:37:18.148491] [FATAL] [VALIDATOR ] BTZ: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:37:18.148567] [FATAL] [MAIN ] BTZ: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:37:18.148604] [WARN ] [MAIN ] BTZ: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:37:18.148659] [INFO ] [LOCAL ] BTZ: Starting deletion of primary data. [2025-09-16T17:37:18.148720] [DEBUG] [LOCAL ] BTZ: No primary data file found to delete. [2025-09-16T17:37:18.148768] [INFO ] [LOCAL ] BTZ: Starting deletion of derivative data. [2025-09-16T17:37:18.148970] [DEBUG] [LOCAL ] BTZ: Deleted derivative data file. [2025-09-16T17:37:18.149045] [INFO ] [MAIN ] Update Progress: 358/2369 symbols completed. BTZ took 30.0s. Estimated remaining: 3h 29m 15s. Rate: 9.6 symbols/min [2025-09-16T17:37:18.774889] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-02. [2025-09-16T17:37:19.238140] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T17:37:19.574919] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-04. [2025-09-16T17:37:21.005936] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:37:21.016523] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-05. [2025-09-16T17:37:21.525981] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-02 (2.75 seconds). [2025-09-16T17:37:21.533879] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-03. [2025-09-16T17:37:22.334096] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T17:37:22.344378] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-06. [2025-09-16T17:37:22.674575] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T17:37:22.774913] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-04. [2025-09-16T17:37:22.882309] [DEBUG] [REMOTE ] BUR: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-09-16T17:37:22.882620] [WARN ] [REMOTE ] BUR: No data returned for month 2025-04 [2025-09-16T17:37:23.574967] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-05. [2025-09-16T17:37:23.721969] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T17:37:24.374964] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-07. [2025-09-16T17:37:24.714130] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T17:37:25.174906] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-06. [2025-09-16T17:37:25.616889] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-09-16T17:37:25.974976] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-08. [2025-09-16T17:37:26.312167] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T17:37:26.774921] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-07. [2025-09-16T17:37:27.210553] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T17:37:27.574913] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-09. [2025-09-16T17:37:27.942863] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T17:37:28.374952] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-08. [2025-09-16T17:37:28.988709] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T17:37:28.994061] [INFO ] [REMOTE ] BURL: Full fetch complete (50411 records). [2025-09-16T17:37:28.994201] [DEBUG] [VALIDATOR ] BURL: Validating primary data. [2025-09-16T17:37:29.376129] [DEBUG] [VALIDATOR ] BURL: All expected trading days are present. [2025-09-16T17:37:29.376228] [DEBUG] [VALIDATOR ] BURL: Primary data validation passed. [2025-09-16T17:37:29.376847] [INFO ] [MAIN ] BURL: Fetched primary data is valid. [2025-09-16T17:37:29.376904] [DEBUG] [LOCAL ] BURL: Saving primary data. [2025-09-16T17:37:29.378076] [DEBUG] [LOCAL ] BURL: Trimmed 50411 records to 33959 (keeping 150 days of data) [2025-09-16T17:37:29.587515] [DEBUG] [LOCAL ] BURL: Primary data saved (33959 records). [2025-09-16T17:37:29.587687] [INFO ] [MAIN ] BURL: Full history saved. [2025-09-16T17:37:29.587803] [INFO ] [MAIN ] Update Progress: 359/2369 symbols completed. BURL took 15.0s. Estimated remaining: 3h 29m 38s. Rate: 9.6 symbols/min [2025-09-16T17:37:29.642527] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-09-16T17:37:29.652459] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-09. [2025-09-16T17:37:30.940403] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T17:37:30.945578] [INFO ] [REMOTE ] BUR: Full fetch complete (41520 records). [2025-09-16T17:37:30.945753] [DEBUG] [VALIDATOR ] BUR: Validating primary data. [2025-09-16T17:37:30.946323] [FATAL] [VALIDATOR ] BUR: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:37:30.946427] [FATAL] [MAIN ] BUR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:37:30.946473] [WARN ] [MAIN ] BUR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:37:30.946536] [INFO ] [LOCAL ] BUR: Starting deletion of primary data. [2025-09-16T17:37:30.946617] [DEBUG] [LOCAL ] BUR: No primary data file found to delete. [2025-09-16T17:37:30.946695] [INFO ] [LOCAL ] BUR: Starting deletion of derivative data. [2025-09-16T17:37:30.947020] [DEBUG] [LOCAL ] BUR: Deleted derivative data file. [2025-09-16T17:37:30.947722] [INFO ] [MAIN ] Update Progress: 360/2369 symbols completed. BUR took 21.0s. Estimated remaining: 3h 29m 4s. Rate: 9.6 symbols/min [2025-09-16T17:37:31.533982] [INFO ] [MAIN ] BW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:31.687583] [INFO ] [MAIN ] BVN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:31.683448] [INFO ] [MAIN ] BV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:31.695779] [INFO ] [MAIN ] BUXX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:31.712961] [DEBUG] [LOCAL ] BUXX: Loaded 11375 existing records from primary storage. [2025-09-16T17:37:31.713381] [INFO ] [MAIN ] BUXX: Loaded 11375 existing records. [2025-09-16T17:37:31.865773] [DEBUG] [LOCAL ] BW: Loaded 28077 existing records from primary storage. [2025-09-16T17:37:31.922407] [INFO ] [MAIN ] BW: Loaded 28077 existing records. [2025-09-16T17:37:31.950262] [DEBUG] [LOCAL ] BVN: Loaded 31889 existing records from primary storage. [2025-09-16T17:37:31.967304] [INFO ] [MAIN ] BVN: Loaded 31889 existing records. [2025-09-16T17:37:31.967176] [WARN ] [MAIN ] BUXX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:31.999073] [DEBUG] [LOCAL ] BV: Loaded 28319 existing records from primary storage. [2025-09-16T17:37:32.055765] [INFO ] [MAIN ] BV: Loaded 28319 existing records. [2025-09-16T17:37:32.095108] [INFO ] [MAIN ] BUXX: Wiping existing primary data before full refresh. [2025-09-16T17:37:32.146463] [INFO ] [LOCAL ] BUXX: Starting deletion of primary data. [2025-09-16T17:37:32.178754] [DEBUG] [LOCAL ] BUXX: Deleted primary data file. [2025-09-16T17:37:32.221249] [INFO ] [MAIN ] BUXX: Starting full primary refresh. [2025-09-16T17:37:32.574671] [WARN ] [MAIN ] BVN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:32.574771] [WARN ] [MAIN ] BW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:32.618449] [INFO ] [MAIN ] BW: Wiping existing primary data before full refresh. [2025-09-16T17:37:32.637989] [INFO ] [MAIN ] BVN: Wiping existing primary data before full refresh. [2025-09-16T17:37:32.638154] [INFO ] [LOCAL ] BW: Starting deletion of primary data. [2025-09-16T17:37:32.646757] [WARN ] [MAIN ] BV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:32.646845] [INFO ] [LOCAL ] BVN: Starting deletion of primary data. [2025-09-16T17:37:32.715949] [DEBUG] [LOCAL ] BVN: Deleted primary data file. [2025-09-16T17:37:32.691280] [DEBUG] [LOCAL ] BW: Deleted primary data file. [2025-09-16T17:37:32.724520] [INFO ] [MAIN ] BW: Starting full primary refresh. [2025-09-16T17:37:32.691213] [INFO ] [MAIN ] BV: Wiping existing primary data before full refresh. [2025-09-16T17:37:32.731561] [INFO ] [LOCAL ] BV: Starting deletion of primary data. [2025-09-16T17:37:32.731467] [INFO ] [MAIN ] BVN: Starting full primary refresh. [2025-09-16T17:37:32.801088] [DEBUG] [LOCAL ] BV: Deleted primary data file. [2025-09-16T17:37:32.855145] [INFO ] [MAIN ] BV: Starting full primary refresh. [2025-09-16T17:37:33.343788] [INFO ] [REMOTE ] BUXX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:33.412682] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-02. [2025-09-16T17:37:33.900429] [INFO ] [REMOTE ] BW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:33.937732] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-02. [2025-09-16T17:37:34.088213] [INFO ] [REMOTE ] BV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:34.093908] [INFO ] [REMOTE ] BVN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:34.094330] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-02. [2025-09-16T17:37:34.094639] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-02. [2025-09-16T17:37:34.378788] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-09-16T17:37:34.381410] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-03. [2025-09-16T17:37:34.884893] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-09-16T17:37:34.888157] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-04. [2025-09-16T17:37:34.993027] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-09-16T17:37:35.000677] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-03. [2025-09-16T17:37:35.364136] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T17:37:35.527683] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-09-16T17:37:35.574843] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-05. [2025-09-16T17:37:35.680464] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T17:37:36.237476] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-09-16T17:37:36.374917] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-03. [2025-09-16T17:37:37.174952] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-06. [2025-09-16T17:37:37.246471] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-03 (2.25 seconds). [2025-09-16T17:37:37.938089] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T17:37:37.974849] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-04. [2025-09-16T17:37:38.081440] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-09-16T17:37:38.774914] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-07. [2025-09-16T17:37:39.534568] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T17:37:39.574893] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-04. [2025-09-16T17:37:39.758087] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-09-16T17:37:40.374898] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-08. [2025-09-16T17:37:40.947471] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T17:37:41.174890] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-05. [2025-09-16T17:37:41.209691] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-09-16T17:37:41.974916] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-05. [2025-09-16T17:37:42.627521] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T17:37:42.774896] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-06. [2025-09-16T17:37:43.171500] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T17:37:43.574900] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-09. [2025-09-16T17:37:44.029042] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T17:37:44.374929] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-06. [2025-09-16T17:37:44.568047] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T17:37:44.570157] [INFO ] [REMOTE ] BUXX: Full fetch complete (16205 records). [2025-09-16T17:37:44.570232] [DEBUG] [VALIDATOR ] BUXX: Validating primary data. [2025-09-16T17:37:44.936544] [DEBUG] [VALIDATOR ] BUXX: All expected trading days are present. [2025-09-16T17:37:44.936664] [DEBUG] [VALIDATOR ] BUXX: Primary data validation passed. [2025-09-16T17:37:44.937025] [INFO ] [MAIN ] BUXX: Fetched primary data is valid. [2025-09-16T17:37:44.937067] [DEBUG] [LOCAL ] BUXX: Saving primary data. [2025-09-16T17:37:44.937739] [DEBUG] [LOCAL ] BUXX: Trimmed 16205 records to 11499 (keeping 150 days of data) [2025-09-16T17:37:45.011490] [DEBUG] [LOCAL ] BUXX: Primary data saved (11499 records). [2025-09-16T17:37:45.011664] [INFO ] [MAIN ] BUXX: Full history saved. [2025-09-16T17:37:45.011756] [INFO ] [MAIN ] Update Progress: 361/2369 symbols completed. BUXX took 14.1s. Estimated remaining: 3h 29m 42s. Rate: 9.6 symbols/min [2025-09-16T17:37:45.180121] [INFO ] [MAIN ] BWA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:45.180242] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-07. [2025-09-16T17:37:45.226559] [DEBUG] [LOCAL ] BWA: Loaded 38415 existing records from primary storage. [2025-09-16T17:37:45.226717] [INFO ] [MAIN ] BWA: Loaded 38415 existing records. [2025-09-16T17:37:45.402182] [WARN ] [MAIN ] BWA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:45.402415] [INFO ] [MAIN ] BWA: Wiping existing primary data before full refresh. [2025-09-16T17:37:45.402460] [INFO ] [LOCAL ] BWA: Starting deletion of primary data. [2025-09-16T17:37:45.403432] [DEBUG] [LOCAL ] BWA: Deleted primary data file. [2025-09-16T17:37:45.403481] [INFO ] [MAIN ] BWA: Starting full primary refresh. [2025-09-16T17:37:45.752133] [INFO ] [REMOTE ] BWA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:45.899435] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T17:37:45.974908] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-07. [2025-09-16T17:37:46.587563] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-09-16T17:37:46.774916] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-08. [2025-09-16T17:37:47.524432] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-09-16T17:37:47.574910] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-08. [2025-09-16T17:37:48.193742] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T17:37:48.374893] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-09. [2025-09-16T17:37:49.072412] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T17:37:49.174907] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-03. [2025-09-16T17:37:49.543176] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-09-16T17:37:49.549314] [INFO ] [REMOTE ] BV: Full fetch complete (42137 records). [2025-09-16T17:37:49.549424] [DEBUG] [VALIDATOR ] BV: Validating primary data. [2025-09-16T17:37:49.950279] [DEBUG] [VALIDATOR ] BV: All expected trading days are present. [2025-09-16T17:37:49.950379] [DEBUG] [VALIDATOR ] BV: Primary data validation passed. [2025-09-16T17:37:49.951040] [INFO ] [MAIN ] BV: Fetched primary data is valid. [2025-09-16T17:37:49.951088] [DEBUG] [LOCAL ] BV: Saving primary data. [2025-09-16T17:37:49.952267] [DEBUG] [LOCAL ] BV: Trimmed 42137 records to 28578 (keeping 150 days of data) [2025-09-16T17:37:50.012776] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-02. [2025-09-16T17:37:50.175584] [DEBUG] [LOCAL ] BV: Primary data saved (28578 records). [2025-09-16T17:37:50.175771] [INFO ] [MAIN ] BV: Full history saved. [2025-09-16T17:37:50.175865] [INFO ] [MAIN ] Update Progress: 362/2369 symbols completed. BV took 19.2s. Estimated remaining: 3h 29m 29s. Rate: 9.6 symbols/min [2025-09-16T17:37:50.409022] [INFO ] [MAIN ] BWG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:50.424848] [DEBUG] [LOCAL ] BWG: Loaded 9569 existing records from primary storage. [2025-09-16T17:37:50.425039] [INFO ] [MAIN ] BWG: Loaded 9569 existing records. [2025-09-16T17:37:50.700124] [WARN ] [MAIN ] BWG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:50.700592] [INFO ] [MAIN ] BWG: Wiping existing primary data before full refresh. [2025-09-16T17:37:50.700744] [INFO ] [LOCAL ] BWG: Starting deletion of primary data. [2025-09-16T17:37:50.701284] [DEBUG] [LOCAL ] BWG: Deleted primary data file. [2025-09-16T17:37:50.701470] [INFO ] [MAIN ] BWG: Starting full primary refresh. [2025-09-16T17:37:50.778589] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-09. [2025-09-16T17:37:51.078062] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-09-16T17:37:51.210803] [INFO ] [REMOTE ] BWG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:51.525834] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-09-16T17:37:51.574895] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-02. [2025-09-16T17:37:52.168244] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T17:37:52.180575] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-09-16T17:37:52.180794] [INFO ] [REMOTE ] BVN: Full fetch complete (47169 records). [2025-09-16T17:37:52.183604] [DEBUG] [VALIDATOR ] BVN: Validating primary data. [2025-09-16T17:37:52.380013] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-03. [2025-09-16T17:37:52.565276] [DEBUG] [VALIDATOR ] BVN: All expected trading days are present. [2025-09-16T17:37:52.565354] [DEBUG] [VALIDATOR ] BVN: Primary data validation passed. [2025-09-16T17:37:52.565940] [INFO ] [MAIN ] BVN: Fetched primary data is valid. [2025-09-16T17:37:52.565985] [DEBUG] [LOCAL ] BVN: Saving primary data. [2025-09-16T17:37:52.567053] [DEBUG] [LOCAL ] BVN: Trimmed 47169 records to 32229 (keeping 150 days of data) [2025-09-16T17:37:52.754019] [DEBUG] [LOCAL ] BVN: Primary data saved (32229 records). [2025-09-16T17:37:52.754163] [INFO ] [MAIN ] BVN: Full history saved. [2025-09-16T17:37:52.754247] [INFO ] [MAIN ] Update Progress: 363/2369 symbols completed. BVN took 21.8s. Estimated remaining: 3h 29m 3s. Rate: 9.6 symbols/min [2025-09-16T17:37:52.923040] [INFO ] [MAIN ] BWLP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:37:52.947830] [DEBUG] [LOCAL ] BWLP: Loaded 23730 existing records from primary storage. [2025-09-16T17:37:52.948011] [INFO ] [MAIN ] BWLP: Loaded 23730 existing records. [2025-09-16T17:37:53.123712] [WARN ] [MAIN ] BWLP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:37:53.123961] [INFO ] [MAIN ] BWLP: Wiping existing primary data before full refresh. [2025-09-16T17:37:53.124008] [INFO ] [LOCAL ] BWLP: Starting deletion of primary data. [2025-09-16T17:37:53.124673] [DEBUG] [LOCAL ] BWLP: Deleted primary data file. [2025-09-16T17:37:53.124727] [INFO ] [MAIN ] BWLP: Starting full primary refresh. [2025-09-16T17:37:53.180053] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-03. [2025-09-16T17:37:53.485285] [INFO ] [REMOTE ] BWLP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:37:53.959162] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-09-16T17:37:53.974843] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-02. [2025-09-16T17:37:54.262239] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-09-16T17:37:54.774920] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-04. [2025-09-16T17:37:55.290958] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-09-16T17:37:55.574920] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-04. [2025-09-16T17:37:56.374906] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-04. [2025-09-16T17:37:56.404460] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-09-16T17:37:56.550174] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-09-16T17:37:57.174942] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-05. [2025-09-16T17:37:57.740594] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T17:37:57.974915] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-05. [2025-09-16T17:37:58.093523] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-09-16T17:37:58.774912] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-06. [2025-09-16T17:37:59.183953] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-09-16T17:37:59.539988] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T17:37:59.574852] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-07. [2025-09-16T17:38:00.374893] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-03. [2025-09-16T17:38:00.538859] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-09-16T17:38:01.174908] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-05. [2025-09-16T17:38:01.974876] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-08. [2025-09-16T17:38:02.384965] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-09-16T17:38:02.676920] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-09-16T17:38:02.774904] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-06. [2025-09-16T17:38:02.903383] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T17:38:03.574936] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-09. [2025-09-16T17:38:04.220476] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T17:38:04.374899] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-04. [2025-09-16T17:38:04.536821] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T17:38:04.539091] [INFO ] [REMOTE ] BWG: Full fetch complete (13285 records). [2025-09-16T17:38:04.539160] [DEBUG] [VALIDATOR ] BWG: Validating primary data. [2025-09-16T17:38:04.896233] [DEBUG] [VALIDATOR ] BWG: All expected trading days are present. [2025-09-16T17:38:04.896314] [DEBUG] [VALIDATOR ] BWG: Primary data validation passed. [2025-09-16T17:38:04.896662] [INFO ] [MAIN ] BWG: Fetched primary data is valid. [2025-09-16T17:38:04.896707] [DEBUG] [LOCAL ] BWG: Saving primary data. [2025-09-16T17:38:04.897203] [DEBUG] [LOCAL ] BWG: Trimmed 13285 records to 9776 (keeping 150 days of data) [2025-09-16T17:38:04.954292] [DEBUG] [LOCAL ] BWG: Primary data saved (9776 records). [2025-09-16T17:38:04.954429] [INFO ] [MAIN ] BWG: Full history saved. [2025-09-16T17:38:04.954513] [INFO ] [MAIN ] Update Progress: 364/2369 symbols completed. BWG took 14.8s. Estimated remaining: 3h 29m 29s. Rate: 9.6 symbols/min [2025-09-16T17:38:05.124135] [INFO ] [MAIN ] BWMX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:38:05.124273] [DEBUG] [LOCAL ] BWMX: No existing data file found in primary storage. [2025-09-16T17:38:05.124313] [WARN ] [MAIN ] BWMX: No existing data found, performing full refresh instead. [2025-09-16T17:38:05.124434] [INFO ] [MAIN ] BWMX: Wiping existing primary data before full refresh. [2025-09-16T17:38:05.124475] [INFO ] [LOCAL ] BWMX: Starting deletion of primary data. [2025-09-16T17:38:05.124518] [DEBUG] [LOCAL ] BWMX: No primary data file found to delete. [2025-09-16T17:38:05.124552] [INFO ] [MAIN ] BWMX: Starting full primary refresh. [2025-09-16T17:38:05.180678] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-07. [2025-09-16T17:38:05.481065] [INFO ] [REMOTE ] BWMX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:38:05.737262] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-09-16T17:38:05.974888] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-05. [2025-09-16T17:38:06.331497] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T17:38:06.774913] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-02. [2025-09-16T17:38:07.327722] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-09-16T17:38:07.553188] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T17:38:07.574851] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-06. [2025-09-16T17:38:08.374935] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-06. [2025-09-16T17:38:08.875993] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:38:09.174900] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-08. [2025-09-16T17:38:09.950985] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T17:38:09.974856] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-07. [2025-09-16T17:38:10.762770] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T17:38:10.774861] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-09. [2025-09-16T17:38:11.311752] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-09-16T17:38:11.574937] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-07. [2025-09-16T17:38:12.147316] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T17:38:12.154068] [INFO ] [REMOTE ] BW: Full fetch complete (42990 records). [2025-09-16T17:38:12.154295] [DEBUG] [VALIDATOR ] BW: Validating primary data. [2025-09-16T17:38:12.386407] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-08. [2025-09-16T17:38:12.566629] [DEBUG] [VALIDATOR ] BW: All expected trading days are present. [2025-09-16T17:38:12.566776] [DEBUG] [VALIDATOR ] BW: Primary data validation passed. [2025-09-16T17:38:12.567583] [INFO ] [MAIN ] BW: Fetched primary data is valid. [2025-09-16T17:38:12.567668] [DEBUG] [LOCAL ] BW: Saving primary data. [2025-09-16T17:38:12.576411] [DEBUG] [LOCAL ] BW: Trimmed 42990 records to 28462 (keeping 150 days of data) [2025-09-16T17:38:12.588164] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-09-16T17:38:12.770149] [DEBUG] [LOCAL ] BW: Primary data saved (28462 records). [2025-09-16T17:38:12.770322] [INFO ] [MAIN ] BW: Full history saved. [2025-09-16T17:38:12.770428] [INFO ] [MAIN ] Update Progress: 365/2369 symbols completed. BW took 41.8s. Estimated remaining: 3h 29m 31s. Rate: 9.6 symbols/min [2025-09-16T17:38:12.940319] [INFO ] [MAIN ] BWNB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:38:12.945225] [DEBUG] [LOCAL ] BWNB: Loaded 3439 existing records from primary storage. [2025-09-16T17:38:12.945303] [INFO ] [MAIN ] BWNB: Loaded 3439 existing records. [2025-09-16T17:38:13.122556] [WARN ] [MAIN ] BWNB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:38:13.122838] [INFO ] [MAIN ] BWNB: Wiping existing primary data before full refresh. [2025-09-16T17:38:13.122885] [INFO ] [LOCAL ] BWNB: Starting deletion of primary data. [2025-09-16T17:38:13.123172] [DEBUG] [LOCAL ] BWNB: Deleted primary data file. [2025-09-16T17:38:13.123219] [INFO ] [MAIN ] BWNB: Starting full primary refresh. [2025-09-16T17:38:13.186195] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-03. [2025-09-16T17:38:13.481498] [INFO ] [REMOTE ] BWNB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:38:13.793465] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-09-16T17:38:13.974917] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-08. [2025-09-16T17:38:14.282161] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-09-16T17:38:14.774896] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-09. [2025-09-16T17:38:14.973888] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-09-16T17:38:15.574913] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-04. [2025-09-16T17:38:16.301685] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-09-16T17:38:16.374862] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-05. [2025-09-16T17:38:16.416877] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T17:38:16.425898] [INFO ] [REMOTE ] BWA: Full fetch complete (56849 records). [2025-09-16T17:38:16.426013] [DEBUG] [VALIDATOR ] BWA: Validating primary data. [2025-09-16T17:38:16.805085] [DEBUG] [VALIDATOR ] BWA: All expected trading days are present. [2025-09-16T17:38:16.805169] [DEBUG] [VALIDATOR ] BWA: Primary data validation passed. [2025-09-16T17:38:16.805897] [INFO ] [MAIN ] BWA: Fetched primary data is valid. [2025-09-16T17:38:16.805943] [DEBUG] [LOCAL ] BWA: Saving primary data. [2025-09-16T17:38:16.807135] [DEBUG] [LOCAL ] BWA: Trimmed 56849 records to 38799 (keeping 150 days of data) [2025-09-16T17:38:17.049722] [DEBUG] [LOCAL ] BWA: Primary data saved (38799 records). [2025-09-16T17:38:17.049880] [INFO ] [MAIN ] BWA: Full history saved. [2025-09-16T17:38:17.050943] [INFO ] [MAIN ] Update Progress: 366/2369 symbols completed. BWA took 32.0s. Estimated remaining: 3h 29m 14s. Rate: 9.6 symbols/min [2025-09-16T17:38:17.178434] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-09. [2025-09-16T17:38:17.206239] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T17:38:17.233385] [INFO ] [MAIN ] BWSN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:38:17.241882] [DEBUG] [LOCAL ] BWSN: Loaded 4084 existing records from primary storage. [2025-09-16T17:38:17.242015] [INFO ] [MAIN ] BWSN: Loaded 4084 existing records. [2025-09-16T17:38:17.419513] [WARN ] [MAIN ] BWSN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:38:17.419773] [INFO ] [MAIN ] BWSN: Wiping existing primary data before full refresh. [2025-09-16T17:38:17.419821] [INFO ] [LOCAL ] BWSN: Starting deletion of primary data. [2025-09-16T17:38:17.420092] [DEBUG] [LOCAL ] BWSN: Deleted primary data file. [2025-09-16T17:38:17.420137] [INFO ] [MAIN ] BWSN: Starting full primary refresh. [2025-09-16T17:38:17.775028] [INFO ] [REMOTE ] BWSN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:38:17.974917] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-02. [2025-09-16T17:38:18.673114] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T17:38:18.679296] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-09-16T17:38:18.680905] [INFO ] [REMOTE ] BWLP: Full fetch complete (35658 records). [2025-09-16T17:38:18.680979] [DEBUG] [VALIDATOR ] BWLP: Validating primary data. [2025-09-16T17:38:18.681498] [FATAL] [VALIDATOR ] BWLP: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:38:18.681575] [FATAL] [MAIN ] BWLP: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:38:18.681612] [WARN ] [MAIN ] BWLP: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:38:18.681672] [INFO ] [LOCAL ] BWLP: Starting deletion of primary data. [2025-09-16T17:38:18.681732] [DEBUG] [LOCAL ] BWLP: No primary data file found to delete. [2025-09-16T17:38:18.681773] [INFO ] [LOCAL ] BWLP: Starting deletion of derivative data. [2025-09-16T17:38:18.682094] [DEBUG] [LOCAL ] BWLP: Deleted derivative data file. [2025-09-16T17:38:18.682173] [INFO ] [MAIN ] Update Progress: 367/2369 symbols completed. BWLP took 25.9s. Estimated remaining: 3h 28m 43s. Rate: 9.6 symbols/min [2025-09-16T17:38:18.780014] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-06. [2025-09-16T17:38:18.851359] [INFO ] [MAIN ] BWXT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:38:18.886157] [DEBUG] [LOCAL ] BWXT: Loaded 34580 existing records from primary storage. [2025-09-16T17:38:18.886301] [INFO ] [MAIN ] BWXT: Loaded 34580 existing records. [2025-09-16T17:38:19.061706] [WARN ] [MAIN ] BWXT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:38:19.061959] [INFO ] [MAIN ] BWXT: Wiping existing primary data before full refresh. [2025-09-16T17:38:19.062013] [INFO ] [LOCAL ] BWXT: Starting deletion of primary data. [2025-09-16T17:38:19.062988] [DEBUG] [LOCAL ] BWXT: Deleted primary data file. [2025-09-16T17:38:19.063044] [INFO ] [MAIN ] BWXT: Starting full primary refresh. [2025-09-16T17:38:19.421090] [INFO ] [REMOTE ] BWXT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:38:19.566473] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-09-16T17:38:19.574836] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-07. [2025-09-16T17:38:20.374880] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-02. [2025-09-16T17:38:20.409225] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-09-16T17:38:21.174902] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-08. [2025-09-16T17:38:21.896108] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-09-16T17:38:21.974862] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-09. [2025-09-16T17:38:22.072504] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-09-16T17:38:22.695313] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-09-16T17:38:22.696574] [INFO ] [REMOTE ] BWMX: Full fetch complete (7846 records). [2025-09-16T17:38:22.696659] [DEBUG] [VALIDATOR ] BWMX: Validating primary data. [2025-09-16T17:38:22.697020] [FATAL] [VALIDATOR ] BWMX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:38:22.697134] [FATAL] [MAIN ] BWMX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:38:22.697171] [WARN ] [MAIN ] BWMX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:38:22.697208] [INFO ] [LOCAL ] BWMX: Starting deletion of primary data. [2025-09-16T17:38:22.697272] [DEBUG] [LOCAL ] BWMX: No primary data file found to delete. [2025-09-16T17:38:22.697309] [INFO ] [LOCAL ] BWMX: Starting deletion of derivative data. [2025-09-16T17:38:22.697351] [DEBUG] [LOCAL ] BWMX: No derivative data file found to delete. [2025-09-16T17:38:22.697408] [INFO ] [MAIN ] Update Progress: 368/2369 symbols completed. BWMX took 17.7s. Estimated remaining: 3h 28m 24s. Rate: 9.6 symbols/min [2025-09-16T17:38:22.780016] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-02. [2025-09-16T17:38:22.868501] [INFO ] [MAIN ] BX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:38:22.908723] [DEBUG] [LOCAL ] BX: Loaded 39477 existing records from primary storage. [2025-09-16T17:38:22.908859] [INFO ] [MAIN ] BX: Loaded 39477 existing records. [2025-09-16T17:38:23.084666] [WARN ] [MAIN ] BX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:38:23.084887] [INFO ] [MAIN ] BX: Wiping existing primary data before full refresh. [2025-09-16T17:38:23.084974] [INFO ] [LOCAL ] BX: Starting deletion of primary data. [2025-09-16T17:38:23.085977] [DEBUG] [LOCAL ] BX: Deleted primary data file. [2025-09-16T17:38:23.086028] [INFO ] [MAIN ] BX: Starting full primary refresh. [2025-09-16T17:38:23.448993] [INFO ] [REMOTE ] BX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:38:23.452170] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-09-16T17:38:23.574930] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-02. [2025-09-16T17:38:24.374945] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-03. [2025-09-16T17:38:25.177368] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-03. [2025-09-16T17:38:25.183108] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T17:38:25.596060] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-09-16T17:38:25.707534] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T17:38:25.974937] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-04. [2025-09-16T17:38:26.395684] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-09-16T17:38:26.774911] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-05. [2025-09-16T17:38:27.275923] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-09-16T17:38:27.574913] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-06. [2025-09-16T17:38:27.950159] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-09-16T17:38:28.374919] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-03. [2025-09-16T17:38:28.900985] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-09-16T17:38:29.174908] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-07. [2025-09-16T17:38:29.468459] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-07 (0.29 seconds). [2025-09-16T17:38:29.974914] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-04. [2025-09-16T17:38:30.774931] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-04. [2025-09-16T17:38:31.108956] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-09-16T17:38:31.259279] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-09-16T17:38:31.574896] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-05. [2025-09-16T17:38:32.374928] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-03. [2025-09-16T17:38:33.174924] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-05. [2025-09-16T17:38:33.304448] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T17:38:33.635705] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-09-16T17:38:33.934277] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T17:38:33.974861] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-04. [2025-09-16T17:38:34.774920] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-06. [2025-09-16T17:38:35.574898] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-06. [2025-09-16T17:38:35.789173] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T17:38:35.964347] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-09-16T17:38:36.156677] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T17:38:36.374904] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-07. [2025-09-16T17:38:37.174902] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-05. [2025-09-16T17:38:37.944788] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T17:38:37.974874] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-08. [2025-09-16T17:38:38.720538] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-09-16T17:38:38.774918] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-06. [2025-09-16T17:38:39.574898] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-08. [2025-09-16T17:38:39.643693] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T17:38:40.094707] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-09-16T17:38:40.286920] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-09-16T17:38:40.374905] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-07. [2025-09-16T17:38:41.174943] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-07. [2025-09-16T17:38:41.619326] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-09-16T17:38:41.948851] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T17:38:41.974870] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-08. [2025-09-16T17:38:42.774982] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-09. [2025-09-16T17:38:43.248009] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-09-16T17:38:43.249307] [INFO ] [REMOTE ] BWSN: Full fetch complete (5350 records). [2025-09-16T17:38:43.249393] [DEBUG] [VALIDATOR ] BWSN: Validating primary data. [2025-09-16T17:38:43.580278] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-08. [2025-09-16T17:38:43.901612] [DEBUG] [VALIDATOR ] BWSN: All expected trading days are present. [2025-09-16T17:38:43.901768] [DEBUG] [VALIDATOR ] BWSN: Primary data validation passed. [2025-09-16T17:38:43.902093] [INFO ] [MAIN ] BWSN: Fetched primary data is valid. [2025-09-16T17:38:43.902197] [DEBUG] [LOCAL ] BWSN: Saving primary data. [2025-09-16T17:38:43.903602] [DEBUG] [LOCAL ] BWSN: Trimmed 5350 records to 4093 (keeping 150 days of data) [2025-09-16T17:38:43.983177] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-09-16T17:38:44.004327] [DEBUG] [LOCAL ] BWSN: Primary data saved (4093 records). [2025-09-16T17:38:44.004501] [INFO ] [MAIN ] BWSN: Full history saved. [2025-09-16T17:38:44.004604] [INFO ] [MAIN ] Update Progress: 369/2369 symbols completed. BWSN took 27.0s. Estimated remaining: 3h 29m 39s. Rate: 9.5 symbols/min [2025-09-16T17:38:44.157770] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-09-16T17:38:44.233535] [INFO ] [MAIN ] BXC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:38:44.245371] [DEBUG] [LOCAL ] BXC: Loaded 10082 existing records from primary storage. [2025-09-16T17:38:44.245524] [INFO ] [MAIN ] BXC: Loaded 10082 existing records. [2025-09-16T17:38:44.380138] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-09. [2025-09-16T17:38:44.440522] [WARN ] [MAIN ] BXC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:38:44.440810] [INFO ] [MAIN ] BXC: Wiping existing primary data before full refresh. [2025-09-16T17:38:44.440863] [INFO ] [LOCAL ] BXC: Starting deletion of primary data. [2025-09-16T17:38:44.441400] [DEBUG] [LOCAL ] BXC: Deleted primary data file. [2025-09-16T17:38:44.441453] [INFO ] [MAIN ] BXC: Starting full primary refresh. [2025-09-16T17:38:44.788935] [INFO ] [REMOTE ] BXC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:38:45.174913] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-09. [2025-09-16T17:38:45.974926] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-02. [2025-09-16T17:38:46.054079] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-09-16T17:38:46.059597] [INFO ] [REMOTE ] BWXT: Full fetch complete (50857 records). [2025-09-16T17:38:46.059725] [DEBUG] [VALIDATOR ] BWXT: Validating primary data. [2025-09-16T17:38:46.438864] [DEBUG] [VALIDATOR ] BWXT: All expected trading days are present. [2025-09-16T17:38:46.438949] [DEBUG] [VALIDATOR ] BWXT: Primary data validation passed. [2025-09-16T17:38:46.439541] [INFO ] [MAIN ] BWXT: Fetched primary data is valid. [2025-09-16T17:38:46.439585] [DEBUG] [LOCAL ] BWXT: Saving primary data. [2025-09-16T17:38:46.440614] [DEBUG] [LOCAL ] BWXT: Trimmed 50857 records to 34954 (keeping 150 days of data) [2025-09-16T17:38:46.661169] [DEBUG] [LOCAL ] BWXT: Primary data saved (34954 records). [2025-09-16T17:38:46.661306] [INFO ] [MAIN ] BWXT: Full history saved. [2025-09-16T17:38:46.661389] [INFO ] [MAIN ] Update Progress: 370/2369 symbols completed. BWXT took 28.0s. Estimated remaining: 3h 29m 14s. Rate: 9.6 symbols/min [2025-09-16T17:38:46.785318] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-09. [2025-09-16T17:38:46.844178] [INFO ] [MAIN ] BXMT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:38:46.844746] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T17:38:46.867601] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-09-16T17:38:46.874258] [INFO ] [REMOTE ] BX: Full fetch complete (58175 records). [2025-09-16T17:38:46.874440] [DEBUG] [VALIDATOR ] BX: Validating primary data. [2025-09-16T17:38:47.059623] [DEBUG] [LOCAL ] BXMT: Loaded 33962 existing records from primary storage. [2025-09-16T17:38:47.075182] [INFO ] [MAIN ] BXMT: Loaded 33962 existing records. [2025-09-16T17:38:47.475074] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-09-16T17:38:47.474890] [WARN ] [MAIN ] BXMT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:38:47.479751] [DEBUG] [VALIDATOR ] BX: All expected trading days are present. [2025-09-16T17:38:47.479957] [INFO ] [REMOTE ] BWNB: Full fetch complete (4808 records). [2025-09-16T17:38:47.480306] [DEBUG] [VALIDATOR ] BWNB: Validating primary data. [2025-09-16T17:38:47.480181] [INFO ] [MAIN ] BXMT: Wiping existing primary data before full refresh. [2025-09-16T17:38:47.480581] [INFO ] [LOCAL ] BXMT: Starting deletion of primary data. [2025-09-16T17:38:47.480248] [DEBUG] [VALIDATOR ] BX: Primary data validation passed. [2025-09-16T17:38:47.502151] [DEBUG] [LOCAL ] BXMT: Deleted primary data file. [2025-09-16T17:38:47.502116] [INFO ] [MAIN ] BX: Fetched primary data is valid. [2025-09-16T17:38:47.512381] [INFO ] [MAIN ] BXMT: Starting full primary refresh. [2025-09-16T17:38:47.529572] [DEBUG] [LOCAL ] BX: Saving primary data. [2025-09-16T17:38:47.592853] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-03. [2025-09-16T17:38:47.712042] [DEBUG] [LOCAL ] BX: Trimmed 58175 records to 39866 (keeping 150 days of data) [2025-09-16T17:38:48.311518] [INFO ] [REMOTE ] BXMT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:38:48.311964] [DEBUG] [VALIDATOR ] BWNB: All expected trading days are present. [2025-09-16T17:38:48.313835] [DEBUG] [VALIDATOR ] BWNB: Primary data validation passed. [2025-09-16T17:38:48.314699] [INFO ] [MAIN ] BWNB: Fetched primary data is valid. [2025-09-16T17:38:48.316677] [DEBUG] [LOCAL ] BWNB: Saving primary data. [2025-09-16T17:38:48.324447] [DEBUG] [LOCAL ] BWNB: Trimmed 4808 records to 3471 (keeping 150 days of data) [2025-09-16T17:38:48.421249] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-02. [2025-09-16T17:38:48.421159] [DEBUG] [LOCAL ] BWNB: Primary data saved (3471 records). [2025-09-16T17:38:48.431145] [INFO ] [MAIN ] BWNB: Full history saved. [2025-09-16T17:38:48.431606] [INFO ] [MAIN ] Update Progress: 371/2369 symbols completed. BWNB took 35.7s. Estimated remaining: 3h 28m 43s. Rate: 9.6 symbols/min [2025-09-16T17:38:48.649911] [INFO ] [MAIN ] BXMX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:38:48.670357] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T17:38:48.682007] [DEBUG] [LOCAL ] BXMX: Loaded 13325 existing records from primary storage. [2025-09-16T17:38:48.683788] [INFO ] [MAIN ] BXMX: Loaded 13325 existing records. [2025-09-16T17:38:48.869951] [WARN ] [MAIN ] BXMX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:38:48.871207] [INFO ] [MAIN ] BXMX: Wiping existing primary data before full refresh. [2025-09-16T17:38:48.873870] [INFO ] [LOCAL ] BXMX: Starting deletion of primary data. [2025-09-16T17:38:48.873926] [DEBUG] [LOCAL ] BX: Primary data saved (39866 records). [2025-09-16T17:38:48.874035] [INFO ] [MAIN ] BX: Full history saved. [2025-09-16T17:38:48.875435] [INFO ] [MAIN ] Update Progress: 372/2369 symbols completed. BX took 26.2s. Estimated remaining: 3h 28m 5s. Rate: 9.6 symbols/min [2025-09-16T17:38:48.881413] [DEBUG] [LOCAL ] BXMX: Deleted primary data file. [2025-09-16T17:38:48.918902] [INFO ] [MAIN ] BXMX: Starting full primary refresh. [2025-09-16T17:38:49.186872] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-04. [2025-09-16T17:38:49.191964] [INFO ] [MAIN ] BXP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:38:49.405329] [INFO ] [REMOTE ] BXMX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:38:49.428128] [DEBUG] [LOCAL ] BXP: Loaded 36565 existing records from primary storage. [2025-09-16T17:38:49.428229] [INFO ] [MAIN ] BXP: Loaded 36565 existing records. [2025-09-16T17:38:49.603335] [WARN ] [MAIN ] BXP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:38:49.603562] [INFO ] [MAIN ] BXP: Wiping existing primary data before full refresh. [2025-09-16T17:38:49.603617] [INFO ] [LOCAL ] BXP: Starting deletion of primary data. [2025-09-16T17:38:49.604513] [DEBUG] [LOCAL ] BXP: Deleted primary data file. [2025-09-16T17:38:49.604561] [INFO ] [MAIN ] BXP: Starting full primary refresh. [2025-09-16T17:38:49.971420] [INFO ] [REMOTE ] BXP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:38:49.982491] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-02. [2025-09-16T17:38:50.000568] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T17:38:50.070586] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-09-16T17:38:50.774943] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-03. [2025-09-16T17:38:51.165118] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-09-16T17:38:51.574906] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-02. [2025-09-16T17:38:52.380056] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-03. [2025-09-16T17:38:52.395353] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T17:38:53.146465] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T17:38:53.174840] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-03. [2025-09-16T17:38:53.508694] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-09-16T17:38:53.974910] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-05. [2025-09-16T17:38:54.703866] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T17:38:54.757282] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-09-16T17:38:54.774857] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-06. [2025-09-16T17:38:55.570468] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-09-16T17:38:55.574860] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-04. [2025-09-16T17:38:56.374900] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-07. [2025-09-16T17:38:57.108258] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-09-16T17:38:57.174868] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-04. [2025-09-16T17:38:57.395108] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T17:38:57.974903] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-05. [2025-09-16T17:38:58.744754] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-09-16T17:38:58.774843] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-05. [2025-09-16T17:38:59.506574] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T17:38:59.574869] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-06. [2025-09-16T17:39:00.374880] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-04. [2025-09-16T17:39:00.441052] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-09-16T17:39:01.174909] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-06. [2025-09-16T17:39:01.500849] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-09-16T17:39:01.974915] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-07. [2025-09-16T17:39:02.107765] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T17:39:02.639751] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T17:39:02.774875] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-07. [2025-09-16T17:39:03.574925] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-08. [2025-09-16T17:39:03.649794] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-09-16T17:39:04.185357] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-09-16T17:39:04.374898] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-08. [2025-09-16T17:39:04.499097] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-09-16T17:39:05.174903] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-09. [2025-09-16T17:39:05.922615] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T17:39:05.974894] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-05. [2025-09-16T17:39:06.084049] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-09-16T17:39:06.085921] [INFO ] [REMOTE ] BXC: Full fetch complete (14604 records). [2025-09-16T17:39:06.086024] [DEBUG] [VALIDATOR ] BXC: Validating primary data. [2025-09-16T17:39:06.455917] [DEBUG] [VALIDATOR ] BXC: All expected trading days are present. [2025-09-16T17:39:06.455999] [DEBUG] [VALIDATOR ] BXC: Primary data validation passed. [2025-09-16T17:39:06.456329] [INFO ] [MAIN ] BXC: Fetched primary data is valid. [2025-09-16T17:39:06.456382] [DEBUG] [LOCAL ] BXC: Saving primary data. [2025-09-16T17:39:06.456956] [DEBUG] [LOCAL ] BXC: Trimmed 14604 records to 10137 (keeping 150 days of data) [2025-09-16T17:39:06.517898] [DEBUG] [LOCAL ] BXC: Primary data saved (10137 records). [2025-09-16T17:39:06.518049] [INFO ] [MAIN ] BXC: Full history saved. [2025-09-16T17:39:06.518154] [INFO ] [MAIN ] Update Progress: 373/2369 symbols completed. BXC took 22.5s. Estimated remaining: 3h 29m 0s. Rate: 9.6 symbols/min [2025-09-16T17:39:06.728964] [INFO ] [MAIN ] BXSL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:39:06.770868] [DEBUG] [LOCAL ] BXSL: Loaded 34368 existing records from primary storage. [2025-09-16T17:39:06.771138] [INFO ] [MAIN ] BXSL: Loaded 34368 existing records. [2025-09-16T17:39:06.782368] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-09. [2025-09-16T17:39:06.957190] [WARN ] [MAIN ] BXSL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:39:06.969957] [INFO ] [MAIN ] BXSL: Wiping existing primary data before full refresh. [2025-09-16T17:39:06.970551] [INFO ] [LOCAL ] BXSL: Starting deletion of primary data. [2025-09-16T17:39:06.971039] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T17:39:06.976156] [DEBUG] [LOCAL ] BXSL: Deleted primary data file. [2025-09-16T17:39:06.976258] [INFO ] [MAIN ] BXSL: Starting full primary refresh. [2025-09-16T17:39:07.332434] [INFO ] [REMOTE ] BXSL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:39:07.574955] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-02. [2025-09-16T17:39:08.314959] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-09-16T17:39:08.320324] [INFO ] [REMOTE ] BXMT: Full fetch complete (51728 records). [2025-09-16T17:39:08.320438] [DEBUG] [VALIDATOR ] BXMT: Validating primary data. [2025-09-16T17:39:08.384081] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-08. [2025-09-16T17:39:08.707449] [DEBUG] [VALIDATOR ] BXMT: All expected trading days are present. [2025-09-16T17:39:08.707550] [DEBUG] [VALIDATOR ] BXMT: Primary data validation passed. [2025-09-16T17:39:08.708239] [INFO ] [MAIN ] BXMT: Fetched primary data is valid. [2025-09-16T17:39:08.708287] [DEBUG] [LOCAL ] BXMT: Saving primary data. [2025-09-16T17:39:08.710057] [DEBUG] [LOCAL ] BXMT: Trimmed 51728 records to 34285 (keeping 150 days of data) [2025-09-16T17:39:08.916261] [DEBUG] [LOCAL ] BXMT: Primary data saved (34285 records). [2025-09-16T17:39:08.916414] [INFO ] [MAIN ] BXMT: Full history saved. [2025-09-16T17:39:08.916512] [INFO ] [MAIN ] Update Progress: 374/2369 symbols completed. BXMT took 22.3s. Estimated remaining: 3h 28m 33s. Rate: 9.6 symbols/min [2025-09-16T17:39:09.087453] [INFO ] [MAIN ] BY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:39:09.103178] [DEBUG] [LOCAL ] BY: Loaded 15598 existing records from primary storage. [2025-09-16T17:39:09.103293] [INFO ] [MAIN ] BY: Loaded 15598 existing records. [2025-09-16T17:39:09.178546] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-06. [2025-09-16T17:39:09.282581] [WARN ] [MAIN ] BY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:39:09.283343] [INFO ] [MAIN ] BY: Wiping existing primary data before full refresh. [2025-09-16T17:39:09.283484] [INFO ] [LOCAL ] BY: Starting deletion of primary data. [2025-09-16T17:39:09.284039] [DEBUG] [LOCAL ] BY: Deleted primary data file. [2025-09-16T17:39:09.284103] [INFO ] [MAIN ] BY: Starting full primary refresh. [2025-09-16T17:39:09.542765] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-09-16T17:39:09.691088] [INFO ] [REMOTE ] BY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:39:09.980109] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-02. [2025-09-16T17:39:09.998423] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-09-16T17:39:10.159366] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T17:39:10.774923] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-09. [2025-09-16T17:39:11.080330] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T17:39:11.574937] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-07. [2025-09-16T17:39:12.354599] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-09-16T17:39:12.361401] [INFO ] [REMOTE ] BXP: Full fetch complete (54166 records). [2025-09-16T17:39:12.361565] [DEBUG] [VALIDATOR ] BXP: Validating primary data. [2025-09-16T17:39:12.383595] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-03. [2025-09-16T17:39:12.756542] [DEBUG] [VALIDATOR ] BXP: All expected trading days are present. [2025-09-16T17:39:12.756630] [DEBUG] [VALIDATOR ] BXP: Primary data validation passed. [2025-09-16T17:39:12.757279] [INFO ] [MAIN ] BXP: Fetched primary data is valid. [2025-09-16T17:39:12.757322] [DEBUG] [LOCAL ] BXP: Saving primary data. [2025-09-16T17:39:12.758894] [DEBUG] [LOCAL ] BXP: Trimmed 54166 records to 36936 (keeping 150 days of data) [2025-09-16T17:39:12.836094] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T17:39:12.997584] [DEBUG] [LOCAL ] BXP: Primary data saved (36936 records). [2025-09-16T17:39:12.997740] [INFO ] [MAIN ] BXP: Full history saved. [2025-09-16T17:39:12.997825] [INFO ] [MAIN ] Update Progress: 375/2369 symbols completed. BXP took 24.1s. Estimated remaining: 3h 28m 15s. Rate: 9.6 symbols/min [2025-09-16T17:39:13.168681] [INFO ] [MAIN ] BYD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:39:13.174878] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-03. [2025-09-16T17:39:13.208225] [DEBUG] [LOCAL ] BYD: Loaded 33316 existing records from primary storage. [2025-09-16T17:39:13.208412] [INFO ] [MAIN ] BYD: Loaded 33316 existing records. [2025-09-16T17:39:13.393760] [WARN ] [MAIN ] BYD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:39:13.394171] [INFO ] [MAIN ] BYD: Wiping existing primary data before full refresh. [2025-09-16T17:39:13.394303] [INFO ] [LOCAL ] BYD: Starting deletion of primary data. [2025-09-16T17:39:13.400571] [DEBUG] [LOCAL ] BYD: Deleted primary data file. [2025-09-16T17:39:13.405603] [INFO ] [MAIN ] BYD: Starting full primary refresh. [2025-09-16T17:39:13.406027] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-09-16T17:39:13.761113] [INFO ] [REMOTE ] BYD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:39:13.974874] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-02. [2025-09-16T17:39:14.568630] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T17:39:14.774960] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-04. [2025-09-16T17:39:15.281252] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T17:39:15.574922] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-03. [2025-09-16T17:39:16.374874] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-04. [2025-09-16T17:39:16.445855] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T17:39:17.174867] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-08. [2025-09-16T17:39:17.271357] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-09-16T17:39:17.520296] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T17:39:17.974902] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-05. [2025-09-16T17:39:17.996935] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-09-16T17:39:18.774904] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-05. [2025-09-16T17:39:19.574929] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-04. [2025-09-16T17:39:19.813189] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T17:39:19.865150] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-09-16T17:39:20.374916] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-09. [2025-09-16T17:39:20.853555] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-09-16T17:39:21.174927] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-06. [2025-09-16T17:39:21.389406] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T17:39:21.391704] [INFO ] [REMOTE ] BXMX: Full fetch complete (21288 records). [2025-09-16T17:39:21.391778] [DEBUG] [VALIDATOR ] BXMX: Validating primary data. [2025-09-16T17:39:21.760670] [DEBUG] [VALIDATOR ] BXMX: All expected trading days are present. [2025-09-16T17:39:21.760749] [DEBUG] [VALIDATOR ] BXMX: Primary data validation passed. [2025-09-16T17:39:21.761115] [INFO ] [MAIN ] BXMX: Fetched primary data is valid. [2025-09-16T17:39:21.761155] [DEBUG] [LOCAL ] BXMX: Saving primary data. [2025-09-16T17:39:21.761757] [DEBUG] [LOCAL ] BXMX: Trimmed 21288 records to 13505 (keeping 150 days of data) [2025-09-16T17:39:21.846548] [DEBUG] [LOCAL ] BXMX: Primary data saved (13505 records). [2025-09-16T17:39:21.846726] [INFO ] [MAIN ] BXMX: Full history saved. [2025-09-16T17:39:21.846815] [INFO ] [MAIN ] Update Progress: 376/2369 symbols completed. BXMX took 33.4s. Estimated remaining: 3h 28m 23s. Rate: 9.6 symbols/min [2025-09-16T17:39:21.980028] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-06. [2025-09-16T17:39:22.047137] [INFO ] [MAIN ] BYM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:39:22.056290] [DEBUG] [LOCAL ] BYM: Loaded 8410 existing records from primary storage. [2025-09-16T17:39:22.056373] [INFO ] [MAIN ] BYM: Loaded 8410 existing records. [2025-09-16T17:39:22.233461] [WARN ] [MAIN ] BYM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:39:22.233720] [INFO ] [MAIN ] BYM: Wiping existing primary data before full refresh. [2025-09-16T17:39:22.233796] [INFO ] [LOCAL ] BYM: Starting deletion of primary data. [2025-09-16T17:39:22.234117] [DEBUG] [LOCAL ] BYM: Deleted primary data file. [2025-09-16T17:39:22.234160] [INFO ] [MAIN ] BYM: Starting full primary refresh. [2025-09-16T17:39:22.603490] [INFO ] [REMOTE ] BYM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:39:22.619816] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T17:39:22.774859] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-02. [2025-09-16T17:39:23.325296] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T17:39:23.574867] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-07. [2025-09-16T17:39:23.663141] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-09-16T17:39:24.374936] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-07. [2025-09-16T17:39:25.176479] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-03. [2025-09-16T17:39:25.179592] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T17:39:25.530369] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-09-16T17:39:25.974913] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-08. [2025-09-16T17:39:26.154368] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T17:39:26.774895] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-08. [2025-09-16T17:39:27.020596] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T17:39:27.574933] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-04. [2025-09-16T17:39:28.227779] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T17:39:28.374909] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-05. [2025-09-16T17:39:28.544620] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-09-16T17:39:29.174959] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-09. [2025-09-16T17:39:29.974861] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-05. [2025-09-16T17:39:30.217061] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T17:39:30.226317] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T17:39:30.234620] [INFO ] [REMOTE ] BY: Full fetch complete (22442 records). [2025-09-16T17:39:30.234991] [DEBUG] [VALIDATOR ] BY: Validating primary data. [2025-09-16T17:39:30.235489] [FATAL] [VALIDATOR ] BY: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:39:30.235585] [FATAL] [MAIN ] BY: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:39:30.235625] [WARN ] [MAIN ] BY: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:39:30.235688] [INFO ] [LOCAL ] BY: Starting deletion of primary data. [2025-09-16T17:39:30.235750] [DEBUG] [LOCAL ] BY: No primary data file found to delete. [2025-09-16T17:39:30.235787] [INFO ] [LOCAL ] BY: Starting deletion of derivative data. [2025-09-16T17:39:30.236033] [DEBUG] [LOCAL ] BY: Deleted derivative data file. [2025-09-16T17:39:30.236113] [INFO ] [MAIN ] Update Progress: 377/2369 symbols completed. BY took 21.3s. Estimated remaining: 3h 28m 28s. Rate: 9.6 symbols/min [2025-09-16T17:39:30.406899] [INFO ] [MAIN ] BZH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:39:30.430282] [DEBUG] [LOCAL ] BZH: Loaded 24078 existing records from primary storage. [2025-09-16T17:39:30.430431] [INFO ] [MAIN ] BZH: Loaded 24078 existing records. [2025-09-16T17:39:30.617559] [WARN ] [MAIN ] BZH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:39:30.617818] [INFO ] [MAIN ] BZH: Wiping existing primary data before full refresh. [2025-09-16T17:39:30.617885] [INFO ] [LOCAL ] BZH: Starting deletion of primary data. [2025-09-16T17:39:30.618495] [DEBUG] [LOCAL ] BZH: Deleted primary data file. [2025-09-16T17:39:30.618552] [INFO ] [MAIN ] BZH: Starting full primary refresh. [2025-09-16T17:39:30.790372] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-09. [2025-09-16T17:39:30.948617] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T17:39:30.987798] [INFO ] [REMOTE ] BZH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:39:31.574949] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-06. [2025-09-16T17:39:32.374903] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-02. [2025-09-16T17:39:32.606015] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-09-16T17:39:32.612250] [INFO ] [REMOTE ] BXSL: Full fetch complete (51506 records). [2025-09-16T17:39:32.612440] [DEBUG] [VALIDATOR ] BXSL: Validating primary data. [2025-09-16T17:39:33.001825] [DEBUG] [VALIDATOR ] BXSL: All expected trading days are present. [2025-09-16T17:39:33.001907] [DEBUG] [VALIDATOR ] BXSL: Primary data validation passed. [2025-09-16T17:39:33.002504] [INFO ] [MAIN ] BXSL: Fetched primary data is valid. [2025-09-16T17:39:33.002547] [DEBUG] [LOCAL ] BXSL: Saving primary data. [2025-09-16T17:39:33.003668] [DEBUG] [LOCAL ] BXSL: Trimmed 51506 records to 34756 (keeping 150 days of data) [2025-09-16T17:39:33.190316] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-06. [2025-09-16T17:39:33.216560] [DEBUG] [LOCAL ] BXSL: Primary data saved (34756 records). [2025-09-16T17:39:33.216944] [INFO ] [MAIN ] BXSL: Full history saved. [2025-09-16T17:39:33.217604] [INFO ] [MAIN ] Update Progress: 378/2369 symbols completed. BXSL took 26.7s. Estimated remaining: 3h 28m 4s. Rate: 9.6 symbols/min [2025-09-16T17:39:33.395905] [INFO ] [MAIN ] C: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:39:33.424051] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-09-16T17:39:33.472009] [DEBUG] [LOCAL ] C: Loaded 39702 existing records from primary storage. [2025-09-16T17:39:33.472161] [INFO ] [MAIN ] C: Loaded 39702 existing records. [2025-09-16T17:39:33.648708] [WARN ] [MAIN ] C: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:39:33.648929] [INFO ] [MAIN ] C: Wiping existing primary data before full refresh. [2025-09-16T17:39:33.648975] [INFO ] [LOCAL ] C: Starting deletion of primary data. [2025-09-16T17:39:33.649833] [DEBUG] [LOCAL ] C: Deleted primary data file. [2025-09-16T17:39:33.649881] [INFO ] [MAIN ] C: Starting full primary refresh. [2025-09-16T17:39:34.012752] [INFO ] [REMOTE ] C: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:39:33.975708] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-07. [2025-09-16T17:39:34.063307] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T17:39:34.088762] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-09-16T17:39:34.774916] [INFO ] [REMOTE ] C: Fetching primary data for 2025-02. [2025-09-16T17:39:35.406231] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-09-16T17:39:35.574934] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-03. [2025-09-16T17:39:36.374971] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-07. [2025-09-16T17:39:36.722482] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-09-16T17:39:36.830560] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T17:39:37.149096] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T17:39:37.174857] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-04. [2025-09-16T17:39:37.974905] [INFO ] [REMOTE ] C: Fetching primary data for 2025-03. [2025-09-16T17:39:38.607291] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T17:39:38.774910] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-08. [2025-09-16T17:39:39.574903] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-08. [2025-09-16T17:39:39.874820] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-09-16T17:39:40.179029] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-09-16T17:39:40.288396] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-09-16T17:39:40.374919] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-09. [2025-09-16T17:39:41.174908] [INFO ] [REMOTE ] C: Fetching primary data for 2025-04. [2025-09-16T17:39:41.673306] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T17:39:41.678798] [INFO ] [REMOTE ] BYD: Full fetch complete (49234 records). [2025-09-16T17:39:41.678895] [DEBUG] [VALIDATOR ] BYD: Validating primary data. [2025-09-16T17:39:41.979953] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-09. [2025-09-16T17:39:42.063627] [DEBUG] [VALIDATOR ] BYD: All expected trading days are present. [2025-09-16T17:39:42.063727] [DEBUG] [VALIDATOR ] BYD: Primary data validation passed. [2025-09-16T17:39:42.064381] [INFO ] [MAIN ] BYD: Fetched primary data is valid. [2025-09-16T17:39:42.064427] [DEBUG] [LOCAL ] BYD: Saving primary data. [2025-09-16T17:39:42.065532] [DEBUG] [LOCAL ] BYD: Trimmed 49234 records to 33670 (keeping 150 days of data) [2025-09-16T17:39:42.283249] [DEBUG] [LOCAL ] BYD: Primary data saved (33670 records). [2025-09-16T17:39:42.283414] [INFO ] [MAIN ] BYD: Full history saved. [2025-09-16T17:39:42.283522] [INFO ] [MAIN ] Update Progress: 379/2369 symbols completed. BYD took 29.3s. Estimated remaining: 3h 28m 12s. Rate: 9.6 symbols/min [2025-09-16T17:39:42.454111] [INFO ] [MAIN ] CAAP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:39:42.468244] [DEBUG] [LOCAL ] CAAP: Loaded 13239 existing records from primary storage. [2025-09-16T17:39:42.468375] [INFO ] [MAIN ] CAAP: Loaded 13239 existing records. [2025-09-16T17:39:42.649493] [WARN ] [MAIN ] CAAP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:39:42.649742] [INFO ] [MAIN ] CAAP: Wiping existing primary data before full refresh. [2025-09-16T17:39:42.649819] [INFO ] [LOCAL ] CAAP: Starting deletion of primary data. [2025-09-16T17:39:42.650267] [DEBUG] [LOCAL ] CAAP: Deleted primary data file. [2025-09-16T17:39:42.650337] [INFO ] [MAIN ] CAAP: Starting full primary refresh. [2025-09-16T17:39:42.780000] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-05. [2025-09-16T17:39:42.965897] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T17:39:43.000553] [INFO ] [REMOTE ] BYM: Full fetch complete (13107 records). [2025-09-16T17:39:43.021410] [DEBUG] [VALIDATOR ] BYM: Validating primary data. [2025-09-16T17:39:43.021339] [INFO ] [REMOTE ] CAAP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:39:43.378041] [DEBUG] [VALIDATOR ] BYM: All expected trading days are present. [2025-09-16T17:39:43.378144] [DEBUG] [VALIDATOR ] BYM: Primary data validation passed. [2025-09-16T17:39:43.378444] [INFO ] [MAIN ] BYM: Fetched primary data is valid. [2025-09-16T17:39:43.378539] [DEBUG] [LOCAL ] BYM: Saving primary data. [2025-09-16T17:39:43.379276] [DEBUG] [LOCAL ] BYM: Trimmed 13107 records to 8475 (keeping 150 days of data) [2025-09-16T17:39:43.454164] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-04 (2.28 seconds). [2025-09-16T17:39:43.474800] [DEBUG] [LOCAL ] BYM: Primary data saved (8475 records). [2025-09-16T17:39:43.474903] [INFO ] [MAIN ] BYM: Full history saved. [2025-09-16T17:39:43.474987] [INFO ] [MAIN ] Update Progress: 380/2369 symbols completed. BYM took 21.6s. Estimated remaining: 3h 27m 39s. Rate: 9.6 symbols/min [2025-09-16T17:39:43.579942] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-02. [2025-09-16T17:39:43.643447] [INFO ] [MAIN ] CABO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:39:43.658106] [DEBUG] [LOCAL ] CABO: Loaded 13796 existing records from primary storage. [2025-09-16T17:39:43.658211] [INFO ] [MAIN ] CABO: Loaded 13796 existing records. [2025-09-16T17:39:43.831697] [WARN ] [MAIN ] CABO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:39:43.831944] [INFO ] [MAIN ] CABO: Wiping existing primary data before full refresh. [2025-09-16T17:39:43.832012] [INFO ] [LOCAL ] CABO: Starting deletion of primary data. [2025-09-16T17:39:43.832394] [DEBUG] [LOCAL ] CABO: Deleted primary data file. [2025-09-16T17:39:43.832439] [INFO ] [MAIN ] CABO: Starting full primary refresh. [2025-09-16T17:39:44.188069] [INFO ] [REMOTE ] CABO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:39:44.291138] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-09-16T17:39:44.374881] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-02. [2025-09-16T17:39:44.656606] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-09-16T17:39:45.174925] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-06. [2025-09-16T17:39:45.433272] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T17:39:45.974906] [INFO ] [REMOTE ] C: Fetching primary data for 2025-05. [2025-09-16T17:39:46.439079] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T17:39:46.774899] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-03. [2025-09-16T17:39:47.574880] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-03. [2025-09-16T17:39:47.762759] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-09-16T17:39:47.864560] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-09-16T17:39:48.374890] [INFO ] [REMOTE ] C: Fetching primary data for 2025-06. [2025-09-16T17:39:48.409517] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T17:39:49.174928] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-07. [2025-09-16T17:39:49.974911] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-04. [2025-09-16T17:39:50.362202] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-09-16T17:39:50.417005] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-09-16T17:39:50.774941] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-04. [2025-09-16T17:39:51.014627] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T17:39:51.574906] [INFO ] [REMOTE ] C: Fetching primary data for 2025-07. [2025-09-16T17:39:51.621804] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-09-16T17:39:52.374918] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-05. [2025-09-16T17:39:53.174932] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-08. [2025-09-16T17:39:53.504288] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-09-16T17:39:53.739175] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-09-16T17:39:53.974974] [INFO ] [REMOTE ] C: Fetching primary data for 2025-08. [2025-09-16T17:39:54.482664] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T17:39:54.774900] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-09. [2025-09-16T17:39:55.574898] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-06. [2025-09-16T17:39:55.997372] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-09-16T17:39:56.258420] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T17:39:56.262766] [INFO ] [REMOTE ] BZH: Full fetch complete (36335 records). [2025-09-16T17:39:56.262861] [DEBUG] [VALIDATOR ] BZH: Validating primary data. [2025-09-16T17:39:56.380000] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-05. [2025-09-16T17:39:56.645681] [DEBUG] [VALIDATOR ] BZH: All expected trading days are present. [2025-09-16T17:39:56.645761] [DEBUG] [VALIDATOR ] BZH: Primary data validation passed. [2025-09-16T17:39:56.646240] [INFO ] [MAIN ] BZH: Fetched primary data is valid. [2025-09-16T17:39:56.646280] [DEBUG] [LOCAL ] BZH: Saving primary data. [2025-09-16T17:39:56.647113] [DEBUG] [LOCAL ] BZH: Trimmed 36335 records to 24301 (keeping 150 days of data) [2025-09-16T17:39:56.786464] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T17:39:56.836363] [DEBUG] [LOCAL ] BZH: Primary data saved (24301 records). [2025-09-16T17:39:56.836547] [INFO ] [MAIN ] BZH: Full history saved. [2025-09-16T17:39:56.836687] [INFO ] [MAIN ] Update Progress: 381/2369 symbols completed. BZH took 26.6s. Estimated remaining: 3h 28m 10s. Rate: 9.5 symbols/min [2025-09-16T17:39:57.020221] [INFO ] [MAIN ] CACI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:39:57.038051] [DEBUG] [LOCAL ] CACI: Loaded 16695 existing records from primary storage. [2025-09-16T17:39:57.038260] [INFO ] [MAIN ] CACI: Loaded 16695 existing records. [2025-09-16T17:39:57.187829] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-07. [2025-09-16T17:39:57.226578] [WARN ] [MAIN ] CACI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:39:57.226829] [INFO ] [MAIN ] CACI: Wiping existing primary data before full refresh. [2025-09-16T17:39:57.226875] [INFO ] [LOCAL ] CACI: Starting deletion of primary data. [2025-09-16T17:39:57.227364] [DEBUG] [LOCAL ] CACI: Deleted primary data file. [2025-09-16T17:39:57.227408] [INFO ] [MAIN ] CACI: Starting full primary refresh. [2025-09-16T17:39:57.583431] [INFO ] [REMOTE ] CACI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:39:57.597763] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T17:39:57.974871] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-02. [2025-09-16T17:39:58.379156] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-09-16T17:39:58.774916] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-06. [2025-09-16T17:39:59.451526] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T17:39:59.538237] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-09-16T17:39:59.574861] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-07. [2025-09-16T17:40:00.374883] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-03. [2025-09-16T17:40:00.682862] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-09-16T17:40:01.175095] [INFO ] [REMOTE ] C: Fetching primary data for 2025-09. [2025-09-16T17:40:01.922022] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T17:40:01.974926] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-08. [2025-09-16T17:40:02.774911] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-08. [2025-09-16T17:40:02.955581] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-09-16T17:40:03.200949] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-09-16T17:40:03.206834] [INFO ] [REMOTE ] C: Full fetch complete (58031 records). [2025-09-16T17:40:03.206932] [DEBUG] [VALIDATOR ] C: Validating primary data. [2025-09-16T17:40:03.207582] [FATAL] [VALIDATOR ] C: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:40:03.207694] [FATAL] [MAIN ] C: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:40:03.207749] [WARN ] [MAIN ] C: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:40:03.207814] [INFO ] [LOCAL ] C: Starting deletion of primary data. [2025-09-16T17:40:03.207879] [DEBUG] [LOCAL ] C: No primary data file found to delete. [2025-09-16T17:40:03.207917] [INFO ] [LOCAL ] C: Starting deletion of derivative data. [2025-09-16T17:40:03.208229] [DEBUG] [LOCAL ] C: Deleted derivative data file. [2025-09-16T17:40:03.209662] [INFO ] [MAIN ] Update Progress: 382/2369 symbols completed. C took 30.0s. Estimated remaining: 3h 28m 4s. Rate: 9.5 symbols/min [2025-09-16T17:40:03.378317] [INFO ] [MAIN ] CADE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:40:03.394853] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-09-16T17:40:03.432829] [DEBUG] [LOCAL ] CADE: Loaded 36349 existing records from primary storage. [2025-09-16T17:40:03.432965] [INFO ] [MAIN ] CADE: Loaded 36349 existing records. [2025-09-16T17:40:03.577588] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-04. [2025-09-16T17:40:03.610533] [WARN ] [MAIN ] CADE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:40:03.610765] [INFO ] [MAIN ] CADE: Wiping existing primary data before full refresh. [2025-09-16T17:40:03.610832] [INFO ] [LOCAL ] CADE: Starting deletion of primary data. [2025-09-16T17:40:03.611716] [DEBUG] [LOCAL ] CADE: Deleted primary data file. [2025-09-16T17:40:03.611772] [INFO ] [MAIN ] CADE: Starting full primary refresh. [2025-09-16T17:40:03.965310] [INFO ] [REMOTE ] CADE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:40:04.374891] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-09. [2025-09-16T17:40:05.157820] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T17:40:05.174858] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-05. [2025-09-16T17:40:05.211559] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T17:40:05.214696] [INFO ] [REMOTE ] CAAP: Full fetch complete (20187 records). [2025-09-16T17:40:05.214768] [DEBUG] [VALIDATOR ] CAAP: Validating primary data. [2025-09-16T17:40:05.574741] [DEBUG] [VALIDATOR ] CAAP: All expected trading days are present. [2025-09-16T17:40:05.574824] [DEBUG] [VALIDATOR ] CAAP: Primary data validation passed. [2025-09-16T17:40:05.575179] [INFO ] [MAIN ] CAAP: Fetched primary data is valid. [2025-09-16T17:40:05.575220] [DEBUG] [LOCAL ] CAAP: Saving primary data. [2025-09-16T17:40:05.575801] [DEBUG] [LOCAL ] CAAP: Trimmed 20187 records to 13352 (keeping 150 days of data) [2025-09-16T17:40:05.658872] [DEBUG] [LOCAL ] CAAP: Primary data saved (13352 records). [2025-09-16T17:40:05.659012] [INFO ] [MAIN ] CAAP: Full history saved. [2025-09-16T17:40:05.659105] [INFO ] [MAIN ] Update Progress: 383/2369 symbols completed. CAAP took 23.4s. Estimated remaining: 3h 27m 38s. Rate: 9.6 symbols/min [2025-09-16T17:40:05.829966] [INFO ] [MAIN ] CAE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:40:05.858083] [DEBUG] [LOCAL ] CAE: Loaded 28270 existing records from primary storage. [2025-09-16T17:40:05.858272] [INFO ] [MAIN ] CAE: Loaded 28270 existing records. [2025-09-16T17:40:05.980018] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-09. [2025-09-16T17:40:06.040314] [WARN ] [MAIN ] CAE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:40:06.040515] [INFO ] [MAIN ] CAE: Wiping existing primary data before full refresh. [2025-09-16T17:40:06.040563] [INFO ] [LOCAL ] CAE: Starting deletion of primary data. [2025-09-16T17:40:06.041216] [DEBUG] [LOCAL ] CAE: Deleted primary data file. [2025-09-16T17:40:06.041265] [INFO ] [MAIN ] CAE: Starting full primary refresh. [2025-09-16T17:40:06.403082] [INFO ] [REMOTE ] CAE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:40:06.422613] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T17:40:06.774917] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-02. [2025-09-16T17:40:07.195887] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-09-16T17:40:07.197784] [INFO ] [REMOTE ] CABO: Full fetch complete (19010 records). [2025-09-16T17:40:07.197855] [DEBUG] [VALIDATOR ] CABO: Validating primary data. [2025-09-16T17:40:07.198293] [FATAL] [VALIDATOR ] CABO: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:40:07.198368] [FATAL] [MAIN ] CABO: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:40:07.198404] [WARN ] [MAIN ] CABO: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:40:07.198484] [INFO ] [LOCAL ] CABO: Starting deletion of primary data. [2025-09-16T17:40:07.198559] [DEBUG] [LOCAL ] CABO: No primary data file found to delete. [2025-09-16T17:40:07.198597] [INFO ] [LOCAL ] CABO: Starting deletion of derivative data. [2025-09-16T17:40:07.198853] [DEBUG] [LOCAL ] CABO: Deleted derivative data file. [2025-09-16T17:40:07.198930] [INFO ] [MAIN ] Update Progress: 384/2369 symbols completed. CABO took 23.7s. Estimated remaining: 3h 27m 8s. Rate: 9.6 symbols/min [2025-09-16T17:40:07.384785] [INFO ] [MAIN ] CAF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:40:07.390525] [DEBUG] [LOCAL ] CAF: Loaded 2269 existing records from primary storage. [2025-09-16T17:40:07.390725] [INFO ] [MAIN ] CAF: Loaded 2269 existing records. [2025-09-16T17:40:07.575793] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-06. [2025-09-16T17:40:07.575616] [WARN ] [MAIN ] CAF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:40:07.577727] [INFO ] [MAIN ] CAF: Wiping existing primary data before full refresh. [2025-09-16T17:40:07.577860] [INFO ] [LOCAL ] CAF: Starting deletion of primary data. [2025-09-16T17:40:07.578095] [DEBUG] [LOCAL ] CAF: Deleted primary data file. [2025-09-16T17:40:07.578144] [INFO ] [MAIN ] CAF: Starting full primary refresh. [2025-09-16T17:40:07.944002] [INFO ] [REMOTE ] CAF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:40:08.162707] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T17:40:08.374939] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-03. [2025-09-16T17:40:08.756970] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-09-16T17:40:09.174942] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-02. [2025-09-16T17:40:09.557215] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-09-16T17:40:09.813260] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-09-16T17:40:09.974930] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-03. [2025-09-16T17:40:10.411150] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-09-16T17:40:10.774908] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-07. [2025-09-16T17:40:11.574916] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-02. [2025-09-16T17:40:11.648570] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T17:40:12.374928] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-08. [2025-09-16T17:40:13.174962] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-04. [2025-09-16T17:40:13.420758] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-09-16T17:40:13.445338] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T17:40:13.662281] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-09-16T17:40:13.974909] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-05. [2025-09-16T17:40:14.548439] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-09-16T17:40:14.774940] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-06. [2025-09-16T17:40:15.218016] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-09-16T17:40:15.574902] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-03. [2025-09-16T17:40:16.374928] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-09. [2025-09-16T17:40:17.174902] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-07. [2025-09-16T17:40:17.342433] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T17:40:17.362764] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T17:40:17.365181] [INFO ] [REMOTE ] CACI: Full fetch complete (29391 records). [2025-09-16T17:40:17.365270] [DEBUG] [VALIDATOR ] CACI: Validating primary data. [2025-09-16T17:40:17.365853] [FATAL] [VALIDATOR ] CACI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:40:17.365945] [FATAL] [MAIN ] CACI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:40:17.365998] [WARN ] [MAIN ] CACI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:40:17.366066] [INFO ] [LOCAL ] CACI: Starting deletion of primary data. [2025-09-16T17:40:17.366207] [DEBUG] [LOCAL ] CACI: No primary data file found to delete. [2025-09-16T17:40:17.366264] [INFO ] [LOCAL ] CACI: Starting deletion of derivative data. [2025-09-16T17:40:17.366589] [DEBUG] [LOCAL ] CACI: Deleted derivative data file. [2025-09-16T17:40:17.366714] [INFO ] [MAIN ] Update Progress: 385/2369 symbols completed. CACI took 20.5s. Estimated remaining: 3h 27m 21s. Rate: 9.6 symbols/min [2025-09-16T17:40:17.542159] [INFO ] [MAIN ] CAG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:40:17.581890] [DEBUG] [LOCAL ] CAG: Loaded 39628 existing records from primary storage. [2025-09-16T17:40:17.582035] [INFO ] [MAIN ] CAG: Loaded 39628 existing records. [2025-09-16T17:40:17.772540] [WARN ] [MAIN ] CAG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:40:17.775975] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-09-16T17:40:17.775815] [INFO ] [MAIN ] CAG: Wiping existing primary data before full refresh. [2025-09-16T17:40:17.777408] [INFO ] [LOCAL ] CAG: Starting deletion of primary data. [2025-09-16T17:40:17.778386] [DEBUG] [LOCAL ] CAG: Deleted primary data file. [2025-09-16T17:40:17.778437] [INFO ] [MAIN ] CAG: Starting full primary refresh. [2025-09-16T17:40:17.981529] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-04. [2025-09-16T17:40:18.144388] [INFO ] [REMOTE ] CAG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:40:18.774913] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-04. [2025-09-16T17:40:19.574899] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-02. [2025-09-16T17:40:19.607229] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-09-16T17:40:19.883032] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-09-16T17:40:20.374923] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-05. [2025-09-16T17:40:21.174924] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-08. [2025-09-16T17:40:21.486928] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-09-16T17:40:21.914412] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-09-16T17:40:21.973194] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-09-16T17:40:21.982242] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-09. [2025-09-16T17:40:22.482463] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-09-16T17:40:22.483569] [INFO ] [REMOTE ] CAF: Full fetch complete (3942 records). [2025-09-16T17:40:22.483638] [DEBUG] [VALIDATOR ] CAF: Validating primary data. [2025-09-16T17:40:22.780008] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-03. [2025-09-16T17:40:22.843315] [DEBUG] [VALIDATOR ] CAF: All expected trading days are present. [2025-09-16T17:40:22.843418] [DEBUG] [VALIDATOR ] CAF: Primary data validation passed. [2025-09-16T17:40:22.843611] [INFO ] [MAIN ] CAF: Fetched primary data is valid. [2025-09-16T17:40:22.843777] [DEBUG] [LOCAL ] CAF: Saving primary data. [2025-09-16T17:40:22.844240] [DEBUG] [LOCAL ] CAF: Trimmed 3942 records to 2322 (keeping 150 days of data) [2025-09-16T17:40:22.857948] [DEBUG] [LOCAL ] CAF: Primary data saved (2322 records). [2025-09-16T17:40:22.858014] [INFO ] [MAIN ] CAF: Full history saved. [2025-09-16T17:40:22.858094] [INFO ] [MAIN ] Update Progress: 386/2369 symbols completed. CAF took 15.7s. Estimated remaining: 3h 27m 11s. Rate: 9.6 symbols/min [2025-09-16T17:40:23.035865] [INFO ] [MAIN ] CAH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:40:23.081426] [DEBUG] [LOCAL ] CAH: Loaded 37932 existing records from primary storage. [2025-09-16T17:40:23.081596] [INFO ] [MAIN ] CAH: Loaded 37932 existing records. [2025-09-16T17:40:23.266243] [WARN ] [MAIN ] CAH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:40:23.266557] [INFO ] [MAIN ] CAH: Wiping existing primary data before full refresh. [2025-09-16T17:40:23.266621] [INFO ] [LOCAL ] CAH: Starting deletion of primary data. [2025-09-16T17:40:23.267708] [DEBUG] [LOCAL ] CAH: Deleted primary data file. [2025-09-16T17:40:23.267774] [INFO ] [MAIN ] CAH: Starting full primary refresh. [2025-09-16T17:40:23.580067] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-06. [2025-09-16T17:40:23.645287] [INFO ] [REMOTE ] CAH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:40:24.341326] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T17:40:24.374918] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-04. [2025-09-16T17:40:25.174925] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-05. [2025-09-16T17:40:25.294026] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T17:40:25.974915] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-07. [2025-09-16T17:40:26.126057] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-09-16T17:40:26.571988] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T17:40:26.774900] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-06. [2025-09-16T17:40:27.574902] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-02. [2025-09-16T17:40:27.714636] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T17:40:28.374931] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-05. [2025-09-16T17:40:28.397237] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T17:40:29.174926] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-08. [2025-09-16T17:40:29.428882] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-09-16T17:40:29.892050] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T17:40:29.974895] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-03. [2025-09-16T17:40:30.774947] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-07. [2025-09-16T17:40:30.896002] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-09-16T17:40:31.574934] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-09. [2025-09-16T17:40:31.867168] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-09-16T17:40:32.371487] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T17:40:32.380171] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-04. [2025-09-16T17:40:33.174925] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-08. [2025-09-16T17:40:33.286687] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T17:40:33.292832] [INFO ] [REMOTE ] CADE: Full fetch complete (52745 records). [2025-09-16T17:40:33.293024] [DEBUG] [VALIDATOR ] CADE: Validating primary data. [2025-09-16T17:40:33.682262] [DEBUG] [VALIDATOR ] CADE: All expected trading days are present. [2025-09-16T17:40:33.682343] [DEBUG] [VALIDATOR ] CADE: Primary data validation passed. [2025-09-16T17:40:33.682998] [INFO ] [MAIN ] CADE: Fetched primary data is valid. [2025-09-16T17:40:33.683052] [DEBUG] [LOCAL ] CADE: Saving primary data. [2025-09-16T17:40:33.684185] [DEBUG] [LOCAL ] CADE: Trimmed 52745 records to 36711 (keeping 150 days of data) [2025-09-16T17:40:33.910200] [DEBUG] [LOCAL ] CADE: Primary data saved (36711 records). [2025-09-16T17:40:33.910428] [INFO ] [MAIN ] CADE: Full history saved. [2025-09-16T17:40:33.911804] [INFO ] [MAIN ] Update Progress: 387/2369 symbols completed. CADE took 30.7s. Estimated remaining: 3h 27m 29s. Rate: 9.6 symbols/min [2025-09-16T17:40:33.986692] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-06. [2025-09-16T17:40:34.049702] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T17:40:34.131297] [INFO ] [MAIN ] CAL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:40:34.164618] [DEBUG] [LOCAL ] CAL: Loaded 31389 existing records from primary storage. [2025-09-16T17:40:34.164805] [INFO ] [MAIN ] CAL: Loaded 31389 existing records. [2025-09-16T17:40:34.392289] [WARN ] [MAIN ] CAL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:40:34.392565] [INFO ] [MAIN ] CAL: Wiping existing primary data before full refresh. [2025-09-16T17:40:34.392830] [INFO ] [LOCAL ] CAL: Starting deletion of primary data. [2025-09-16T17:40:34.393573] [DEBUG] [LOCAL ] CAL: Deleted primary data file. [2025-09-16T17:40:34.393635] [INFO ] [MAIN ] CAL: Starting full primary refresh. [2025-09-16T17:40:34.577989] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-09-16T17:40:34.780001] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-09. [2025-09-16T17:40:34.808184] [INFO ] [REMOTE ] CAL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:40:35.574925] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-05. [2025-09-16T17:40:36.137888] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T17:40:36.142344] [INFO ] [REMOTE ] CAE: Full fetch complete (42459 records). [2025-09-16T17:40:36.142454] [DEBUG] [VALIDATOR ] CAE: Validating primary data. [2025-09-16T17:40:36.379497] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-02. [2025-09-16T17:40:36.511576] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-06 (2.52 seconds). [2025-09-16T17:40:36.589012] [DEBUG] [VALIDATOR ] CAE: All expected trading days are present. [2025-09-16T17:40:36.589106] [DEBUG] [VALIDATOR ] CAE: Primary data validation passed. [2025-09-16T17:40:36.589709] [INFO ] [MAIN ] CAE: Fetched primary data is valid. [2025-09-16T17:40:36.589759] [DEBUG] [LOCAL ] CAE: Saving primary data. [2025-09-16T17:40:36.590858] [DEBUG] [LOCAL ] CAE: Trimmed 42459 records to 28505 (keeping 150 days of data) [2025-09-16T17:40:36.760699] [DEBUG] [LOCAL ] CAE: Primary data saved (28505 records). [2025-09-16T17:40:36.760854] [INFO ] [MAIN ] CAE: Full history saved. [2025-09-16T17:40:36.760942] [INFO ] [MAIN ] Update Progress: 388/2369 symbols completed. CAE took 31.1s. Estimated remaining: 3h 27m 6s. Rate: 9.6 symbols/min [2025-09-16T17:40:36.933717] [INFO ] [MAIN ] CALX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:40:36.961831] [DEBUG] [LOCAL ] CALX: Loaded 28396 existing records from primary storage. [2025-09-16T17:40:36.962001] [INFO ] [MAIN ] CALX: Loaded 28396 existing records. [2025-09-16T17:40:37.144781] [WARN ] [MAIN ] CALX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:40:37.145170] [INFO ] [MAIN ] CALX: Wiping existing primary data before full refresh. [2025-09-16T17:40:37.145229] [INFO ] [LOCAL ] CALX: Starting deletion of primary data. [2025-09-16T17:40:37.145940] [DEBUG] [LOCAL ] CALX: Deleted primary data file. [2025-09-16T17:40:37.145995] [INFO ] [MAIN ] CALX: Starting full primary refresh. [2025-09-16T17:40:37.179309] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-07. [2025-09-16T17:40:37.559206] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-09-16T17:40:37.564039] [INFO ] [REMOTE ] CALX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:40:37.653343] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T17:40:37.974905] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-06. [2025-09-16T17:40:38.774950] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-03. [2025-09-16T17:40:39.337083] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-09-16T17:40:39.574889] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-08. [2025-09-16T17:40:39.788109] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-09-16T17:40:40.190062] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T17:40:40.374968] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-02. [2025-09-16T17:40:41.126891] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T17:40:41.174936] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-09. [2025-09-16T17:40:41.797888] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T17:40:41.974964] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-04. [2025-09-16T17:40:42.774933] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-07. [2025-09-16T17:40:42.987630] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-09-16T17:40:42.994683] [INFO ] [REMOTE ] CAG: Full fetch complete (58335 records). [2025-09-16T17:40:42.994852] [DEBUG] [VALIDATOR ] CAG: Validating primary data. [2025-09-16T17:40:43.396927] [DEBUG] [VALIDATOR ] CAG: All expected trading days are present. [2025-09-16T17:40:43.397029] [DEBUG] [VALIDATOR ] CAG: Primary data validation passed. [2025-09-16T17:40:43.397720] [INFO ] [MAIN ] CAG: Fetched primary data is valid. [2025-09-16T17:40:43.397767] [DEBUG] [LOCAL ] CAG: Saving primary data. [2025-09-16T17:40:43.399094] [DEBUG] [LOCAL ] CAG: Trimmed 58335 records to 40018 (keeping 150 days of data) [2025-09-16T17:40:43.592102] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-03. [2025-09-16T17:40:43.665254] [DEBUG] [LOCAL ] CAG: Primary data saved (40018 records). [2025-09-16T17:40:43.665392] [INFO ] [MAIN ] CAG: Full history saved. [2025-09-16T17:40:43.665476] [INFO ] [MAIN ] Update Progress: 389/2369 symbols completed. CAG took 26.3s. Estimated remaining: 3h 27m 2s. Rate: 9.6 symbols/min [2025-09-16T17:40:43.844072] [INFO ] [MAIN ] CANG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:40:43.864887] [DEBUG] [LOCAL ] CANG: Loaded 20242 existing records from primary storage. [2025-09-16T17:40:43.865039] [INFO ] [MAIN ] CANG: Loaded 20242 existing records. [2025-09-16T17:40:44.047192] [WARN ] [MAIN ] CANG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:40:44.047468] [INFO ] [MAIN ] CANG: Wiping existing primary data before full refresh. [2025-09-16T17:40:44.047516] [INFO ] [LOCAL ] CANG: Starting deletion of primary data. [2025-09-16T17:40:44.048071] [DEBUG] [LOCAL ] CANG: Deleted primary data file. [2025-09-16T17:40:44.048154] [INFO ] [MAIN ] CANG: Starting full primary refresh. [2025-09-16T17:40:44.413760] [INFO ] [REMOTE ] CANG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:40:44.414109] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-02. [2025-09-16T17:40:44.625160] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T17:40:44.667328] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T17:40:45.129277] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-09-16T17:40:45.174906] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-03. [2025-09-16T17:40:45.974936] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-04. [2025-09-16T17:40:46.453041] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-09-16T17:40:46.774882] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-04. [2025-09-16T17:40:46.827188] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-04 (4.85 seconds). [2025-09-16T17:40:47.515145] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-09-16T17:40:47.574930] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-05. [2025-09-16T17:40:47.797148] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-09-16T17:40:48.374975] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-08. [2025-09-16T17:40:48.826584] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T17:40:49.174949] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-06. [2025-09-16T17:40:49.974877] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-05. [2025-09-16T17:40:50.012819] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T17:40:50.525513] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T17:40:50.774958] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-09. [2025-09-16T17:40:51.522834] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-09-16T17:40:51.574911] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-06. [2025-09-16T17:40:52.194557] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T17:40:52.200888] [INFO ] [REMOTE ] CAH: Full fetch complete (56136 records). [2025-09-16T17:40:52.201033] [DEBUG] [VALIDATOR ] CAH: Validating primary data. [2025-09-16T17:40:52.380038] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-05. [2025-09-16T17:40:52.601501] [DEBUG] [VALIDATOR ] CAH: All expected trading days are present. [2025-09-16T17:40:52.601763] [DEBUG] [VALIDATOR ] CAH: Primary data validation passed. [2025-09-16T17:40:52.602688] [INFO ] [MAIN ] CAH: Fetched primary data is valid. [2025-09-16T17:40:52.602739] [DEBUG] [LOCAL ] CAH: Saving primary data. [2025-09-16T17:40:52.604183] [DEBUG] [LOCAL ] CAH: Trimmed 56136 records to 38301 (keeping 150 days of data) [2025-09-16T17:40:52.853219] [DEBUG] [LOCAL ] CAH: Primary data saved (38301 records). [2025-09-16T17:40:52.853383] [INFO ] [MAIN ] CAH: Full history saved. [2025-09-16T17:40:52.855150] [INFO ] [MAIN ] Update Progress: 390/2369 symbols completed. CAH took 30.0s. Estimated remaining: 3h 27m 11s. Rate: 9.6 symbols/min [2025-09-16T17:40:53.032265] [INFO ] [MAIN ] CAPL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:40:53.039252] [DEBUG] [LOCAL ] CAPL: Loaded 3877 existing records from primary storage. [2025-09-16T17:40:53.039386] [INFO ] [MAIN ] CAPL: Loaded 3877 existing records. [2025-09-16T17:40:53.185390] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-07. [2025-09-16T17:40:53.224461] [WARN ] [MAIN ] CAPL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:40:53.225975] [INFO ] [MAIN ] CAPL: Wiping existing primary data before full refresh. [2025-09-16T17:40:53.226086] [INFO ] [LOCAL ] CAPL: Starting deletion of primary data. [2025-09-16T17:40:53.226615] [DEBUG] [LOCAL ] CAPL: Deleted primary data file. [2025-09-16T17:40:53.226851] [INFO ] [MAIN ] CAPL: Starting full primary refresh. [2025-09-16T17:40:53.441997] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-09-16T17:40:53.654198] [INFO ] [REMOTE ] CAPL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:40:53.844052] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T17:40:53.974928] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-02. [2025-09-16T17:40:54.569562] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-09-16T17:40:54.774959] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-06. [2025-09-16T17:40:54.950784] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-09-16T17:40:55.574922] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-08. [2025-09-16T17:40:55.660410] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T17:40:56.374916] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-03. [2025-09-16T17:40:56.703589] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T17:40:57.171987] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T17:40:57.174843] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-04. [2025-09-16T17:40:57.724989] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T17:40:57.974948] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-05. [2025-09-16T17:40:58.674081] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-09-16T17:40:58.774946] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-07. [2025-09-16T17:40:59.574918] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-06. [2025-09-16T17:41:00.242978] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-09-16T17:41:00.340692] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T17:41:00.374890] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-08. [2025-09-16T17:41:01.174902] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-09. [2025-09-16T17:41:01.955514] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T17:41:01.974846] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-09. [2025-09-16T17:41:02.533588] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T17:41:02.538282] [INFO ] [REMOTE ] CALX: Full fetch complete (41816 records). [2025-09-16T17:41:02.538403] [DEBUG] [VALIDATOR ] CALX: Validating primary data. [2025-09-16T17:41:02.780127] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-07. [2025-09-16T17:41:02.931264] [DEBUG] [VALIDATOR ] CALX: All expected trading days are present. [2025-09-16T17:41:02.931351] [DEBUG] [VALIDATOR ] CALX: Primary data validation passed. [2025-09-16T17:41:02.931993] [INFO ] [MAIN ] CALX: Fetched primary data is valid. [2025-09-16T17:41:02.932057] [DEBUG] [LOCAL ] CALX: Saving primary data. [2025-09-16T17:41:02.933085] [DEBUG] [LOCAL ] CALX: Trimmed 41816 records to 28624 (keeping 150 days of data) [2025-09-16T17:41:03.106692] [DEBUG] [LOCAL ] CALX: Primary data saved (28624 records). [2025-09-16T17:41:03.106836] [INFO ] [MAIN ] CALX: Full history saved. [2025-09-16T17:41:03.106931] [INFO ] [MAIN ] Update Progress: 391/2369 symbols completed. CALX took 26.3s. Estimated remaining: 3h 27m 25s. Rate: 9.5 symbols/min [2025-09-16T17:41:03.279027] [INFO ] [MAIN ] CARR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:03.317225] [DEBUG] [LOCAL ] CARR: Loaded 39576 existing records from primary storage. [2025-09-16T17:41:03.317387] [INFO ] [MAIN ] CARR: Loaded 39576 existing records. [2025-09-16T17:41:03.452115] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-09-16T17:41:03.511365] [WARN ] [MAIN ] CARR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:41:03.513880] [INFO ] [MAIN ] CARR: Wiping existing primary data before full refresh. [2025-09-16T17:41:03.514067] [INFO ] [LOCAL ] CARR: Starting deletion of primary data. [2025-09-16T17:41:03.520147] [DEBUG] [LOCAL ] CARR: Deleted primary data file. [2025-09-16T17:41:03.529041] [INFO ] [MAIN ] CARR: Starting full primary refresh. [2025-09-16T17:41:03.576693] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-08. [2025-09-16T17:41:03.535975] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T17:41:03.635865] [INFO ] [REMOTE ] CAL: Full fetch complete (46971 records). [2025-09-16T17:41:03.647782] [DEBUG] [VALIDATOR ] CAL: Validating primary data. [2025-09-16T17:41:04.164254] [INFO ] [REMOTE ] CARR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:04.320257] [DEBUG] [VALIDATOR ] CAL: All expected trading days are present. [2025-09-16T17:41:04.320487] [DEBUG] [VALIDATOR ] CAL: Primary data validation passed. [2025-09-16T17:41:04.325982] [INFO ] [MAIN ] CAL: Fetched primary data is valid. [2025-09-16T17:41:04.326611] [DEBUG] [LOCAL ] CAL: Saving primary data. [2025-09-16T17:41:04.326886] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T17:41:04.330244] [DEBUG] [LOCAL ] CAL: Trimmed 46971 records to 31697 (keeping 150 days of data) [2025-09-16T17:41:04.387465] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-09. [2025-09-16T17:41:04.537306] [DEBUG] [LOCAL ] CAL: Primary data saved (31697 records). [2025-09-16T17:41:04.537461] [INFO ] [MAIN ] CAL: Full history saved. [2025-09-16T17:41:04.537549] [INFO ] [MAIN ] Update Progress: 392/2369 symbols completed. CAL took 30.6s. Estimated remaining: 3h 26m 54s. Rate: 9.6 symbols/min [2025-09-16T17:41:04.712235] [INFO ] [MAIN ] CARS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:04.740090] [DEBUG] [LOCAL ] CARS: Loaded 28899 existing records from primary storage. [2025-09-16T17:41:04.740248] [INFO ] [MAIN ] CARS: Loaded 28899 existing records. [2025-09-16T17:41:04.917343] [WARN ] [MAIN ] CARS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:41:04.917610] [INFO ] [MAIN ] CARS: Wiping existing primary data before full refresh. [2025-09-16T17:41:04.917677] [INFO ] [LOCAL ] CARS: Starting deletion of primary data. [2025-09-16T17:41:04.918359] [DEBUG] [LOCAL ] CARS: Deleted primary data file. [2025-09-16T17:41:04.918408] [INFO ] [MAIN ] CARS: Starting full primary refresh. [2025-09-16T17:41:05.175423] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-02. [2025-09-16T17:41:05.189711] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T17:41:05.207852] [INFO ] [REMOTE ] CAPL: Full fetch complete (6482 records). [2025-09-16T17:41:05.208790] [DEBUG] [VALIDATOR ] CAPL: Validating primary data. [2025-09-16T17:41:05.395487] [INFO ] [REMOTE ] CARS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:05.678412] [DEBUG] [VALIDATOR ] CAPL: All expected trading days are present. [2025-09-16T17:41:05.678493] [DEBUG] [VALIDATOR ] CAPL: Primary data validation passed. [2025-09-16T17:41:05.678740] [INFO ] [MAIN ] CAPL: Fetched primary data is valid. [2025-09-16T17:41:05.678786] [DEBUG] [LOCAL ] CAPL: Saving primary data. [2025-09-16T17:41:05.679209] [DEBUG] [LOCAL ] CAPL: Trimmed 6482 records to 3928 (keeping 150 days of data) [2025-09-16T17:41:05.705351] [DEBUG] [LOCAL ] CAPL: Primary data saved (3928 records). [2025-09-16T17:41:05.705455] [INFO ] [MAIN ] CAPL: Full history saved. [2025-09-16T17:41:05.705547] [INFO ] [MAIN ] Update Progress: 393/2369 symbols completed. CAPL took 12.9s. Estimated remaining: 3h 26m 22s. Rate: 9.6 symbols/min [2025-09-16T17:41:05.880846] [INFO ] [MAIN ] CAT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:05.921392] [DEBUG] [LOCAL ] CAT: Loaded 39023 existing records from primary storage. [2025-09-16T17:41:05.921536] [INFO ] [MAIN ] CAT: Loaded 39023 existing records. [2025-09-16T17:41:05.985895] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-02. [2025-09-16T17:41:06.104982] [WARN ] [MAIN ] CAT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:41:06.105233] [INFO ] [MAIN ] CAT: Wiping existing primary data before full refresh. [2025-09-16T17:41:06.105287] [INFO ] [LOCAL ] CAT: Starting deletion of primary data. [2025-09-16T17:41:06.106198] [DEBUG] [LOCAL ] CAT: Deleted primary data file. [2025-09-16T17:41:06.106258] [INFO ] [MAIN ] CAT: Starting full primary refresh. [2025-09-16T17:41:06.475173] [INFO ] [REMOTE ] CAT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:06.774851] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-02. [2025-09-16T17:41:07.071269] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-09-16T17:41:07.574906] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-03. [2025-09-16T17:41:07.607920] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T17:41:08.374901] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-03. [2025-09-16T17:41:08.738717] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-09-16T17:41:09.174909] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-07. [2025-09-16T17:41:09.335057] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T17:41:09.902216] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T17:41:09.974977] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-04. [2025-09-16T17:41:10.545869] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T17:41:10.774944] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-04. [2025-09-16T17:41:11.574921] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-03. [2025-09-16T17:41:12.000880] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-09-16T17:41:12.083903] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T17:41:12.374910] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-05. [2025-09-16T17:41:13.174921] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-08. [2025-09-16T17:41:13.591781] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T17:41:13.926103] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-03 (2.35 seconds). [2025-09-16T17:41:13.974874] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-04. [2025-09-16T17:41:14.294867] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T17:41:14.774941] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-06. [2025-09-16T17:41:15.574903] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-05. [2025-09-16T17:41:16.315018] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-04 (2.34 seconds). [2025-09-16T17:41:16.352510] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T17:41:16.374859] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-07. [2025-09-16T17:41:17.174875] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-09. [2025-09-16T17:41:17.288037] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-09-16T17:41:17.685325] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T17:41:17.974935] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-05. [2025-09-16T17:41:18.359815] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T17:41:18.364053] [INFO ] [REMOTE ] CANG: Full fetch complete (29124 records). [2025-09-16T17:41:18.364160] [DEBUG] [VALIDATOR ] CANG: Validating primary data. [2025-09-16T17:41:18.736997] [DEBUG] [VALIDATOR ] CANG: All expected trading days are present. [2025-09-16T17:41:18.737077] [DEBUG] [VALIDATOR ] CANG: Primary data validation passed. [2025-09-16T17:41:18.737520] [INFO ] [MAIN ] CANG: Fetched primary data is valid. [2025-09-16T17:41:18.737564] [DEBUG] [LOCAL ] CANG: Saving primary data. [2025-09-16T17:41:18.738331] [DEBUG] [LOCAL ] CANG: Trimmed 29124 records to 20458 (keeping 150 days of data) [2025-09-16T17:41:18.784334] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-08. [2025-09-16T17:41:18.856178] [DEBUG] [LOCAL ] CANG: Primary data saved (20458 records). [2025-09-16T17:41:18.856303] [INFO ] [MAIN ] CANG: Full history saved. [2025-09-16T17:41:18.856396] [INFO ] [MAIN ] Update Progress: 394/2369 symbols completed. CANG took 35.2s. Estimated remaining: 3h 26m 50s. Rate: 9.5 symbols/min [2025-09-16T17:41:19.046281] [INFO ] [MAIN ] CATO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:19.053300] [DEBUG] [LOCAL ] CATO: Loaded 5040 existing records from primary storage. [2025-09-16T17:41:19.053460] [INFO ] [MAIN ] CATO: Loaded 5040 existing records. [2025-09-16T17:41:19.251006] [WARN ] [MAIN ] CATO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:41:19.251236] [INFO ] [MAIN ] CATO: Wiping existing primary data before full refresh. [2025-09-16T17:41:19.251281] [INFO ] [LOCAL ] CATO: Starting deletion of primary data. [2025-09-16T17:41:19.251593] [DEBUG] [LOCAL ] CATO: Deleted primary data file. [2025-09-16T17:41:19.251640] [INFO ] [MAIN ] CATO: Starting full primary refresh. [2025-09-16T17:41:19.580145] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-06. [2025-09-16T17:41:19.640195] [INFO ] [REMOTE ] CATO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:20.235783] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-05 (2.26 seconds). [2025-09-16T17:41:20.312331] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T17:41:20.374894] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-09. [2025-09-16T17:41:21.174940] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-02. [2025-09-16T17:41:21.409808] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T17:41:21.731443] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T17:41:21.736536] [INFO ] [REMOTE ] CARS: Full fetch complete (43611 records). [2025-09-16T17:41:21.736715] [DEBUG] [VALIDATOR ] CARS: Validating primary data. [2025-09-16T17:41:21.984473] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-06. [2025-09-16T17:41:22.119365] [DEBUG] [VALIDATOR ] CARS: All expected trading days are present. [2025-09-16T17:41:22.119476] [DEBUG] [VALIDATOR ] CARS: Primary data validation passed. [2025-09-16T17:41:22.120132] [INFO ] [MAIN ] CARS: Fetched primary data is valid. [2025-09-16T17:41:22.122957] [DEBUG] [LOCAL ] CARS: Saving primary data. [2025-09-16T17:41:22.126278] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-09-16T17:41:22.128926] [DEBUG] [LOCAL ] CARS: Trimmed 43611 records to 29216 (keeping 150 days of data) [2025-09-16T17:41:22.303948] [DEBUG] [LOCAL ] CARS: Primary data saved (29216 records). [2025-09-16T17:41:22.304096] [INFO ] [MAIN ] CARS: Full history saved. [2025-09-16T17:41:22.304200] [INFO ] [MAIN ] Update Progress: 395/2369 symbols completed. CARS took 17.8s. Estimated remaining: 3h 26m 30s. Rate: 9.6 symbols/min [2025-09-16T17:41:22.472602] [INFO ] [MAIN ] CAVA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:22.472807] [DEBUG] [LOCAL ] CAVA: No existing data file found in primary storage. [2025-09-16T17:41:22.472844] [WARN ] [MAIN ] CAVA: No existing data found, performing full refresh instead. [2025-09-16T17:41:22.473013] [INFO ] [MAIN ] CAVA: Wiping existing primary data before full refresh. [2025-09-16T17:41:22.473062] [INFO ] [LOCAL ] CAVA: Starting deletion of primary data. [2025-09-16T17:41:22.473106] [DEBUG] [LOCAL ] CAVA: No primary data file found to delete. [2025-09-16T17:41:22.473139] [INFO ] [MAIN ] CAVA: Starting full primary refresh. [2025-09-16T17:41:22.779963] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-03. [2025-09-16T17:41:22.830544] [INFO ] [REMOTE ] CAVA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:23.574886] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-02. [2025-09-16T17:41:23.703306] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-09-16T17:41:24.037130] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-06 (2.05 seconds). [2025-09-16T17:41:24.374900] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-07. [2025-09-16T17:41:25.174921] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-04. [2025-09-16T17:41:25.702171] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-02 (2.13 seconds). [2025-09-16T17:41:25.721606] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T17:41:25.974875] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-05. [2025-09-16T17:41:26.049399] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-09-16T17:41:26.616274] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-09-16T17:41:26.774885] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-08. [2025-09-16T17:41:27.574905] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-06. [2025-09-16T17:41:28.100732] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-09-16T17:41:28.374903] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-07. [2025-09-16T17:41:28.801440] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-08 (2.03 seconds). [2025-09-16T17:41:29.174905] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-03. [2025-09-16T17:41:29.974898] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-07. [2025-09-16T17:41:30.495433] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-09-16T17:41:30.621876] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-09-16T17:41:30.774886] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-08. [2025-09-16T17:41:31.111669] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-09-16T17:41:31.574897] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-09. [2025-09-16T17:41:32.374873] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-08. [2025-09-16T17:41:32.549579] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T17:41:33.061545] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T17:41:33.069025] [INFO ] [REMOTE ] CARR: Full fetch complete (58278 records). [2025-09-16T17:41:33.069158] [DEBUG] [VALIDATOR ] CARR: Validating primary data. [2025-09-16T17:41:33.177150] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-04. [2025-09-16T17:41:33.236986] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T17:41:33.462867] [DEBUG] [VALIDATOR ] CARR: All expected trading days are present. [2025-09-16T17:41:33.462951] [DEBUG] [VALIDATOR ] CARR: Primary data validation passed. [2025-09-16T17:41:33.463682] [INFO ] [MAIN ] CARR: Fetched primary data is valid. [2025-09-16T17:41:33.463728] [DEBUG] [LOCAL ] CARR: Saving primary data. [2025-09-16T17:41:33.465240] [DEBUG] [LOCAL ] CARR: Trimmed 58278 records to 39965 (keeping 150 days of data) [2025-09-16T17:41:33.713014] [DEBUG] [LOCAL ] CARR: Primary data saved (39965 records). [2025-09-16T17:41:33.713172] [INFO ] [MAIN ] CARR: Full history saved. [2025-09-16T17:41:33.713285] [INFO ] [MAIN ] Update Progress: 396/2369 symbols completed. CARR took 30.6s. Estimated remaining: 3h 26m 49s. Rate: 9.5 symbols/min [2025-09-16T17:41:33.887531] [INFO ] [MAIN ] CB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:33.887761] [DEBUG] [LOCAL ] CB: No existing data file found in primary storage. [2025-09-16T17:41:33.887801] [WARN ] [MAIN ] CB: No existing data found, performing full refresh instead. [2025-09-16T17:41:33.887992] [INFO ] [MAIN ] CB: Wiping existing primary data before full refresh. [2025-09-16T17:41:33.888048] [INFO ] [LOCAL ] CB: Starting deletion of primary data. [2025-09-16T17:41:33.888092] [DEBUG] [LOCAL ] CB: No primary data file found to delete. [2025-09-16T17:41:33.888126] [INFO ] [MAIN ] CB: Starting full primary refresh. [2025-09-16T17:41:33.984213] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-09. [2025-09-16T17:41:34.253894] [INFO ] [REMOTE ] CB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:34.718092] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T17:41:34.719904] [INFO ] [REMOTE ] CATO: Full fetch complete (8206 records). [2025-09-16T17:41:34.719973] [DEBUG] [VALIDATOR ] CATO: Validating primary data. [2025-09-16T17:41:34.782616] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-09. [2025-09-16T17:41:35.086257] [DEBUG] [VALIDATOR ] CATO: All expected trading days are present. [2025-09-16T17:41:35.086378] [DEBUG] [VALIDATOR ] CATO: Primary data validation passed. [2025-09-16T17:41:35.086711] [INFO ] [MAIN ] CATO: Fetched primary data is valid. [2025-09-16T17:41:35.086974] [DEBUG] [LOCAL ] CATO: Saving primary data. [2025-09-16T17:41:35.087736] [DEBUG] [LOCAL ] CATO: Trimmed 8206 records to 5079 (keeping 150 days of data) [2025-09-16T17:41:35.118421] [DEBUG] [LOCAL ] CATO: Primary data saved (5079 records). [2025-09-16T17:41:35.118730] [INFO ] [MAIN ] CATO: Full history saved. [2025-09-16T17:41:35.118936] [INFO ] [MAIN ] Update Progress: 397/2369 symbols completed. CATO took 16.3s. Estimated remaining: 3h 26m 19s. Rate: 9.6 symbols/min [2025-09-16T17:41:35.188929] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-09-16T17:41:35.574933] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-05. [2025-09-16T17:41:36.374906] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-02. [2025-09-16T17:41:36.882181] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-09-16T17:41:36.889807] [INFO ] [REMOTE ] CAT: Full fetch complete (57567 records). [2025-09-16T17:41:36.889957] [DEBUG] [VALIDATOR ] CAT: Validating primary data. [2025-09-16T17:41:37.278927] [DEBUG] [VALIDATOR ] CAT: All expected trading days are present. [2025-09-16T17:41:37.279020] [DEBUG] [VALIDATOR ] CAT: Primary data validation passed. [2025-09-16T17:41:37.280293] [INFO ] [MAIN ] CAT: Fetched primary data is valid. [2025-09-16T17:41:37.280347] [DEBUG] [LOCAL ] CAT: Saving primary data. [2025-09-16T17:41:37.281764] [DEBUG] [LOCAL ] CAT: Trimmed 57567 records to 39407 (keeping 150 days of data) [2025-09-16T17:41:37.538704] [DEBUG] [LOCAL ] CAT: Primary data saved (39407 records). [2025-09-16T17:41:37.538860] [INFO ] [MAIN ] CAT: Full history saved. [2025-09-16T17:41:37.540020] [INFO ] [MAIN ] Update Progress: 398/2369 symbols completed. CAT took 31.8s. Estimated remaining: 3h 25m 53s. Rate: 9.6 symbols/min [2025-09-16T17:41:37.698527] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-09-16T17:41:37.709914] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-06. [2025-09-16T17:41:38.150749] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T17:41:38.160713] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-03. [2025-09-16T17:41:39.616952] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-09-16T17:41:39.628127] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-07. [2025-09-16T17:41:39.762842] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T17:41:39.773379] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-04. [2025-09-16T17:41:41.440063] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T17:41:41.451821] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-05. [2025-09-16T17:41:41.743943] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-09-16T17:41:41.756756] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-08. [2025-09-16T17:41:43.007521] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T17:41:43.018287] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-06. [2025-09-16T17:41:43.794851] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-08 (2.04 seconds). [2025-09-16T17:41:43.806302] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-09. [2025-09-16T17:41:44.740365] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-09-16T17:41:44.750680] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-07. [2025-09-16T17:41:45.501612] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T17:41:45.507827] [INFO ] [REMOTE ] CAVA: Full fetch complete (57819 records). [2025-09-16T17:41:45.507950] [DEBUG] [VALIDATOR ] CAVA: Validating primary data. [2025-09-16T17:41:45.508739] [FATAL] [VALIDATOR ] CAVA: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:41:45.508827] [FATAL] [MAIN ] CAVA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:41:45.508879] [WARN ] [MAIN ] CAVA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:41:45.508945] [INFO ] [LOCAL ] CAVA: Starting deletion of primary data. [2025-09-16T17:41:45.509029] [DEBUG] [LOCAL ] CAVA: No primary data file found to delete. [2025-09-16T17:41:45.509081] [INFO ] [LOCAL ] CAVA: Starting deletion of derivative data. [2025-09-16T17:41:45.509150] [DEBUG] [LOCAL ] CAVA: No derivative data file found to delete. [2025-09-16T17:41:45.509250] [INFO ] [MAIN ] Update Progress: 399/2369 symbols completed. CAVA took 23.2s. Estimated remaining: 3h 25m 55s. Rate: 9.6 symbols/min [2025-09-16T17:41:46.658812] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T17:41:46.670157] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-08. [2025-09-16T17:41:48.727131] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-09-16T17:41:48.738068] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-09. [2025-09-16T17:41:50.852798] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-09 (2.11 seconds). [2025-09-16T17:41:50.858879] [INFO ] [REMOTE ] CB: Full fetch complete (54403 records). [2025-09-16T17:41:50.859021] [DEBUG] [VALIDATOR ] CB: Validating primary data. [2025-09-16T17:41:51.233166] [DEBUG] [VALIDATOR ] CB: All expected trading days are present. [2025-09-16T17:41:51.233249] [DEBUG] [VALIDATOR ] CB: Primary data validation passed. [2025-09-16T17:41:51.233909] [INFO ] [MAIN ] CB: Fetched primary data is valid. [2025-09-16T17:41:51.233957] [DEBUG] [LOCAL ] CB: Saving primary data. [2025-09-16T17:41:51.235092] [DEBUG] [LOCAL ] CB: Trimmed 54403 records to 37022 (keeping 150 days of data) [2025-09-16T17:41:51.479624] [DEBUG] [LOCAL ] CB: Primary data saved (37022 records). [2025-09-16T17:41:51.479799] [INFO ] [MAIN ] CB: Full history saved. [2025-09-16T17:41:51.479896] [INFO ] [MAIN ] Update Progress: 400/2369 symbols completed. CB took 17.8s. Estimated remaining: 3h 25m 48s. Rate: 9.6 symbols/min [2025-09-16T17:41:52.092095] [INFO ] [MAIN ] CBO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:52.211187] [DEBUG] [LOCAL ] CBO: No existing data file found in primary storage. [2025-09-16T17:41:52.212102] [INFO ] [MAIN ] CBL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:52.212940] [INFO ] [MAIN ] CBAN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:52.208152] [INFO ] [MAIN ] CBNA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:41:52.213898] [WARN ] [MAIN ] CBO: No existing data found, performing full refresh instead. [2025-09-16T17:41:52.215182] [INFO ] [MAIN ] CBO: Wiping existing primary data before full refresh. [2025-09-16T17:41:52.215231] [INFO ] [LOCAL ] CBO: Starting deletion of primary data. [2025-09-16T17:41:52.215279] [DEBUG] [LOCAL ] CBO: No primary data file found to delete. [2025-09-16T17:41:52.215326] [INFO ] [MAIN ] CBO: Starting full primary refresh. [2025-09-16T17:41:52.308601] [DEBUG] [LOCAL ] CBNA: Loaded 2270 existing records from primary storage. [2025-09-16T17:41:52.317187] [INFO ] [MAIN ] CBNA: Loaded 2270 existing records. [2025-09-16T17:41:52.343853] [DEBUG] [LOCAL ] CBAN: Loaded 7399 existing records from primary storage. [2025-09-16T17:41:52.356541] [INFO ] [MAIN ] CBAN: Loaded 7399 existing records. [2025-09-16T17:41:52.380700] [DEBUG] [LOCAL ] CBL: Loaded 11888 existing records from primary storage. [2025-09-16T17:41:52.449432] [INFO ] [MAIN ] CBL: Loaded 11888 existing records. [2025-09-16T17:41:52.968610] [WARN ] [MAIN ] CBNA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:41:53.069573] [INFO ] [MAIN ] CBNA: Wiping existing primary data before full refresh. [2025-09-16T17:41:53.126930] [INFO ] [REMOTE ] CBO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:53.158684] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-02. [2025-09-16T17:41:53.151577] [WARN ] [MAIN ] CBAN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:41:53.151714] [INFO ] [LOCAL ] CBNA: Starting deletion of primary data. [2025-09-16T17:41:53.179175] [INFO ] [MAIN ] CBAN: Wiping existing primary data before full refresh. [2025-09-16T17:41:53.180534] [DEBUG] [LOCAL ] CBNA: Deleted primary data file. [2025-09-16T17:41:53.178925] [WARN ] [MAIN ] CBL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:41:53.180791] [INFO ] [LOCAL ] CBAN: Starting deletion of primary data. [2025-09-16T17:41:53.180975] [INFO ] [MAIN ] CBNA: Starting full primary refresh. [2025-09-16T17:41:53.180879] [INFO ] [MAIN ] CBL: Wiping existing primary data before full refresh. [2025-09-16T17:41:53.192043] [INFO ] [LOCAL ] CBL: Starting deletion of primary data. [2025-09-16T17:41:53.192110] [DEBUG] [LOCAL ] CBAN: Deleted primary data file. [2025-09-16T17:41:53.210488] [DEBUG] [LOCAL ] CBL: Deleted primary data file. [2025-09-16T17:41:53.222208] [INFO ] [MAIN ] CBAN: Starting full primary refresh. [2025-09-16T17:41:53.222035] [INFO ] [MAIN ] CBL: Starting full primary refresh. [2025-09-16T17:41:53.458250] [DEBUG] [REMOTE ] CBO: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:41:53.495592] [WARN ] [REMOTE ] CBO: No data returned for month 2025-02 [2025-09-16T17:41:53.499987] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-03. [2025-09-16T17:41:54.146227] [INFO ] [REMOTE ] CBO: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-09-16T17:41:54.184399] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-04. [2025-09-16T17:41:54.214670] [INFO ] [REMOTE ] CBL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:54.248721] [INFO ] [REMOTE ] CBNA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:54.265190] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-02. [2025-09-16T17:41:54.284173] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-02. [2025-09-16T17:41:54.304083] [INFO ] [REMOTE ] CBAN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:41:54.305395] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-02. [2025-09-16T17:41:54.464325] [INFO ] [REMOTE ] CBO: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-09-16T17:41:54.465236] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-05. [2025-09-16T17:41:54.865359] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-09-16T17:41:54.866632] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-03. [2025-09-16T17:41:55.052881] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T17:41:55.055872] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-03. [2025-09-16T17:41:55.160481] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-09-16T17:41:55.161849] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-04. [2025-09-16T17:41:55.169017] [INFO ] [REMOTE ] CBO: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-09-16T17:41:55.171207] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-06. [2025-09-16T17:41:55.185132] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-09-16T17:41:55.189607] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-03. [2025-09-16T17:41:55.451387] [INFO ] [REMOTE ] CBO: Incremental fetch complete for 2025-06 (0.28 seconds). [2025-09-16T17:41:55.574868] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-07. [2025-09-16T17:41:55.619038] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-09-16T17:41:56.023012] [INFO ] [REMOTE ] CBO: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-09-16T17:41:56.034333] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T17:41:56.062622] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-09-16T17:41:56.374921] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-05. [2025-09-16T17:41:56.910627] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-09-16T17:41:57.174912] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-04. [2025-09-16T17:41:57.974892] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-06. [2025-09-16T17:41:58.243133] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-09-16T17:41:58.529040] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-09-16T17:41:58.774904] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-07. [2025-09-16T17:41:59.208858] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-09-16T17:41:59.574906] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-04. [2025-09-16T17:42:00.374904] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-08. [2025-09-16T17:42:00.403619] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-09-16T17:42:01.031831] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-09-16T17:42:01.174898] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-09. [2025-09-16T17:42:01.767543] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-09-16T17:42:01.768687] [INFO ] [REMOTE ] CBNA: Full fetch complete (3176 records). [2025-09-16T17:42:01.768762] [DEBUG] [VALIDATOR ] CBNA: Validating primary data. [2025-09-16T17:42:01.984739] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-08. [2025-09-16T17:42:02.166414] [DEBUG] [VALIDATOR ] CBNA: All expected trading days are present. [2025-09-16T17:42:02.166502] [DEBUG] [VALIDATOR ] CBNA: Primary data validation passed. [2025-09-16T17:42:02.166618] [INFO ] [MAIN ] CBNA: Fetched primary data is valid. [2025-09-16T17:42:02.166674] [DEBUG] [LOCAL ] CBNA: Saving primary data. [2025-09-16T17:42:02.167048] [DEBUG] [LOCAL ] CBNA: Trimmed 3176 records to 2286 (keeping 150 days of data) [2025-09-16T17:42:02.181463] [DEBUG] [LOCAL ] CBNA: Primary data saved (2286 records). [2025-09-16T17:42:02.181585] [INFO ] [MAIN ] CBNA: Full history saved. [2025-09-16T17:42:02.181703] [INFO ] [MAIN ] Update Progress: 401/2369 symbols completed. CBNA took 10.7s. Estimated remaining: 3h 26m 3s. Rate: 9.6 symbols/min [2025-09-16T17:42:02.291620] [DEBUG] [REMOTE ] CBO: Error fetching primary data for 2025-08: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:42:02.313043] [WARN ] [REMOTE ] CBO: No data returned for month 2025-08 [2025-09-16T17:42:02.367496] [INFO ] [MAIN ] CBRE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:02.408804] [DEBUG] [LOCAL ] CBRE: Loaded 37044 existing records from primary storage. [2025-09-16T17:42:02.408953] [INFO ] [MAIN ] CBRE: Loaded 37044 existing records. [2025-09-16T17:42:02.590769] [WARN ] [MAIN ] CBRE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:42:02.591025] [INFO ] [MAIN ] CBRE: Wiping existing primary data before full refresh. [2025-09-16T17:42:02.591071] [INFO ] [LOCAL ] CBRE: Starting deletion of primary data. [2025-09-16T17:42:02.591974] [DEBUG] [LOCAL ] CBRE: Deleted primary data file. [2025-09-16T17:42:02.592024] [INFO ] [MAIN ] CBRE: Starting full primary refresh. [2025-09-16T17:42:02.783144] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-05. [2025-09-16T17:42:02.954884] [INFO ] [REMOTE ] CBRE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:03.574946] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-09. [2025-09-16T17:42:03.697512] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T17:42:03.941223] [DEBUG] [REMOTE ] CBO: Error fetching primary data for 2025-09: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:42:03.941736] [WARN ] [REMOTE ] CBO: No data returned for month 2025-09 [2025-09-16T17:42:03.941839] [INFO ] [REMOTE ] CBO: Full fetch complete (1016 records). [2025-09-16T17:42:03.941959] [DEBUG] [VALIDATOR ] CBO: Validating primary data. [2025-09-16T17:42:03.942791] [FATAL] [VALIDATOR ] CBO: Data is stale - most recent date (2025-07-10) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:42:03.942866] [FATAL] [MAIN ] CBO: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:42:03.942903] [WARN ] [MAIN ] CBO: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:42:03.942961] [INFO ] [LOCAL ] CBO: Starting deletion of primary data. [2025-09-16T17:42:03.943042] [DEBUG] [LOCAL ] CBO: No primary data file found to delete. [2025-09-16T17:42:03.943082] [INFO ] [LOCAL ] CBO: Starting deletion of derivative data. [2025-09-16T17:42:03.943132] [DEBUG] [LOCAL ] CBO: No derivative data file found to delete. [2025-09-16T17:42:03.943202] [INFO ] [MAIN ] Update Progress: 402/2369 symbols completed. CBO took 12.5s. Estimated remaining: 3h 25m 35s. Rate: 9.6 symbols/min [2025-09-16T17:42:04.127129] [INFO ] [MAIN ] CBT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:04.152943] [DEBUG] [LOCAL ] CBT: Loaded 22706 existing records from primary storage. [2025-09-16T17:42:04.153142] [INFO ] [MAIN ] CBT: Loaded 22706 existing records. [2025-09-16T17:42:04.347628] [WARN ] [MAIN ] CBT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:42:04.347963] [INFO ] [MAIN ] CBT: Wiping existing primary data before full refresh. [2025-09-16T17:42:04.348016] [INFO ] [LOCAL ] CBT: Starting deletion of primary data. [2025-09-16T17:42:04.348725] [DEBUG] [LOCAL ] CBT: Deleted primary data file. [2025-09-16T17:42:04.348786] [INFO ] [MAIN ] CBT: Starting full primary refresh. [2025-09-16T17:42:04.380568] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-05. [2025-09-16T17:42:04.736549] [INFO ] [REMOTE ] CBT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:05.174886] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-06. [2025-09-16T17:42:05.372634] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T17:42:05.974908] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-06. [2025-09-16T17:42:06.253155] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T17:42:06.486537] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-09-16T17:42:06.774892] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-07. [2025-09-16T17:42:07.574864] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-02. [2025-09-16T17:42:07.597764] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-09-16T17:42:08.374909] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-02. [2025-09-16T17:42:09.064242] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T17:42:09.174953] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-03. [2025-09-16T17:42:09.974914] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-08. [2025-09-16T17:42:10.125147] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T17:42:10.590273] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T17:42:10.749032] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-09-16T17:42:10.774852] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-09. [2025-09-16T17:42:11.574918] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-04. [2025-09-16T17:42:11.599027] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-09-16T17:42:11.600548] [INFO ] [REMOTE ] CBAN: Full fetch complete (12304 records). [2025-09-16T17:42:11.600658] [DEBUG] [VALIDATOR ] CBAN: Validating primary data. [2025-09-16T17:42:11.972786] [DEBUG] [VALIDATOR ] CBAN: All expected trading days are present. [2025-09-16T17:42:11.972945] [DEBUG] [VALIDATOR ] CBAN: Primary data validation passed. [2025-09-16T17:42:11.973275] [INFO ] [MAIN ] CBAN: Fetched primary data is valid. [2025-09-16T17:42:11.973323] [DEBUG] [LOCAL ] CBAN: Saving primary data. [2025-09-16T17:42:11.974040] [DEBUG] [LOCAL ] CBAN: Trimmed 12304 records to 7457 (keeping 150 days of data) [2025-09-16T17:42:12.025025] [DEBUG] [LOCAL ] CBAN: Primary data saved (7457 records). [2025-09-16T17:42:12.025179] [INFO ] [MAIN ] CBAN: Full history saved. [2025-09-16T17:42:12.025281] [INFO ] [MAIN ] Update Progress: 403/2369 symbols completed. CBAN took 20.5s. Estimated remaining: 3h 25m 37s. Rate: 9.6 symbols/min [2025-09-16T17:42:12.199191] [INFO ] [MAIN ] CBU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:12.219303] [DEBUG] [LOCAL ] CBU: Loaded 16204 existing records from primary storage. [2025-09-16T17:42:12.219490] [INFO ] [MAIN ] CBU: Loaded 16204 existing records. [2025-09-16T17:42:12.375935] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-03. [2025-09-16T17:42:12.394403] [WARN ] [MAIN ] CBU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:42:12.395541] [INFO ] [MAIN ] CBU: Wiping existing primary data before full refresh. [2025-09-16T17:42:12.395590] [INFO ] [LOCAL ] CBU: Starting deletion of primary data. [2025-09-16T17:42:12.396126] [DEBUG] [LOCAL ] CBU: Deleted primary data file. [2025-09-16T17:42:12.396242] [INFO ] [MAIN ] CBU: Starting full primary refresh. [2025-09-16T17:42:12.746272] [INFO ] [REMOTE ] CBU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:12.794908] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-09-16T17:42:13.174898] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-02. [2025-09-16T17:42:13.930766] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T17:42:13.953263] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-09-16T17:42:13.974855] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-03. [2025-09-16T17:42:14.774902] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-04. [2025-09-16T17:42:15.166281] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-09-16T17:42:15.574930] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-07. [2025-09-16T17:42:16.374883] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-05. [2025-09-16T17:42:16.627817] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-09-16T17:42:16.668031] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T17:42:17.174947] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-08. [2025-09-16T17:42:17.588882] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-09-16T17:42:17.783049] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-09-16T17:42:17.974899] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-06. [2025-09-16T17:42:18.774915] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-05. [2025-09-16T17:42:19.192856] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-09-16T17:42:19.574912] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-04. [2025-09-16T17:42:20.374873] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-07. [2025-09-16T17:42:20.766454] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:42:20.804356] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-09-16T17:42:21.174950] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-06. [2025-09-16T17:42:21.509041] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-09-16T17:42:21.974952] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-09. [2025-09-16T17:42:22.396379] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-09-16T17:42:22.774960] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-05. [2025-09-16T17:42:22.806743] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-09-16T17:42:22.809050] [INFO ] [REMOTE ] CBL: Full fetch complete (18646 records). [2025-09-16T17:42:22.809168] [DEBUG] [VALIDATOR ] CBL: Validating primary data. [2025-09-16T17:42:23.180164] [DEBUG] [VALIDATOR ] CBL: All expected trading days are present. [2025-09-16T17:42:23.180248] [DEBUG] [VALIDATOR ] CBL: Primary data validation passed. [2025-09-16T17:42:23.180605] [INFO ] [MAIN ] CBL: Fetched primary data is valid. [2025-09-16T17:42:23.180662] [DEBUG] [LOCAL ] CBL: Saving primary data. [2025-09-16T17:42:23.181268] [DEBUG] [LOCAL ] CBL: Trimmed 18646 records to 11971 (keeping 150 days of data) [2025-09-16T17:42:23.259340] [DEBUG] [LOCAL ] CBL: Primary data saved (11971 records). [2025-09-16T17:42:23.259518] [INFO ] [MAIN ] CBL: Full history saved. [2025-09-16T17:42:23.259618] [INFO ] [MAIN ] Update Progress: 404/2369 symbols completed. CBL took 31.8s. Estimated remaining: 3h 25m 55s. Rate: 9.5 symbols/min [2025-09-16T17:42:23.439899] [INFO ] [MAIN ] CBX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:23.440133] [DEBUG] [LOCAL ] CBX: No existing data file found in primary storage. [2025-09-16T17:42:23.440186] [WARN ] [MAIN ] CBX: No existing data found, performing full refresh instead. [2025-09-16T17:42:23.440325] [INFO ] [MAIN ] CBX: Wiping existing primary data before full refresh. [2025-09-16T17:42:23.440370] [INFO ] [LOCAL ] CBX: Starting deletion of primary data. [2025-09-16T17:42:23.440414] [DEBUG] [LOCAL ] CBX: No primary data file found to delete. [2025-09-16T17:42:23.440447] [INFO ] [MAIN ] CBX: Starting full primary refresh. [2025-09-16T17:42:23.588048] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-08. [2025-09-16T17:42:23.859780] [INFO ] [REMOTE ] CBX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:23.869675] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-09-16T17:42:24.374949] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-07. [2025-09-16T17:42:24.769743] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T17:42:25.174910] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-02. [2025-09-16T17:42:25.515576] [DEBUG] [REMOTE ] CBX: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:42:25.515923] [WARN ] [REMOTE ] CBX: No data returned for month 2025-02 [2025-09-16T17:42:25.977560] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-09. [2025-09-16T17:42:25.983663] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-09-16T17:42:26.774883] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-08. [2025-09-16T17:42:26.840711] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T17:42:26.844483] [INFO ] [REMOTE ] CBT: Full fetch complete (34453 records). [2025-09-16T17:42:26.844582] [DEBUG] [VALIDATOR ] CBT: Validating primary data. [2025-09-16T17:42:27.211338] [DEBUG] [VALIDATOR ] CBT: All expected trading days are present. [2025-09-16T17:42:27.211416] [DEBUG] [VALIDATOR ] CBT: Primary data validation passed. [2025-09-16T17:42:27.211920] [INFO ] [MAIN ] CBT: Fetched primary data is valid. [2025-09-16T17:42:27.211999] [DEBUG] [LOCAL ] CBT: Saving primary data. [2025-09-16T17:42:27.212787] [DEBUG] [LOCAL ] CBT: Trimmed 34453 records to 22881 (keeping 150 days of data) [2025-09-16T17:42:27.356163] [DEBUG] [LOCAL ] CBT: Primary data saved (22881 records). [2025-09-16T17:42:27.356295] [INFO ] [MAIN ] CBT: Full history saved. [2025-09-16T17:42:27.356379] [INFO ] [MAIN ] Update Progress: 405/2369 symbols completed. CBT took 23.4s. Estimated remaining: 3h 25m 38s. Rate: 9.6 symbols/min [2025-09-16T17:42:27.527016] [INFO ] [MAIN ] CBZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:27.549676] [DEBUG] [LOCAL ] CBZ: Loaded 24586 existing records from primary storage. [2025-09-16T17:42:27.549801] [INFO ] [MAIN ] CBZ: Loaded 24586 existing records. [2025-09-16T17:42:27.580564] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-03. [2025-09-16T17:42:27.734821] [WARN ] [MAIN ] CBZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:42:27.735037] [INFO ] [MAIN ] CBZ: Wiping existing primary data before full refresh. [2025-09-16T17:42:27.735092] [INFO ] [LOCAL ] CBZ: Starting deletion of primary data. [2025-09-16T17:42:27.735765] [DEBUG] [LOCAL ] CBZ: Deleted primary data file. [2025-09-16T17:42:27.735821] [INFO ] [MAIN ] CBZ: Starting full primary refresh. [2025-09-16T17:42:27.896734] [DEBUG] [REMOTE ] CBX: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:42:27.961930] [WARN ] [REMOTE ] CBX: No data returned for month 2025-03 [2025-09-16T17:42:28.102370] [INFO ] [REMOTE ] CBZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:28.247286] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-09-16T17:42:28.374858] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-09. [2025-09-16T17:42:29.174906] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-02. [2025-09-16T17:42:29.618901] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T17:42:29.625039] [INFO ] [REMOTE ] CBRE: Full fetch complete (55221 records). [2025-09-16T17:42:29.625149] [DEBUG] [VALIDATOR ] CBRE: Validating primary data. [2025-09-16T17:42:29.980051] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-06. [2025-09-16T17:42:30.021108] [DEBUG] [VALIDATOR ] CBRE: All expected trading days are present. [2025-09-16T17:42:30.021194] [DEBUG] [VALIDATOR ] CBRE: Primary data validation passed. [2025-09-16T17:42:30.021870] [INFO ] [MAIN ] CBRE: Fetched primary data is valid. [2025-09-16T17:42:30.021915] [DEBUG] [LOCAL ] CBRE: Saving primary data. [2025-09-16T17:42:30.023032] [DEBUG] [LOCAL ] CBRE: Trimmed 55221 records to 37405 (keeping 150 days of data) [2025-09-16T17:42:30.255259] [DEBUG] [LOCAL ] CBRE: Primary data saved (37405 records). [2025-09-16T17:42:30.255497] [INFO ] [MAIN ] CBRE: Full history saved. [2025-09-16T17:42:30.257564] [INFO ] [MAIN ] Update Progress: 406/2369 symbols completed. CBRE took 28.1s. Estimated remaining: 3h 25m 16s. Rate: 9.6 symbols/min [2025-09-16T17:42:30.364387] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T17:42:30.449314] [INFO ] [MAIN ] CC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:30.492753] [DEBUG] [LOCAL ] CC: Loaded 39175 existing records from primary storage. [2025-09-16T17:42:30.492891] [INFO ] [MAIN ] CC: Loaded 39175 existing records. [2025-09-16T17:42:30.666714] [WARN ] [MAIN ] CC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:42:30.666966] [INFO ] [MAIN ] CC: Wiping existing primary data before full refresh. [2025-09-16T17:42:30.667031] [INFO ] [LOCAL ] CC: Starting deletion of primary data. [2025-09-16T17:42:30.667947] [DEBUG] [LOCAL ] CC: Deleted primary data file. [2025-09-16T17:42:30.667996] [INFO ] [MAIN ] CC: Starting full primary refresh. [2025-09-16T17:42:30.780026] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-04. [2025-09-16T17:42:31.021994] [INFO ] [REMOTE ] CC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:31.033667] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T17:42:31.263699] [DEBUG] [REMOTE ] CBX: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:42:31.264019] [WARN ] [REMOTE ] CBX: No data returned for month 2025-04 [2025-09-16T17:42:31.574905] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-02. [2025-09-16T17:42:32.374892] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-03. [2025-09-16T17:42:32.888997] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T17:42:33.175001] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-07. [2025-09-16T17:42:33.356821] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T17:42:33.960275] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T17:42:33.974877] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-08. [2025-09-16T17:42:34.774915] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-04. [2025-09-16T17:42:34.933606] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-09-16T17:42:35.574899] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-03. [2025-09-16T17:42:35.865260] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T17:42:36.374899] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-05. [2025-09-16T17:42:37.103467] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T17:42:37.174853] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-04. [2025-09-16T17:42:37.273487] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-09-16T17:42:37.974913] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-05. [2025-09-16T17:42:38.508349] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T17:42:38.669122] [INFO ] [REMOTE ] CBX: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-09-16T17:42:38.774888] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-05. [2025-09-16T17:42:39.574949] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-06. [2025-09-16T17:42:39.803770] [INFO ] [REMOTE ] CBX: Incremental fetch complete for 2025-06 (0.23 seconds). [2025-09-16T17:42:40.106247] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T17:42:40.374903] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-06. [2025-09-16T17:42:41.175460] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-09. [2025-09-16T17:42:41.974963] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-07. [2025-09-16T17:42:42.135100] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T17:42:42.183131] [INFO ] [REMOTE ] CBX: Incremental fetch complete for 2025-07 (0.21 seconds). [2025-09-16T17:42:42.491134] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T17:42:42.495260] [INFO ] [REMOTE ] CBU: Full fetch complete (24372 records). [2025-09-16T17:42:42.495394] [DEBUG] [VALIDATOR ] CBU: Validating primary data. [2025-09-16T17:42:42.780120] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-07. [2025-09-16T17:42:42.934201] [DEBUG] [VALIDATOR ] CBU: All expected trading days are present. [2025-09-16T17:42:42.934390] [DEBUG] [VALIDATOR ] CBU: Primary data validation passed. [2025-09-16T17:42:42.935027] [INFO ] [MAIN ] CBU: Fetched primary data is valid. [2025-09-16T17:42:42.935082] [DEBUG] [LOCAL ] CBU: Saving primary data. [2025-09-16T17:42:42.936318] [DEBUG] [LOCAL ] CBU: Trimmed 24372 records to 16385 (keeping 150 days of data) [2025-09-16T17:42:43.041406] [DEBUG] [LOCAL ] CBU: Primary data saved (16385 records). [2025-09-16T17:42:43.041582] [INFO ] [MAIN ] CBU: Full history saved. [2025-09-16T17:42:43.041705] [INFO ] [MAIN ] Update Progress: 407/2369 symbols completed. CBU took 31.0s. Estimated remaining: 3h 25m 41s. Rate: 9.5 symbols/min [2025-09-16T17:42:43.219250] [INFO ] [MAIN ] CCI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:43.262724] [DEBUG] [LOCAL ] CCI: Loaded 39172 existing records from primary storage. [2025-09-16T17:42:43.262887] [INFO ] [MAIN ] CCI: Loaded 39172 existing records. [2025-09-16T17:42:43.450842] [WARN ] [MAIN ] CCI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:42:43.451080] [INFO ] [MAIN ] CCI: Wiping existing primary data before full refresh. [2025-09-16T17:42:43.451139] [INFO ] [LOCAL ] CCI: Starting deletion of primary data. [2025-09-16T17:42:43.452120] [DEBUG] [LOCAL ] CCI: Deleted primary data file. [2025-09-16T17:42:43.452169] [INFO ] [MAIN ] CCI: Starting full primary refresh. [2025-09-16T17:42:43.580057] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-08. [2025-09-16T17:42:43.818840] [INFO ] [REMOTE ] CCI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:43.888357] [DEBUG] [REMOTE ] CBX: Error fetching primary data for 2025-08: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:42:43.888725] [WARN ] [REMOTE ] CBX: No data returned for month 2025-08 [2025-09-16T17:42:44.374881] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-02. [2025-09-16T17:42:44.480470] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T17:42:45.174941] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-09. [2025-09-16T17:42:45.644694] [DEBUG] [REMOTE ] CBX: Error fetching primary data for 2025-09: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:42:45.645008] [WARN ] [REMOTE ] CBX: No data returned for month 2025-09 [2025-09-16T17:42:45.645091] [INFO ] [REMOTE ] CBX: Full fetch complete (1040 records). [2025-09-16T17:42:45.645156] [DEBUG] [VALIDATOR ] CBX: Validating primary data. [2025-09-16T17:42:45.645772] [FATAL] [VALIDATOR ] CBX: Data is stale - most recent date (2025-07-02) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:42:45.645843] [FATAL] [MAIN ] CBX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:42:45.645880] [WARN ] [MAIN ] CBX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:42:45.645924] [INFO ] [LOCAL ] CBX: Starting deletion of primary data. [2025-09-16T17:42:45.645980] [DEBUG] [LOCAL ] CBX: No primary data file found to delete. [2025-09-16T17:42:45.646014] [INFO ] [LOCAL ] CBX: Starting deletion of derivative data. [2025-09-16T17:42:45.646054] [DEBUG] [LOCAL ] CBX: No derivative data file found to delete. [2025-09-16T17:42:45.646111] [INFO ] [MAIN ] Update Progress: 408/2369 symbols completed. CBX took 22.4s. Estimated remaining: 3h 25m 17s. Rate: 9.6 symbols/min [2025-09-16T17:42:45.821517] [INFO ] [MAIN ] CCIA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:45.821775] [DEBUG] [LOCAL ] CCIA: No existing data file found in primary storage. [2025-09-16T17:42:45.821960] [WARN ] [MAIN ] CCIA: No existing data found, performing full refresh instead. [2025-09-16T17:42:45.822157] [INFO ] [MAIN ] CCIA: Wiping existing primary data before full refresh. [2025-09-16T17:42:45.822232] [INFO ] [LOCAL ] CCIA: Starting deletion of primary data. [2025-09-16T17:42:45.822285] [DEBUG] [LOCAL ] CCIA: No primary data file found to delete. [2025-09-16T17:42:45.822375] [INFO ] [MAIN ] CCIA: Starting full primary refresh. [2025-09-16T17:42:45.979345] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-08. [2025-09-16T17:42:46.218682] [INFO ] [REMOTE ] CCIA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:46.221094] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-09-16T17:42:46.774918] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-06. [2025-09-16T17:42:47.479933] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T17:42:47.574923] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-02. [2025-09-16T17:42:47.789187] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T17:42:47.913114] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-09-16T17:42:48.374923] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-03. [2025-09-16T17:42:48.685840] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-09-16T17:42:49.174919] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-03. [2025-09-16T17:42:49.974914] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-09. [2025-09-16T17:42:50.564529] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T17:42:50.774936] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-04. [2025-09-16T17:42:51.574915] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-07. [2025-09-16T17:42:51.622532] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-09-16T17:42:51.629104] [INFO ] [REMOTE ] CC: Full fetch complete (57587 records). [2025-09-16T17:42:51.629229] [DEBUG] [VALIDATOR ] CC: Validating primary data. [2025-09-16T17:42:52.006365] [DEBUG] [VALIDATOR ] CC: All expected trading days are present. [2025-09-16T17:42:52.006449] [DEBUG] [VALIDATOR ] CC: Primary data validation passed. [2025-09-16T17:42:52.007177] [INFO ] [MAIN ] CC: Fetched primary data is valid. [2025-09-16T17:42:52.007223] [DEBUG] [LOCAL ] CC: Saving primary data. [2025-09-16T17:42:52.008420] [DEBUG] [LOCAL ] CC: Trimmed 57587 records to 39560 (keeping 150 days of data) [2025-09-16T17:42:52.266530] [DEBUG] [LOCAL ] CC: Primary data saved (39560 records). [2025-09-16T17:42:52.266811] [INFO ] [MAIN ] CC: Full history saved. [2025-09-16T17:42:52.266917] [INFO ] [MAIN ] Update Progress: 409/2369 symbols completed. CC took 22.0s. Estimated remaining: 3h 25m 12s. Rate: 9.6 symbols/min [2025-09-16T17:42:52.382603] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-04. [2025-09-16T17:42:52.478332] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-09-16T17:42:52.480783] [INFO ] [MAIN ] CCIF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:52.503681] [DEBUG] [LOCAL ] CCIF: Loaded 11217 existing records from primary storage. [2025-09-16T17:42:52.503827] [INFO ] [MAIN ] CCIF: Loaded 11217 existing records. [2025-09-16T17:42:52.678603] [WARN ] [MAIN ] CCIF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:42:52.678880] [INFO ] [MAIN ] CCIF: Wiping existing primary data before full refresh. [2025-09-16T17:42:52.678927] [INFO ] [LOCAL ] CCIF: Starting deletion of primary data. [2025-09-16T17:42:52.679320] [DEBUG] [LOCAL ] CCIF: Deleted primary data file. [2025-09-16T17:42:52.679379] [INFO ] [MAIN ] CCIF: Starting full primary refresh. [2025-09-16T17:42:52.767420] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-09-16T17:42:53.049512] [INFO ] [REMOTE ] CCIF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:53.072224] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T17:42:53.174903] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-08. [2025-09-16T17:42:53.974904] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-02. [2025-09-16T17:42:54.380755] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-09-16T17:42:54.774892] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-09. [2025-09-16T17:42:55.009202] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-09-16T17:42:55.574905] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-05. [2025-09-16T17:42:55.896381] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-09-16T17:42:55.901495] [INFO ] [REMOTE ] CBZ: Full fetch complete (35525 records). [2025-09-16T17:42:55.901595] [DEBUG] [VALIDATOR ] CBZ: Validating primary data. [2025-09-16T17:42:56.198875] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-09-16T17:42:56.295976] [DEBUG] [VALIDATOR ] CBZ: All expected trading days are present. [2025-09-16T17:42:56.296069] [DEBUG] [VALIDATOR ] CBZ: Primary data validation passed. [2025-09-16T17:42:56.296548] [INFO ] [MAIN ] CBZ: Fetched primary data is valid. [2025-09-16T17:42:56.296591] [DEBUG] [LOCAL ] CBZ: Saving primary data. [2025-09-16T17:42:56.297580] [DEBUG] [LOCAL ] CBZ: Trimmed 35525 records to 24900 (keeping 150 days of data) [2025-09-16T17:42:56.384288] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-06. [2025-09-16T17:42:56.448505] [DEBUG] [LOCAL ] CBZ: Primary data saved (24900 records). [2025-09-16T17:42:56.448653] [INFO ] [MAIN ] CBZ: Full history saved. [2025-09-16T17:42:56.448741] [INFO ] [MAIN ] Update Progress: 410/2369 symbols completed. CBZ took 29.1s. Estimated remaining: 3h 24m 56s. Rate: 9.6 symbols/min [2025-09-16T17:42:56.618789] [INFO ] [MAIN ] CCJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:42:56.657413] [DEBUG] [LOCAL ] CCJ: Loaded 39663 existing records from primary storage. [2025-09-16T17:42:56.657560] [INFO ] [MAIN ] CCJ: Loaded 39663 existing records. [2025-09-16T17:42:56.763384] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-09-16T17:42:56.839417] [WARN ] [MAIN ] CCJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:42:56.839660] [INFO ] [MAIN ] CCJ: Wiping existing primary data before full refresh. [2025-09-16T17:42:56.839708] [INFO ] [LOCAL ] CCJ: Starting deletion of primary data. [2025-09-16T17:42:56.840595] [DEBUG] [LOCAL ] CCJ: Deleted primary data file. [2025-09-16T17:42:56.840667] [INFO ] [MAIN ] CCJ: Starting full primary refresh. [2025-09-16T17:42:57.178030] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-03. [2025-09-16T17:42:57.198633] [INFO ] [REMOTE ] CCJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:42:57.967150] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T17:42:57.974856] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-04. [2025-09-16T17:42:58.774876] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-05. [2025-09-16T17:42:58.874291] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-09-16T17:42:59.574913] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-05. [2025-09-16T17:43:00.348730] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T17:43:00.374843] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-06. [2025-09-16T17:43:00.890685] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T17:43:01.174932] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-07. [2025-09-16T17:43:01.720688] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-09-16T17:43:01.916736] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-09-16T17:43:01.974909] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-07. [2025-09-16T17:43:02.774907] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-02. [2025-09-16T17:43:03.574904] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-08. [2025-09-16T17:43:03.799986] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-09-16T17:43:03.939064] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-09-16T17:43:04.374889] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-06. [2025-09-16T17:43:05.014380] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-09-16T17:43:05.174858] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-03. [2025-09-16T17:43:05.433979] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-09-16T17:43:05.974905] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-07. [2025-09-16T17:43:06.722874] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-09-16T17:43:06.774893] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-08. [2025-09-16T17:43:07.107500] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-09-16T17:43:07.574905] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-08. [2025-09-16T17:43:07.772159] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-09-16T17:43:08.374893] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-09. [2025-09-16T17:43:09.174881] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-04. [2025-09-16T17:43:09.226491] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T17:43:09.228759] [INFO ] [REMOTE ] CCIF: Full fetch complete (16495 records). [2025-09-16T17:43:09.228834] [DEBUG] [VALIDATOR ] CCIF: Validating primary data. [2025-09-16T17:43:09.619617] [DEBUG] [VALIDATOR ] CCIF: All expected trading days are present. [2025-09-16T17:43:09.634103] [DEBUG] [VALIDATOR ] CCIF: Primary data validation passed. [2025-09-16T17:43:09.634561] [INFO ] [MAIN ] CCIF: Fetched primary data is valid. [2025-09-16T17:43:09.634682] [DEBUG] [LOCAL ] CCIF: Saving primary data. [2025-09-16T17:43:09.637144] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-09-16T17:43:09.637571] [DEBUG] [LOCAL ] CCIF: Trimmed 16495 records to 11293 (keeping 150 days of data) [2025-09-16T17:43:09.714811] [DEBUG] [LOCAL ] CCIF: Primary data saved (11293 records). [2025-09-16T17:43:09.714951] [INFO ] [MAIN ] CCIF: Full history saved. [2025-09-16T17:43:09.715035] [INFO ] [MAIN ] Update Progress: 411/2369 symbols completed. CCIF took 17.4s. Estimated remaining: 3h 25m 23s. Rate: 9.5 symbols/min [2025-09-16T17:43:09.892659] [INFO ] [MAIN ] CCK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:09.929335] [DEBUG] [LOCAL ] CCK: Loaded 34145 existing records from primary storage. [2025-09-16T17:43:09.929568] [INFO ] [MAIN ] CCK: Loaded 34145 existing records. [2025-09-16T17:43:09.984712] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-09. [2025-09-16T17:43:10.107279] [WARN ] [MAIN ] CCK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:43:10.107622] [INFO ] [MAIN ] CCK: Wiping existing primary data before full refresh. [2025-09-16T17:43:10.107687] [INFO ] [LOCAL ] CCK: Starting deletion of primary data. [2025-09-16T17:43:10.108750] [DEBUG] [LOCAL ] CCK: Deleted primary data file. [2025-09-16T17:43:10.108800] [INFO ] [MAIN ] CCK: Starting full primary refresh. [2025-09-16T17:43:10.486310] [INFO ] [REMOTE ] CCK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:43:10.493697] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-09-16T17:43:10.494546] [INFO ] [REMOTE ] CCIA: Full fetch complete (1556 records). [2025-09-16T17:43:10.494613] [DEBUG] [VALIDATOR ] CCIA: Validating primary data. [2025-09-16T17:43:10.670365] [WARN ] [VALIDATOR ] CCIA: Validation failed - insufficient historical data. Have 148 unique days, need 150 days [2025-09-16T17:43:10.670562] [FATAL] [MAIN ] CCIA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:43:10.670612] [WARN ] [MAIN ] CCIA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:43:10.670671] [INFO ] [LOCAL ] CCIA: Starting deletion of primary data. [2025-09-16T17:43:10.670735] [DEBUG] [LOCAL ] CCIA: No primary data file found to delete. [2025-09-16T17:43:10.670773] [INFO ] [LOCAL ] CCIA: Starting deletion of derivative data. [2025-09-16T17:43:10.670824] [DEBUG] [LOCAL ] CCIA: No derivative data file found to delete. [2025-09-16T17:43:10.670883] [INFO ] [MAIN ] Update Progress: 412/2369 symbols completed. CCIA took 25.0s. Estimated remaining: 3h 24m 51s. Rate: 9.6 symbols/min [2025-09-16T17:43:10.775500] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-02. [2025-09-16T17:43:10.840313] [INFO ] [MAIN ] CCL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:10.893469] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T17:43:10.916867] [DEBUG] [LOCAL ] CCL: Loaded 39760 existing records from primary storage. [2025-09-16T17:43:10.916998] [INFO ] [MAIN ] CCL: Loaded 39760 existing records. [2025-09-16T17:43:11.104945] [WARN ] [MAIN ] CCL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:43:11.105222] [INFO ] [MAIN ] CCL: Wiping existing primary data before full refresh. [2025-09-16T17:43:11.105269] [INFO ] [LOCAL ] CCL: Starting deletion of primary data. [2025-09-16T17:43:11.106137] [DEBUG] [LOCAL ] CCL: Deleted primary data file. [2025-09-16T17:43:11.106187] [INFO ] [MAIN ] CCL: Starting full primary refresh. [2025-09-16T17:43:11.472439] [INFO ] [REMOTE ] CCL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:43:11.574899] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-02. [2025-09-16T17:43:12.374905] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-09. [2025-09-16T17:43:12.575289] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-09-16T17:43:13.174917] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-05. [2025-09-16T17:43:13.888884] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-09-16T17:43:13.895191] [INFO ] [REMOTE ] CCI: Full fetch complete (57804 records). [2025-09-16T17:43:13.895296] [DEBUG] [VALIDATOR ] CCI: Validating primary data. [2025-09-16T17:43:13.977572] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-03. [2025-09-16T17:43:14.290683] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-02 (2.72 seconds). [2025-09-16T17:43:14.376443] [DEBUG] [VALIDATOR ] CCI: All expected trading days are present. [2025-09-16T17:43:14.376525] [DEBUG] [VALIDATOR ] CCI: Primary data validation passed. [2025-09-16T17:43:14.377186] [INFO ] [MAIN ] CCI: Fetched primary data is valid. [2025-09-16T17:43:14.377261] [DEBUG] [LOCAL ] CCI: Saving primary data. [2025-09-16T17:43:14.378902] [DEBUG] [LOCAL ] CCI: Trimmed 57804 records to 39559 (keeping 150 days of data) [2025-09-16T17:43:14.630944] [DEBUG] [LOCAL ] CCI: Primary data saved (39559 records). [2025-09-16T17:43:14.631117] [INFO ] [MAIN ] CCI: Full history saved. [2025-09-16T17:43:14.631214] [INFO ] [MAIN ] Update Progress: 413/2369 symbols completed. CCI took 31.6s. Estimated remaining: 3h 24m 34s. Rate: 9.6 symbols/min [2025-09-16T17:43:14.775281] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-03. [2025-09-16T17:43:14.808828] [INFO ] [MAIN ] CCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:14.844331] [DEBUG] [LOCAL ] CCO: Loaded 35835 existing records from primary storage. [2025-09-16T17:43:14.844518] [INFO ] [MAIN ] CCO: Loaded 35835 existing records. [2025-09-16T17:43:15.043438] [WARN ] [MAIN ] CCO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:43:15.044032] [INFO ] [MAIN ] CCO: Wiping existing primary data before full refresh. [2025-09-16T17:43:15.044112] [INFO ] [LOCAL ] CCO: Starting deletion of primary data. [2025-09-16T17:43:15.045015] [DEBUG] [LOCAL ] CCO: Deleted primary data file. [2025-09-16T17:43:15.045074] [INFO ] [MAIN ] CCO: Starting full primary refresh. [2025-09-16T17:43:15.416960] [INFO ] [REMOTE ] CCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:43:15.435133] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-05 (2.26 seconds). [2025-09-16T17:43:15.574906] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-02. [2025-09-16T17:43:15.675797] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-09-16T17:43:16.374957] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-06. [2025-09-16T17:43:16.806544] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-09-16T17:43:16.964290] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-03 (2.19 seconds). [2025-09-16T17:43:17.174902] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-03. [2025-09-16T17:43:17.974904] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-04. [2025-09-16T17:43:18.516065] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-06 (2.14 seconds). [2025-09-16T17:43:18.614000] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T17:43:18.774853] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-04. [2025-09-16T17:43:19.574913] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-07. [2025-09-16T17:43:19.665148] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T17:43:19.968023] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:43:20.374898] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-05. [2025-09-16T17:43:21.174901] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-04. [2025-09-16T17:43:21.677617] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-09-16T17:43:21.974924] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-08. [2025-09-16T17:43:22.061458] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-09-16T17:43:22.774922] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-05. [2025-09-16T17:43:23.532462] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-04 (2.36 seconds). [2025-09-16T17:43:23.574876] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-05. [2025-09-16T17:43:23.975353] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-09-16T17:43:24.294038] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T17:43:24.374893] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-06. [2025-09-16T17:43:25.174914] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-06. [2025-09-16T17:43:25.746206] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-09-16T17:43:25.849133] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-09-16T17:43:25.974907] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-06. [2025-09-16T17:43:26.666354] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T17:43:26.774904] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-09. [2025-09-16T17:43:27.574928] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-07. [2025-09-16T17:43:27.923974] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-09-16T17:43:28.374962] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-07. [2025-09-16T17:43:28.553164] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T17:43:28.559360] [INFO ] [REMOTE ] CCJ: Full fetch complete (58354 records). [2025-09-16T17:43:28.559467] [DEBUG] [VALIDATOR ] CCJ: Validating primary data. [2025-09-16T17:43:28.949439] [DEBUG] [VALIDATOR ] CCJ: All expected trading days are present. [2025-09-16T17:43:28.949519] [DEBUG] [VALIDATOR ] CCJ: Primary data validation passed. [2025-09-16T17:43:28.950197] [INFO ] [MAIN ] CCJ: Fetched primary data is valid. [2025-09-16T17:43:28.950251] [DEBUG] [LOCAL ] CCJ: Saving primary data. [2025-09-16T17:43:28.951378] [DEBUG] [LOCAL ] CCJ: Trimmed 58354 records to 40053 (keeping 150 days of data) [2025-09-16T17:43:29.175145] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-07. [2025-09-16T17:43:29.189625] [DEBUG] [LOCAL ] CCJ: Primary data saved (40053 records). [2025-09-16T17:43:29.189777] [INFO ] [MAIN ] CCJ: Full history saved. [2025-09-16T17:43:29.189872] [INFO ] [MAIN ] Update Progress: 414/2369 symbols completed. CCJ took 32.7s. Estimated remaining: 3h 25m 7s. Rate: 9.5 symbols/min [2025-09-16T17:43:29.359237] [INFO ] [MAIN ] CCRD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:29.368004] [DEBUG] [LOCAL ] CCRD: Loaded 7738 existing records from primary storage. [2025-09-16T17:43:29.368107] [INFO ] [MAIN ] CCRD: Loaded 7738 existing records. [2025-09-16T17:43:29.549670] [WARN ] [MAIN ] CCRD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:43:29.550044] [INFO ] [MAIN ] CCRD: Wiping existing primary data before full refresh. [2025-09-16T17:43:29.550142] [INFO ] [LOCAL ] CCRD: Starting deletion of primary data. [2025-09-16T17:43:29.550545] [DEBUG] [LOCAL ] CCRD: Deleted primary data file. [2025-09-16T17:43:29.550622] [INFO ] [MAIN ] CCRD: Starting full primary refresh. [2025-09-16T17:43:29.683363] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-09-16T17:43:29.947456] [INFO ] [REMOTE ] CCRD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:43:29.975180] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-02. [2025-09-16T17:43:29.975119] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T17:43:30.774923] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-08. [2025-09-16T17:43:31.065481] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T17:43:31.266034] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-09-16T17:43:31.574951] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-08. [2025-09-16T17:43:31.807856] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T17:43:32.374915] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-08. [2025-09-16T17:43:32.747878] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T17:43:33.174899] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-09. [2025-09-16T17:43:33.974909] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-09. [2025-09-16T17:43:34.728775] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-08 (2.35 seconds). [2025-09-16T17:43:34.774945] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-09. [2025-09-16T17:43:34.849323] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-09-16T17:43:34.855332] [INFO ] [REMOTE ] CCO: Full fetch complete (51133 records). [2025-09-16T17:43:34.855500] [DEBUG] [VALIDATOR ] CCO: Validating primary data. [2025-09-16T17:43:35.233900] [DEBUG] [VALIDATOR ] CCO: All expected trading days are present. [2025-09-16T17:43:35.234021] [DEBUG] [VALIDATOR ] CCO: Primary data validation passed. [2025-09-16T17:43:35.234818] [INFO ] [MAIN ] CCO: Fetched primary data is valid. [2025-09-16T17:43:35.234876] [DEBUG] [LOCAL ] CCO: Saving primary data. [2025-09-16T17:43:35.236318] [DEBUG] [LOCAL ] CCO: Trimmed 51133 records to 36204 (keeping 150 days of data) [2025-09-16T17:43:35.346755] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T17:43:35.354900] [INFO ] [REMOTE ] CCK: Full fetch complete (50784 records). [2025-09-16T17:43:35.358055] [DEBUG] [VALIDATOR ] CCK: Validating primary data. [2025-09-16T17:43:35.598638] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-03. [2025-09-16T17:43:35.850832] [DEBUG] [VALIDATOR ] CCK: All expected trading days are present. [2025-09-16T17:43:35.851673] [DEBUG] [VALIDATOR ] CCK: Primary data validation passed. [2025-09-16T17:43:35.855275] [INFO ] [MAIN ] CCK: Fetched primary data is valid. [2025-09-16T17:43:35.856876] [DEBUG] [LOCAL ] CCK: Saving primary data. [2025-09-16T17:43:35.860393] [DEBUG] [LOCAL ] CCO: Primary data saved (36204 records). [2025-09-16T17:43:35.860477] [INFO ] [MAIN ] CCO: Full history saved. [2025-09-16T17:43:35.860590] [INFO ] [MAIN ] Update Progress: 415/2369 symbols completed. CCO took 21.2s. Estimated remaining: 3h 25m 2s. Rate: 9.5 symbols/min [2025-09-16T17:43:35.961138] [DEBUG] [LOCAL ] CCK: Trimmed 50784 records to 34509 (keeping 150 days of data) [2025-09-16T17:43:36.099183] [INFO ] [MAIN ] CCS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:36.133854] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-09-16T17:43:36.169545] [DEBUG] [LOCAL ] CCS: Loaded 22400 existing records from primary storage. [2025-09-16T17:43:36.170860] [INFO ] [MAIN ] CCS: Loaded 22400 existing records. [2025-09-16T17:43:36.359043] [WARN ] [MAIN ] CCS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:43:36.362734] [INFO ] [MAIN ] CCS: Wiping existing primary data before full refresh. [2025-09-16T17:43:36.362823] [INFO ] [LOCAL ] CCS: Starting deletion of primary data. [2025-09-16T17:43:36.363498] [DEBUG] [LOCAL ] CCS: Deleted primary data file. [2025-09-16T17:43:36.401855] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-04. [2025-09-16T17:43:36.428963] [INFO ] [MAIN ] CCS: Starting full primary refresh. [2025-09-16T17:43:36.851557] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-09-16T17:43:36.849712] [INFO ] [REMOTE ] CCS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:43:36.860588] [INFO ] [REMOTE ] CCL: Full fetch complete (58480 records). [2025-09-16T17:43:36.863983] [DEBUG] [LOCAL ] CCK: Primary data saved (34509 records). [2025-09-16T17:43:36.864064] [DEBUG] [VALIDATOR ] CCL: Validating primary data. [2025-09-16T17:43:36.864186] [INFO ] [MAIN ] CCK: Full history saved. [2025-09-16T17:43:36.865110] [INFO ] [MAIN ] Update Progress: 416/2369 symbols completed. CCK took 27.1s. Estimated remaining: 3h 24m 31s. Rate: 9.5 symbols/min [2025-09-16T17:43:37.090340] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-09-16T17:43:37.199442] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-05. [2025-09-16T17:43:37.275487] [INFO ] [MAIN ] CCU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:37.477522] [DEBUG] [LOCAL ] CCU: Loaded 16315 existing records from primary storage. [2025-09-16T17:43:37.509731] [INFO ] [MAIN ] CCU: Loaded 16315 existing records. [2025-09-16T17:43:37.527172] [DEBUG] [VALIDATOR ] CCL: All expected trading days are present. [2025-09-16T17:43:37.585877] [DEBUG] [VALIDATOR ] CCL: Primary data validation passed. [2025-09-16T17:43:37.619470] [INFO ] [MAIN ] CCL: Fetched primary data is valid. [2025-09-16T17:43:37.619672] [DEBUG] [LOCAL ] CCL: Saving primary data. [2025-09-16T17:43:37.696882] [DEBUG] [LOCAL ] CCL: Trimmed 58480 records to 40150 (keeping 150 days of data) [2025-09-16T17:43:37.697025] [WARN ] [MAIN ] CCU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:43:37.697840] [INFO ] [MAIN ] CCU: Wiping existing primary data before full refresh. [2025-09-16T17:43:37.697910] [INFO ] [LOCAL ] CCU: Starting deletion of primary data. [2025-09-16T17:43:37.698668] [DEBUG] [LOCAL ] CCU: Deleted primary data file. [2025-09-16T17:43:37.698728] [INFO ] [MAIN ] CCU: Starting full primary refresh. [2025-09-16T17:43:37.902003] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-09-16T17:43:37.980994] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-02. [2025-09-16T17:43:38.175194] [INFO ] [REMOTE ] CCU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:43:38.320149] [DEBUG] [LOCAL ] CCL: Primary data saved (40150 records). [2025-09-16T17:43:38.320293] [INFO ] [MAIN ] CCL: Full history saved. [2025-09-16T17:43:38.320390] [INFO ] [MAIN ] Update Progress: 417/2369 symbols completed. CCL took 27.6s. Estimated remaining: 3h 24m 2s. Rate: 9.6 symbols/min [2025-09-16T17:43:38.516249] [INFO ] [MAIN ] CCZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:38.516470] [DEBUG] [LOCAL ] CCZ: No existing data file found in primary storage. [2025-09-16T17:43:38.516513] [WARN ] [MAIN ] CCZ: No existing data found, performing full refresh instead. [2025-09-16T17:43:38.516684] [INFO ] [MAIN ] CCZ: Wiping existing primary data before full refresh. [2025-09-16T17:43:38.516742] [INFO ] [LOCAL ] CCZ: Starting deletion of primary data. [2025-09-16T17:43:38.516800] [DEBUG] [LOCAL ] CCZ: No primary data file found to delete. [2025-09-16T17:43:38.516847] [INFO ] [MAIN ] CCZ: Starting full primary refresh. [2025-09-16T17:43:38.780006] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-06. [2025-09-16T17:43:38.899987] [INFO ] [REMOTE ] CCZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:43:39.109073] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-09-16T17:43:39.453470] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-09-16T17:43:39.574900] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-07. [2025-09-16T17:43:40.130604] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-09-16T17:43:40.374900] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-02. [2025-09-16T17:43:41.169413] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T17:43:41.176095] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-03. [2025-09-16T17:43:41.974898] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-02. [2025-09-16T17:43:42.025779] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-09-16T17:43:42.441960] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-09-16T17:43:42.774915] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-04. [2025-09-16T17:43:43.574925] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-03. [2025-09-16T17:43:44.250626] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T17:43:44.374903] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-05. [2025-09-16T17:43:44.534947] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-09-16T17:43:45.174926] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-03. [2025-09-16T17:43:45.284713] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T17:43:45.512243] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-09-16T17:43:45.974937] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-08. [2025-09-16T17:43:46.774902] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-06. [2025-09-16T17:43:46.981613] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T17:43:47.574976] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-04. [2025-09-16T17:43:47.719106] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-09-16T17:43:47.845673] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-04 (0.27 seconds). [2025-09-16T17:43:48.374924] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-07. [2025-09-16T17:43:49.174903] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-09. [2025-09-16T17:43:49.545606] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T17:43:49.974863] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-08. [2025-09-16T17:43:50.037494] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T17:43:50.039171] [INFO ] [REMOTE ] CCRD: Full fetch complete (9307 records). [2025-09-16T17:43:50.039238] [DEBUG] [VALIDATOR ] CCRD: Validating primary data. [2025-09-16T17:43:50.417403] [DEBUG] [VALIDATOR ] CCRD: All expected trading days are present. [2025-09-16T17:43:50.417482] [DEBUG] [VALIDATOR ] CCRD: Primary data validation passed. [2025-09-16T17:43:50.417736] [INFO ] [MAIN ] CCRD: Fetched primary data is valid. [2025-09-16T17:43:50.417781] [DEBUG] [LOCAL ] CCRD: Saving primary data. [2025-09-16T17:43:50.418331] [DEBUG] [LOCAL ] CCRD: Trimmed 9307 records to 7777 (keeping 150 days of data) [2025-09-16T17:43:50.464332] [DEBUG] [LOCAL ] CCRD: Primary data saved (7777 records). [2025-09-16T17:43:50.464444] [INFO ] [MAIN ] CCRD: Full history saved. [2025-09-16T17:43:50.464542] [INFO ] [MAIN ] Update Progress: 418/2369 symbols completed. CCRD took 21.3s. Estimated remaining: 3h 24m 23s. Rate: 9.5 symbols/min [2025-09-16T17:43:50.642668] [INFO ] [MAIN ] CDE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:50.682232] [DEBUG] [LOCAL ] CDE: Loaded 39604 existing records from primary storage. [2025-09-16T17:43:50.682372] [INFO ] [MAIN ] CDE: Loaded 39604 existing records. [2025-09-16T17:43:50.780097] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-05. [2025-09-16T17:43:50.879347] [WARN ] [MAIN ] CDE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:43:50.879588] [INFO ] [MAIN ] CDE: Wiping existing primary data before full refresh. [2025-09-16T17:43:50.879658] [INFO ] [LOCAL ] CDE: Starting deletion of primary data. [2025-09-16T17:43:50.880401] [DEBUG] [LOCAL ] CDE: Deleted primary data file. [2025-09-16T17:43:50.880450] [INFO ] [MAIN ] CDE: Starting full primary refresh. [2025-09-16T17:43:51.054003] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-05 (0.27 seconds). [2025-09-16T17:43:51.258429] [INFO ] [REMOTE ] CDE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:43:51.274494] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-09-16T17:43:51.574890] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-09. [2025-09-16T17:43:52.374952] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-04. [2025-09-16T17:43:52.446059] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-09-16T17:43:52.449369] [INFO ] [REMOTE ] CCS: Full fetch complete (33239 records). [2025-09-16T17:43:52.449492] [DEBUG] [VALIDATOR ] CCS: Validating primary data. [2025-09-16T17:43:52.450048] [FATAL] [VALIDATOR ] CCS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:43:52.450126] [FATAL] [MAIN ] CCS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:43:52.450179] [WARN ] [MAIN ] CCS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:43:52.450227] [INFO ] [LOCAL ] CCS: Starting deletion of primary data. [2025-09-16T17:43:52.450288] [DEBUG] [LOCAL ] CCS: No primary data file found to delete. [2025-09-16T17:43:52.450333] [INFO ] [LOCAL ] CCS: Starting deletion of derivative data. [2025-09-16T17:43:52.450689] [DEBUG] [LOCAL ] CCS: Deleted derivative data file. [2025-09-16T17:43:52.450779] [INFO ] [MAIN ] Update Progress: 419/2369 symbols completed. CCS took 16.6s. Estimated remaining: 3h 23m 57s. Rate: 9.6 symbols/min [2025-09-16T17:43:52.634513] [INFO ] [MAIN ] CDLR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:52.643214] [DEBUG] [LOCAL ] CDLR: Loaded 6990 existing records from primary storage. [2025-09-16T17:43:52.643319] [INFO ] [MAIN ] CDLR: Loaded 6990 existing records. [2025-09-16T17:43:52.819689] [WARN ] [MAIN ] CDLR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:43:52.819944] [INFO ] [MAIN ] CDLR: Wiping existing primary data before full refresh. [2025-09-16T17:43:52.819991] [INFO ] [LOCAL ] CDLR: Starting deletion of primary data. [2025-09-16T17:43:52.820248] [DEBUG] [LOCAL ] CDLR: Deleted primary data file. [2025-09-16T17:43:52.820292] [INFO ] [MAIN ] CDLR: Starting full primary refresh. [2025-09-16T17:43:53.201028] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-06. [2025-09-16T17:43:53.208942] [INFO ] [REMOTE ] CDLR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:43:53.323157] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-09-16T17:43:53.554307] [DEBUG] [REMOTE ] CCZ: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:43:53.554665] [WARN ] [REMOTE ] CCZ: No data returned for month 2025-06 [2025-09-16T17:43:53.974912] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-02. [2025-09-16T17:43:54.378748] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-09-16T17:43:54.774897] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-05. [2025-09-16T17:43:55.465827] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-09-16T17:43:55.574864] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-07. [2025-09-16T17:43:55.785966] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-07 (0.21 seconds). [2025-09-16T17:43:56.374900] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-08. [2025-09-16T17:43:56.631052] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-08 (0.26 seconds). [2025-09-16T17:43:57.174898] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-06. [2025-09-16T17:43:57.882438] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-09-16T17:43:57.974912] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-07. [2025-09-16T17:43:58.774909] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-09. [2025-09-16T17:43:59.005414] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-09-16T17:43:59.067327] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-09 (0.29 seconds). [2025-09-16T17:43:59.067951] [INFO ] [REMOTE ] CCZ: Full fetch complete (76 records). [2025-09-16T17:43:59.068047] [DEBUG] [VALIDATOR ] CCZ: Validating primary data. [2025-09-16T17:43:59.068399] [FATAL] [VALIDATOR ] CCZ: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:43:59.068479] [FATAL] [MAIN ] CCZ: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:43:59.068524] [WARN ] [MAIN ] CCZ: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:43:59.068595] [INFO ] [LOCAL ] CCZ: Starting deletion of primary data. [2025-09-16T17:43:59.068681] [DEBUG] [LOCAL ] CCZ: No primary data file found to delete. [2025-09-16T17:43:59.068737] [INFO ] [LOCAL ] CCZ: Starting deletion of derivative data. [2025-09-16T17:43:59.068788] [DEBUG] [LOCAL ] CCZ: No derivative data file found to delete. [2025-09-16T17:43:59.068869] [INFO ] [MAIN ] Update Progress: 420/2369 symbols completed. CCZ took 20.7s. Estimated remaining: 3h 23m 52s. Rate: 9.6 symbols/min [2025-09-16T17:43:59.242171] [INFO ] [MAIN ] CDP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:43:59.276530] [DEBUG] [LOCAL ] CDP: Loaded 31021 existing records from primary storage. [2025-09-16T17:43:59.276680] [INFO ] [MAIN ] CDP: Loaded 31021 existing records. [2025-09-16T17:43:59.451237] [WARN ] [MAIN ] CDP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:43:59.451458] [INFO ] [MAIN ] CDP: Wiping existing primary data before full refresh. [2025-09-16T17:43:59.451513] [INFO ] [LOCAL ] CDP: Starting deletion of primary data. [2025-09-16T17:43:59.452267] [DEBUG] [LOCAL ] CDP: Deleted primary data file. [2025-09-16T17:43:59.452316] [INFO ] [MAIN ] CDP: Starting full primary refresh. [2025-09-16T17:43:59.591263] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-08. [2025-09-16T17:43:59.805213] [INFO ] [REMOTE ] CDP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:44:00.374901] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-02. [2025-09-16T17:44:01.096480] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T17:44:01.174862] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-09. [2025-09-16T17:44:01.974889] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-03. [2025-09-16T17:44:02.112439] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-09-16T17:44:02.115255] [INFO ] [REMOTE ] CCU: Full fetch complete (26323 records). [2025-09-16T17:44:02.115338] [DEBUG] [VALIDATOR ] CCU: Validating primary data. [2025-09-16T17:44:02.485740] [DEBUG] [VALIDATOR ] CCU: All expected trading days are present. [2025-09-16T17:44:02.485819] [DEBUG] [VALIDATOR ] CCU: Primary data validation passed. [2025-09-16T17:44:02.486217] [INFO ] [MAIN ] CCU: Fetched primary data is valid. [2025-09-16T17:44:02.486262] [DEBUG] [LOCAL ] CCU: Saving primary data. [2025-09-16T17:44:02.486960] [DEBUG] [LOCAL ] CCU: Trimmed 26323 records to 16433 (keeping 150 days of data) [2025-09-16T17:44:02.589709] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-09-16T17:44:02.623621] [DEBUG] [LOCAL ] CCU: Primary data saved (16433 records). [2025-09-16T17:44:02.623789] [INFO ] [MAIN ] CCU: Full history saved. [2025-09-16T17:44:02.623881] [INFO ] [MAIN ] Update Progress: 421/2369 symbols completed. CCU took 25.8s. Estimated remaining: 3h 23m 33s. Rate: 9.6 symbols/min [2025-09-16T17:44:02.791794] [INFO ] [MAIN ] CDRE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:44:02.781569] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-02. [2025-09-16T17:44:02.811788] [DEBUG] [LOCAL ] CDRE: Loaded 18482 existing records from primary storage. [2025-09-16T17:44:02.811962] [INFO ] [MAIN ] CDRE: Loaded 18482 existing records. [2025-09-16T17:44:02.992900] [WARN ] [MAIN ] CDRE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:44:02.998507] [INFO ] [MAIN ] CDRE: Wiping existing primary data before full refresh. [2025-09-16T17:44:03.002617] [INFO ] [LOCAL ] CDRE: Starting deletion of primary data. [2025-09-16T17:44:03.003466] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-09-16T17:44:03.004760] [DEBUG] [LOCAL ] CDRE: Deleted primary data file. [2025-09-16T17:44:03.007103] [INFO ] [MAIN ] CDRE: Starting full primary refresh. [2025-09-16T17:44:03.358546] [INFO ] [REMOTE ] CDRE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:44:03.574893] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-02. [2025-09-16T17:44:04.338433] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T17:44:04.374861] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-04. [2025-09-16T17:44:04.689866] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-09-16T17:44:05.059378] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T17:44:05.174916] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-05. [2025-09-16T17:44:05.974914] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-03. [2025-09-16T17:44:06.032014] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T17:44:06.774893] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-06. [2025-09-16T17:44:07.177827] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T17:44:07.574901] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-03. [2025-09-16T17:44:07.619422] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-09-16T17:44:08.374900] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-04. [2025-09-16T17:44:09.174895] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-07. [2025-09-16T17:44:09.256174] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-09-16T17:44:09.499709] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-09-16T17:44:09.961740] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T17:44:09.974859] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-05. [2025-09-16T17:44:10.774913] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-04. [2025-09-16T17:44:10.884364] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T17:44:11.574911] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-06. [2025-09-16T17:44:12.374915] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-08. [2025-09-16T17:44:12.824692] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-09-16T17:44:12.871783] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:44:13.030405] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-09-16T17:44:13.174911] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-09. [2025-09-16T17:44:13.808786] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-09-16T17:44:13.810291] [INFO ] [REMOTE ] CDLR: Full fetch complete (10628 records). [2025-09-16T17:44:13.810358] [DEBUG] [VALIDATOR ] CDLR: Validating primary data. [2025-09-16T17:44:13.981491] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-03. [2025-09-16T17:44:14.170443] [DEBUG] [VALIDATOR ] CDLR: All expected trading days are present. [2025-09-16T17:44:14.170521] [DEBUG] [VALIDATOR ] CDLR: Primary data validation passed. [2025-09-16T17:44:14.170816] [INFO ] [MAIN ] CDLR: Fetched primary data is valid. [2025-09-16T17:44:14.170880] [DEBUG] [LOCAL ] CDLR: Saving primary data. [2025-09-16T17:44:14.171551] [DEBUG] [LOCAL ] CDLR: Trimmed 10628 records to 7019 (keeping 150 days of data) [2025-09-16T17:44:14.212380] [DEBUG] [LOCAL ] CDLR: Primary data saved (7019 records). [2025-09-16T17:44:14.212488] [INFO ] [MAIN ] CDLR: Full history saved. [2025-09-16T17:44:14.212581] [INFO ] [MAIN ] Update Progress: 422/2369 symbols completed. CDLR took 21.8s. Estimated remaining: 3h 23m 52s. Rate: 9.6 symbols/min [2025-09-16T17:44:14.385485] [INFO ] [MAIN ] CE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:44:14.428690] [DEBUG] [LOCAL ] CE: Loaded 37804 existing records from primary storage. [2025-09-16T17:44:14.428863] [INFO ] [MAIN ] CE: Loaded 37804 existing records. [2025-09-16T17:44:14.609672] [WARN ] [MAIN ] CE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:44:14.609932] [INFO ] [MAIN ] CE: Wiping existing primary data before full refresh. [2025-09-16T17:44:14.609990] [INFO ] [LOCAL ] CE: Starting deletion of primary data. [2025-09-16T17:44:14.610955] [DEBUG] [LOCAL ] CE: Deleted primary data file. [2025-09-16T17:44:14.611008] [INFO ] [MAIN ] CE: Starting full primary refresh. [2025-09-16T17:44:14.775228] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-05. [2025-09-16T17:44:14.974708] [INFO ] [REMOTE ] CE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:44:15.574918] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-02. [2025-09-16T17:44:15.981320] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-09-16T17:44:16.374906] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-07. [2025-09-16T17:44:16.514995] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T17:44:17.134529] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T17:44:17.174852] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-03. [2025-09-16T17:44:17.463041] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T17:44:17.974908] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-04. [2025-09-16T17:44:18.774906] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-06. [2025-09-16T17:44:18.863927] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-09-16T17:44:19.558004] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T17:44:19.574843] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-05. [2025-09-16T17:44:20.374884] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-04. [2025-09-16T17:44:20.424425] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T17:44:21.041259] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T17:44:21.174911] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-07. [2025-09-16T17:44:21.981156] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-06. [2025-09-16T17:44:22.002033] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-09-16T17:44:22.774908] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-05. [2025-09-16T17:44:23.285616] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-09-16T17:44:23.490240] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-09-16T17:44:23.574860] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-07. [2025-09-16T17:44:24.374921] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-08. [2025-09-16T17:44:24.474971] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T17:44:24.884045] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T17:44:25.174908] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-08. [2025-09-16T17:44:25.974906] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-08. [2025-09-16T17:44:26.346666] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-09-16T17:44:26.774894] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-09. [2025-09-16T17:44:26.943948] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T17:44:27.015323] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-09-16T17:44:27.574962] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-09. [2025-09-16T17:44:28.374897] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-06. [2025-09-16T17:44:28.558758] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T17:44:28.561605] [INFO ] [REMOTE ] CDRE: Full fetch complete (26546 records). [2025-09-16T17:44:28.561702] [DEBUG] [VALIDATOR ] CDRE: Validating primary data. [2025-09-16T17:44:28.946523] [DEBUG] [VALIDATOR ] CDRE: All expected trading days are present. [2025-09-16T17:44:28.946663] [DEBUG] [VALIDATOR ] CDRE: Primary data validation passed. [2025-09-16T17:44:28.947087] [INFO ] [MAIN ] CDRE: Fetched primary data is valid. [2025-09-16T17:44:28.947222] [DEBUG] [LOCAL ] CDRE: Saving primary data. [2025-09-16T17:44:28.948157] [DEBUG] [LOCAL ] CDRE: Trimmed 26546 records to 18653 (keeping 150 days of data) [2025-09-16T17:44:29.057273] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-09 (2.28 seconds). [2025-09-16T17:44:29.065974] [INFO ] [REMOTE ] CDE: Full fetch complete (58320 records). [2025-09-16T17:44:29.068294] [DEBUG] [VALIDATOR ] CDE: Validating primary data. [2025-09-16T17:44:29.177434] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-09. [2025-09-16T17:44:29.481791] [DEBUG] [VALIDATOR ] CDE: All expected trading days are present. [2025-09-16T17:44:29.483593] [DEBUG] [VALIDATOR ] CDE: Primary data validation passed. [2025-09-16T17:44:29.485739] [INFO ] [MAIN ] CDE: Fetched primary data is valid. [2025-09-16T17:44:29.486396] [DEBUG] [LOCAL ] CDE: Saving primary data. [2025-09-16T17:44:29.493409] [DEBUG] [LOCAL ] CDRE: Primary data saved (18653 records). [2025-09-16T17:44:29.493730] [INFO ] [MAIN ] CDRE: Full history saved. [2025-09-16T17:44:29.494036] [INFO ] [MAIN ] Update Progress: 423/2369 symbols completed. CDRE took 26.9s. Estimated remaining: 3h 24m 27s. Rate: 9.5 symbols/min [2025-09-16T17:44:29.494339] [DEBUG] [LOCAL ] CDE: Trimmed 58320 records to 39994 (keeping 150 days of data) [2025-09-16T17:44:29.765189] [INFO ] [MAIN ] CEE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:44:29.780012] [DEBUG] [LOCAL ] CEE: Loaded 3994 existing records from primary storage. [2025-09-16T17:44:29.783297] [INFO ] [MAIN ] CEE: Loaded 3994 existing records. [2025-09-16T17:44:29.980843] [WARN ] [MAIN ] CEE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:44:29.984988] [INFO ] [MAIN ] CEE: Wiping existing primary data before full refresh. [2025-09-16T17:44:29.987762] [INFO ] [LOCAL ] CEE: Starting deletion of primary data. [2025-09-16T17:44:29.988110] [DEBUG] [LOCAL ] CEE: Deleted primary data file. [2025-09-16T17:44:29.988162] [INFO ] [MAIN ] CEE: Starting full primary refresh. [2025-09-16T17:44:30.472514] [INFO ] [REMOTE ] CEE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:44:30.473139] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-02. [2025-09-16T17:44:30.506962] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-09-16T17:44:30.530027] [DEBUG] [LOCAL ] CDE: Primary data saved (39994 records). [2025-09-16T17:44:30.530155] [INFO ] [MAIN ] CDE: Full history saved. [2025-09-16T17:44:30.530236] [INFO ] [MAIN ] Update Progress: 424/2369 symbols completed. CDE took 40.1s. Estimated remaining: 3h 23m 56s. Rate: 9.5 symbols/min [2025-09-16T17:44:30.702008] [INFO ] [MAIN ] CEPU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:44:30.719369] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T17:44:30.725148] [INFO ] [REMOTE ] CDP: Full fetch complete (46630 records). [2025-09-16T17:44:30.725275] [DEBUG] [VALIDATOR ] CDP: Validating primary data. [2025-09-16T17:44:30.794149] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-07. [2025-09-16T17:44:30.866159] [DEBUG] [LOCAL ] CEPU: Loaded 18551 existing records from primary storage. [2025-09-16T17:44:30.889878] [INFO ] [MAIN ] CEPU: Loaded 18551 existing records. [2025-09-16T17:44:31.252942] [WARN ] [MAIN ] CEPU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:44:31.290483] [DEBUG] [VALIDATOR ] CDP: All expected trading days are present. [2025-09-16T17:44:31.290344] [INFO ] [MAIN ] CEPU: Wiping existing primary data before full refresh. [2025-09-16T17:44:31.290572] [DEBUG] [VALIDATOR ] CDP: Primary data validation passed. [2025-09-16T17:44:31.291253] [INFO ] [MAIN ] CDP: Fetched primary data is valid. [2025-09-16T17:44:31.291395] [INFO ] [LOCAL ] CEPU: Starting deletion of primary data. [2025-09-16T17:44:31.291331] [DEBUG] [LOCAL ] CDP: Saving primary data. [2025-09-16T17:44:31.299193] [DEBUG] [LOCAL ] CEPU: Deleted primary data file. [2025-09-16T17:44:31.299532] [INFO ] [MAIN ] CEPU: Starting full primary refresh. [2025-09-16T17:44:31.300186] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-09-16T17:44:31.357469] [DEBUG] [LOCAL ] CDP: Trimmed 46630 records to 31306 (keeping 150 days of data) [2025-09-16T17:44:31.575751] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-03. [2025-09-16T17:44:31.772944] [INFO ] [REMOTE ] CEPU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:44:31.876661] [DEBUG] [LOCAL ] CDP: Primary data saved (31306 records). [2025-09-16T17:44:31.876817] [INFO ] [MAIN ] CDP: Full history saved. [2025-09-16T17:44:31.878888] [INFO ] [MAIN ] Update Progress: 425/2369 symbols completed. CDP took 32.8s. Estimated remaining: 3h 23m 27s. Rate: 9.6 symbols/min [2025-09-16T17:44:32.049078] [INFO ] [MAIN ] CF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:44:32.091664] [DEBUG] [LOCAL ] CF: Loaded 38590 existing records from primary storage. [2025-09-16T17:44:32.091822] [INFO ] [MAIN ] CF: Loaded 38590 existing records. [2025-09-16T17:44:32.266534] [WARN ] [MAIN ] CF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:44:32.266802] [INFO ] [MAIN ] CF: Wiping existing primary data before full refresh. [2025-09-16T17:44:32.266852] [INFO ] [LOCAL ] CF: Starting deletion of primary data. [2025-09-16T17:44:32.267775] [DEBUG] [LOCAL ] CF: Deleted primary data file. [2025-09-16T17:44:32.267826] [INFO ] [MAIN ] CF: Starting full primary refresh. [2025-09-16T17:44:32.379973] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-02. [2025-09-16T17:44:32.622050] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-09-16T17:44:32.640768] [INFO ] [REMOTE ] CF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:44:32.675997] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T17:44:33.174898] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-04. [2025-09-16T17:44:33.919965] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-09-16T17:44:33.979089] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-05. [2025-09-16T17:44:33.982635] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T17:44:34.658187] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-09-16T17:44:34.774892] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-06. [2025-09-16T17:44:35.387881] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-09-16T17:44:35.574900] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-07. [2025-09-16T17:44:36.039776] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-09-16T17:44:36.374900] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-08. [2025-09-16T17:44:36.874298] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-09-16T17:44:37.174926] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-08. [2025-09-16T17:44:37.974930] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-09. [2025-09-16T17:44:38.534546] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T17:44:38.535704] [INFO ] [REMOTE ] CEE: Full fetch complete (8728 records). [2025-09-16T17:44:38.535781] [DEBUG] [VALIDATOR ] CEE: Validating primary data. [2025-09-16T17:44:38.783582] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-02. [2025-09-16T17:44:38.988266] [DEBUG] [VALIDATOR ] CEE: All expected trading days are present. [2025-09-16T17:44:38.988470] [DEBUG] [VALIDATOR ] CEE: Primary data validation passed. [2025-09-16T17:44:38.989057] [INFO ] [MAIN ] CEE: Fetched primary data is valid. [2025-09-16T17:44:38.989261] [DEBUG] [LOCAL ] CEE: Saving primary data. [2025-09-16T17:44:38.990375] [DEBUG] [LOCAL ] CEE: Trimmed 8728 records to 4013 (keeping 150 days of data) [2025-09-16T17:44:39.026305] [DEBUG] [LOCAL ] CEE: Primary data saved (4013 records). [2025-09-16T17:44:39.026616] [INFO ] [MAIN ] CEE: Full history saved. [2025-09-16T17:44:39.026914] [INFO ] [MAIN ] Update Progress: 426/2369 symbols completed. CEE took 9.5s. Estimated remaining: 3h 23m 25s. Rate: 9.6 symbols/min [2025-09-16T17:44:39.225729] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-09-16T17:44:39.384073] [INFO ] [MAIN ] CFG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:44:39.442124] [DEBUG] [LOCAL ] CFG: Loaded 39585 existing records from primary storage. [2025-09-16T17:44:39.442331] [INFO ] [MAIN ] CFG: Loaded 39585 existing records. [2025-09-16T17:44:39.583307] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-09. [2025-09-16T17:44:39.686062] [WARN ] [MAIN ] CFG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:44:39.686378] [INFO ] [MAIN ] CFG: Wiping existing primary data before full refresh. [2025-09-16T17:44:39.686450] [INFO ] [LOCAL ] CFG: Starting deletion of primary data. [2025-09-16T17:44:39.687515] [DEBUG] [LOCAL ] CFG: Deleted primary data file. [2025-09-16T17:44:39.687594] [INFO ] [MAIN ] CFG: Starting full primary refresh. [2025-09-16T17:44:40.103697] [INFO ] [REMOTE ] CFG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:44:40.374884] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-03. [2025-09-16T17:44:40.449035] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T17:44:41.174900] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-03. [2025-09-16T17:44:41.446101] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-09-16T17:44:41.452113] [INFO ] [REMOTE ] CE: Full fetch complete (56252 records). [2025-09-16T17:44:41.452228] [DEBUG] [VALIDATOR ] CE: Validating primary data. [2025-09-16T17:44:41.838431] [DEBUG] [VALIDATOR ] CE: All expected trading days are present. [2025-09-16T17:44:41.838516] [DEBUG] [VALIDATOR ] CE: Primary data validation passed. [2025-09-16T17:44:41.839162] [INFO ] [MAIN ] CE: Fetched primary data is valid. [2025-09-16T17:44:41.839208] [DEBUG] [LOCAL ] CE: Saving primary data. [2025-09-16T17:44:41.840400] [DEBUG] [LOCAL ] CE: Trimmed 56252 records to 38140 (keeping 150 days of data) [2025-09-16T17:44:41.973542] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T17:44:41.981022] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-02. [2025-09-16T17:44:42.098696] [DEBUG] [LOCAL ] CE: Primary data saved (38140 records). [2025-09-16T17:44:42.098834] [INFO ] [MAIN ] CE: Full history saved. [2025-09-16T17:44:42.098915] [INFO ] [MAIN ] Update Progress: 427/2369 symbols completed. CE took 27.9s. Estimated remaining: 3h 23m 4s. Rate: 9.6 symbols/min [2025-09-16T17:44:42.267344] [INFO ] [MAIN ] CFND: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:44:42.267493] [DEBUG] [LOCAL ] CFND: No existing data file found in primary storage. [2025-09-16T17:44:42.267530] [WARN ] [MAIN ] CFND: No existing data found, performing full refresh instead. [2025-09-16T17:44:42.267666] [INFO ] [MAIN ] CFND: Wiping existing primary data before full refresh. [2025-09-16T17:44:42.267707] [INFO ] [LOCAL ] CFND: Starting deletion of primary data. [2025-09-16T17:44:42.267747] [DEBUG] [LOCAL ] CFND: No primary data file found to delete. [2025-09-16T17:44:42.267779] [INFO ] [MAIN ] CFND: Starting full primary refresh. [2025-09-16T17:44:42.622915] [INFO ] [REMOTE ] CFND: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:44:42.656864] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T17:44:42.774861] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-04. [2025-09-16T17:44:43.574882] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-02. [2025-09-16T17:44:43.666614] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T17:44:43.808458] [DEBUG] [REMOTE ] CFND: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:44:43.808780] [WARN ] [REMOTE ] CFND: No data returned for month 2025-02 [2025-09-16T17:44:43.848401] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-09-16T17:44:44.374911] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-03. [2025-09-16T17:44:45.174909] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-04. [2025-09-16T17:44:45.916257] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-09-16T17:44:45.974876] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-04. [2025-09-16T17:44:46.656761] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T17:44:46.774961] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-03. [2025-09-16T17:44:47.008292] [DEBUG] [REMOTE ] CFND: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:44:47.008730] [WARN ] [REMOTE ] CFND: No data returned for month 2025-03 [2025-09-16T17:44:47.574906] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-04. [2025-09-16T17:44:47.741417] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T17:44:47.842071] [DEBUG] [REMOTE ] CFND: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:44:47.842373] [WARN ] [REMOTE ] CFND: No data returned for month 2025-04 [2025-09-16T17:44:48.374935] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-05. [2025-09-16T17:44:49.174914] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-05. [2025-09-16T17:44:49.771979] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T17:44:49.974917] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-06. [2025-09-16T17:44:50.163565] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T17:44:50.774919] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-06. [2025-09-16T17:44:51.364779] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T17:44:51.442151] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-09-16T17:44:51.574909] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-07. [2025-09-16T17:44:52.374902] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-07. [2025-09-16T17:44:53.007187] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-09-16T17:44:53.100235] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-09-16T17:44:53.174947] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-05. [2025-09-16T17:44:53.974910] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-08. [2025-09-16T17:44:54.666068] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-09-16T17:44:54.774962] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-05. [2025-09-16T17:44:54.920382] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-09-16T17:44:54.968347] [DEBUG] [REMOTE ] CFND: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:44:54.968779] [WARN ] [REMOTE ] CFND: No data returned for month 2025-05 [2025-09-16T17:44:55.574922] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-09. [2025-09-16T17:44:56.374906] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-06. [2025-09-16T17:44:56.532728] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T17:44:56.536459] [INFO ] [REMOTE ] CEPU: Full fetch complete (27977 records). [2025-09-16T17:44:56.536528] [DEBUG] [VALIDATOR ] CEPU: Validating primary data. [2025-09-16T17:44:56.606954] [DEBUG] [REMOTE ] CFND: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:44:56.638462] [WARN ] [REMOTE ] CFND: No data returned for month 2025-06 [2025-09-16T17:44:56.905815] [DEBUG] [VALIDATOR ] CEPU: All expected trading days are present. [2025-09-16T17:44:56.905892] [DEBUG] [VALIDATOR ] CEPU: Primary data validation passed. [2025-09-16T17:44:56.906304] [INFO ] [MAIN ] CEPU: Fetched primary data is valid. [2025-09-16T17:44:56.906344] [DEBUG] [LOCAL ] CEPU: Saving primary data. [2025-09-16T17:44:56.907019] [DEBUG] [LOCAL ] CEPU: Trimmed 27977 records to 18809 (keeping 150 days of data) [2025-09-16T17:44:57.018613] [DEBUG] [LOCAL ] CEPU: Primary data saved (18809 records). [2025-09-16T17:44:57.018783] [INFO ] [MAIN ] CEPU: Full history saved. [2025-09-16T17:44:57.018880] [INFO ] [MAIN ] Update Progress: 428/2369 symbols completed. CEPU took 26.5s. Estimated remaining: 3h 23m 37s. Rate: 9.5 symbols/min [2025-09-16T17:44:57.176275] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-07. [2025-09-16T17:44:57.196288] [INFO ] [MAIN ] CFR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:44:57.196503] [DEBUG] [LOCAL ] CFR: No existing data file found in primary storage. [2025-09-16T17:44:57.196545] [WARN ] [MAIN ] CFR: No existing data found, performing full refresh instead. [2025-09-16T17:44:57.196694] [INFO ] [MAIN ] CFR: Wiping existing primary data before full refresh. [2025-09-16T17:44:57.196738] [INFO ] [LOCAL ] CFR: Starting deletion of primary data. [2025-09-16T17:44:57.196780] [DEBUG] [LOCAL ] CFR: No primary data file found to delete. [2025-09-16T17:44:57.196813] [INFO ] [MAIN ] CFR: Starting full primary refresh. [2025-09-16T17:44:57.470329] [DEBUG] [REMOTE ] CFND: Error fetching primary data for 2025-07: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:44:57.473215] [WARN ] [REMOTE ] CFND: No data returned for month 2025-07 [2025-09-16T17:44:57.548822] [INFO ] [REMOTE ] CFR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:44:57.974900] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-06. [2025-09-16T17:44:58.774982] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-08. [2025-09-16T17:44:59.250901] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T17:44:59.574965] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-07. [2025-09-16T17:44:59.742476] [INFO ] [REMOTE ] CFND: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-09-16T17:45:00.374952] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-09. [2025-09-16T17:45:01.036216] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T17:45:01.169665] [INFO ] [REMOTE ] CFND: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T17:45:01.171106] [INFO ] [REMOTE ] CFND: Full fetch complete (2261 records). [2025-09-16T17:45:01.171180] [DEBUG] [VALIDATOR ] CFND: Validating primary data. [2025-09-16T17:45:01.182680] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-08. [2025-09-16T17:45:01.373568] [WARN ] [VALIDATOR ] CFND: Validation failed - insufficient historical data. Have 28 unique days, need 150 days [2025-09-16T17:45:01.373982] [FATAL] [MAIN ] CFND: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:45:01.374074] [WARN ] [MAIN ] CFND: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:45:01.374126] [INFO ] [LOCAL ] CFND: Starting deletion of primary data. [2025-09-16T17:45:01.374201] [DEBUG] [LOCAL ] CFND: No primary data file found to delete. [2025-09-16T17:45:01.374239] [INFO ] [LOCAL ] CFND: Starting deletion of derivative data. [2025-09-16T17:45:01.374299] [DEBUG] [LOCAL ] CFND: No derivative data file found to delete. [2025-09-16T17:45:01.374372] [INFO ] [MAIN ] Update Progress: 429/2369 symbols completed. CFND took 19.3s. Estimated remaining: 3h 23m 22s. Rate: 9.5 symbols/min [2025-09-16T17:45:01.562501] [INFO ] [MAIN ] CGAU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:45:01.599385] [DEBUG] [LOCAL ] CGAU: Loaded 31020 existing records from primary storage. [2025-09-16T17:45:01.599679] [INFO ] [MAIN ] CGAU: Loaded 31020 existing records. [2025-09-16T17:45:01.789939] [WARN ] [MAIN ] CGAU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:45:01.790199] [INFO ] [MAIN ] CGAU: Wiping existing primary data before full refresh. [2025-09-16T17:45:01.790246] [INFO ] [LOCAL ] CGAU: Starting deletion of primary data. [2025-09-16T17:45:01.791170] [DEBUG] [LOCAL ] CGAU: Deleted primary data file. [2025-09-16T17:45:01.791223] [INFO ] [MAIN ] CGAU: Starting full primary refresh. [2025-09-16T17:45:01.981550] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-08. [2025-09-16T17:45:02.192463] [INFO ] [REMOTE ] CGAU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:45:02.774929] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-02. [2025-09-16T17:45:03.415227] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-08 (2.23 seconds). [2025-09-16T17:45:03.574919] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-02. [2025-09-16T17:45:03.684426] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-09-16T17:45:03.898763] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-09-16T17:45:04.374908] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-03. [2025-09-16T17:45:04.804805] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-09-16T17:45:05.174924] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-03. [2025-09-16T17:45:05.569391] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-09-16T17:45:05.974932] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-09. [2025-09-16T17:45:06.252388] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T17:45:06.774903] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-09. [2025-09-16T17:45:07.229688] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-09-16T17:45:07.235859] [INFO ] [REMOTE ] CF: Full fetch complete (56896 records). [2025-09-16T17:45:07.235955] [DEBUG] [VALIDATOR ] CF: Validating primary data. [2025-09-16T17:45:07.595953] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-04. [2025-09-16T17:45:07.614403] [DEBUG] [VALIDATOR ] CF: All expected trading days are present. [2025-09-16T17:45:07.614473] [DEBUG] [VALIDATOR ] CF: Primary data validation passed. [2025-09-16T17:45:07.615277] [INFO ] [MAIN ] CF: Fetched primary data is valid. [2025-09-16T17:45:07.615427] [DEBUG] [LOCAL ] CF: Saving primary data. [2025-09-16T17:45:07.616731] [DEBUG] [LOCAL ] CF: Trimmed 56896 records to 38963 (keeping 150 days of data) [2025-09-16T17:45:07.860239] [DEBUG] [LOCAL ] CF: Primary data saved (38963 records). [2025-09-16T17:45:07.860388] [INFO ] [MAIN ] CF: Full history saved. [2025-09-16T17:45:07.861358] [INFO ] [MAIN ] Update Progress: 430/2369 symbols completed. CF took 36.0s. Estimated remaining: 3h 23m 17s. Rate: 9.5 symbols/min [2025-09-16T17:45:08.034952] [INFO ] [MAIN ] CGV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:45:08.038629] [DEBUG] [LOCAL ] CGV: Loaded 943 existing records from primary storage. [2025-09-16T17:45:08.038728] [INFO ] [MAIN ] CGV: Loaded 943 existing records. [2025-09-16T17:45:08.229810] [WARN ] [MAIN ] CGV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:45:08.230062] [INFO ] [MAIN ] CGV: Wiping existing primary data before full refresh. [2025-09-16T17:45:08.230108] [INFO ] [LOCAL ] CGV: Starting deletion of primary data. [2025-09-16T17:45:08.230362] [DEBUG] [LOCAL ] CGV: Deleted primary data file. [2025-09-16T17:45:08.230414] [INFO ] [MAIN ] CGV: Starting full primary refresh. [2025-09-16T17:45:08.380884] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-04. [2025-09-16T17:45:08.601373] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-09-16T17:45:08.605253] [INFO ] [REMOTE ] CGV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:45:08.612271] [INFO ] [REMOTE ] CFG: Full fetch complete (58292 records). [2025-09-16T17:45:08.612402] [DEBUG] [VALIDATOR ] CFG: Validating primary data. [2025-09-16T17:45:08.994678] [DEBUG] [VALIDATOR ] CFG: All expected trading days are present. [2025-09-16T17:45:08.994771] [DEBUG] [VALIDATOR ] CFG: Primary data validation passed. [2025-09-16T17:45:08.995485] [INFO ] [MAIN ] CFG: Fetched primary data is valid. [2025-09-16T17:45:08.995530] [DEBUG] [LOCAL ] CFG: Saving primary data. [2025-09-16T17:45:08.996812] [DEBUG] [LOCAL ] CFG: Trimmed 58292 records to 39975 (keeping 150 days of data) [2025-09-16T17:45:09.177798] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-02. [2025-09-16T17:45:09.246762] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T17:45:09.283023] [DEBUG] [LOCAL ] CFG: Primary data saved (39975 records). [2025-09-16T17:45:09.283206] [INFO ] [MAIN ] CFG: Full history saved. [2025-09-16T17:45:09.283319] [INFO ] [MAIN ] Update Progress: 431/2369 symbols completed. CFG took 30.3s. Estimated remaining: 3h 22m 48s. Rate: 9.6 symbols/min [2025-09-16T17:45:09.458050] [INFO ] [MAIN ] CHCT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:45:09.488373] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-09-16T17:45:09.509289] [DEBUG] [LOCAL ] CHCT: Loaded 17753 existing records from primary storage. [2025-09-16T17:45:09.509438] [INFO ] [MAIN ] CHCT: Loaded 17753 existing records. [2025-09-16T17:45:09.664810] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-09-16T17:45:09.700803] [WARN ] [MAIN ] CHCT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:45:09.701051] [INFO ] [MAIN ] CHCT: Wiping existing primary data before full refresh. [2025-09-16T17:45:09.701099] [INFO ] [LOCAL ] CHCT: Starting deletion of primary data. [2025-09-16T17:45:09.701670] [DEBUG] [LOCAL ] CHCT: Deleted primary data file. [2025-09-16T17:45:09.701729] [INFO ] [MAIN ] CHCT: Starting full primary refresh. [2025-09-16T17:45:09.980154] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-03. [2025-09-16T17:45:10.088252] [INFO ] [REMOTE ] CHCT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:45:10.458338] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-09-16T17:45:10.774898] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-05. [2025-09-16T17:45:11.574903] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-02. [2025-09-16T17:45:12.201409] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-09-16T17:45:12.374913] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-04. [2025-09-16T17:45:12.502341] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-09-16T17:45:12.791561] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-09-16T17:45:13.174902] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-05. [2025-09-16T17:45:13.584201] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-09-16T17:45:13.974929] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-06. [2025-09-16T17:45:14.774913] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-05. [2025-09-16T17:45:15.343515] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T17:45:15.574919] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-06. [2025-09-16T17:45:15.912583] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-09-16T17:45:15.973066] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T17:45:16.374929] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-07. [2025-09-16T17:45:16.884173] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-09-16T17:45:17.174901] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-03. [2025-09-16T17:45:17.974940] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-07. [2025-09-16T17:45:18.318253] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T17:45:18.774904] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-08. [2025-09-16T17:45:19.128313] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-09-16T17:45:19.200983] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-09-16T17:45:19.574902] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-09. [2025-09-16T17:45:20.085768] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-09-16T17:45:20.086502] [INFO ] [REMOTE ] CGV: Full fetch complete (1636 records). [2025-09-16T17:45:20.086569] [DEBUG] [VALIDATOR ] CGV: Validating primary data. [2025-09-16T17:45:20.379972] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-06. [2025-09-16T17:45:20.445976] [DEBUG] [VALIDATOR ] CGV: All expected trading days are present. [2025-09-16T17:45:20.446054] [DEBUG] [VALIDATOR ] CGV: Primary data validation passed. [2025-09-16T17:45:20.446144] [INFO ] [MAIN ] CGV: Fetched primary data is valid. [2025-09-16T17:45:20.446215] [DEBUG] [LOCAL ] CGV: Saving primary data. [2025-09-16T17:45:20.446570] [DEBUG] [LOCAL ] CGV: Trimmed 1636 records to 954 (keeping 150 days of data) [2025-09-16T17:45:20.453234] [DEBUG] [LOCAL ] CGV: Primary data saved (954 records). [2025-09-16T17:45:20.453301] [INFO ] [MAIN ] CGV: Full history saved. [2025-09-16T17:45:20.453375] [INFO ] [MAIN ] Update Progress: 432/2369 symbols completed. CGV took 12.6s. Estimated remaining: 3h 23m 4s. Rate: 9.5 symbols/min [2025-09-16T17:45:20.623198] [INFO ] [MAIN ] CHD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:45:20.667612] [DEBUG] [LOCAL ] CHD: Loaded 38809 existing records from primary storage. [2025-09-16T17:45:20.667807] [INFO ] [MAIN ] CHD: Loaded 38809 existing records. [2025-09-16T17:45:20.851596] [WARN ] [MAIN ] CHD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:45:20.851898] [INFO ] [MAIN ] CHD: Wiping existing primary data before full refresh. [2025-09-16T17:45:20.851945] [INFO ] [LOCAL ] CHD: Starting deletion of primary data. [2025-09-16T17:45:20.852724] [DEBUG] [LOCAL ] CHD: Deleted primary data file. [2025-09-16T17:45:20.852774] [INFO ] [MAIN ] CHD: Starting full primary refresh. [2025-09-16T17:45:21.181063] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-04. [2025-09-16T17:45:21.218379] [INFO ] [REMOTE ] CHD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:45:21.941857] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T17:45:21.974871] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-07. [2025-09-16T17:45:22.377525] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-09-16T17:45:22.774901] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-08. [2025-09-16T17:45:23.073340] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-09-16T17:45:23.574912] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-08. [2025-09-16T17:45:24.235501] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-09-16T17:45:24.374951] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-02. [2025-09-16T17:45:24.990594] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T17:45:25.174919] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-05. [2025-09-16T17:45:25.830415] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-09-16T17:45:25.974924] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-09. [2025-09-16T17:45:26.258462] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-09-16T17:45:26.774966] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-03. [2025-09-16T17:45:27.331141] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T17:45:27.335771] [INFO ] [REMOTE ] CFR: Full fetch complete (36706 records). [2025-09-16T17:45:27.335879] [DEBUG] [VALIDATOR ] CFR: Validating primary data. [2025-09-16T17:45:27.580155] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-06. [2025-09-16T17:45:27.754705] [DEBUG] [VALIDATOR ] CFR: All expected trading days are present. [2025-09-16T17:45:27.754795] [DEBUG] [VALIDATOR ] CFR: Primary data validation passed. [2025-09-16T17:45:27.755381] [INFO ] [MAIN ] CFR: Fetched primary data is valid. [2025-09-16T17:45:27.755428] [DEBUG] [LOCAL ] CFR: Saving primary data. [2025-09-16T17:45:27.756310] [DEBUG] [LOCAL ] CFR: Trimmed 36706 records to 24180 (keeping 150 days of data) [2025-09-16T17:45:27.907479] [DEBUG] [LOCAL ] CFR: Primary data saved (24180 records). [2025-09-16T17:45:27.907612] [INFO ] [MAIN ] CFR: Full history saved. [2025-09-16T17:45:27.907704] [INFO ] [MAIN ] Update Progress: 433/2369 symbols completed. CFR took 30.9s. Estimated remaining: 3h 23m 3s. Rate: 9.5 symbols/min [2025-09-16T17:45:28.076500] [INFO ] [MAIN ] CHE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:45:28.088394] [DEBUG] [LOCAL ] CHE: Loaded 11150 existing records from primary storage. [2025-09-16T17:45:28.088524] [INFO ] [MAIN ] CHE: Loaded 11150 existing records. [2025-09-16T17:45:28.267828] [WARN ] [MAIN ] CHE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:45:28.268129] [INFO ] [MAIN ] CHE: Wiping existing primary data before full refresh. [2025-09-16T17:45:28.268184] [INFO ] [LOCAL ] CHE: Starting deletion of primary data. [2025-09-16T17:45:28.268574] [DEBUG] [LOCAL ] CHE: Deleted primary data file. [2025-09-16T17:45:28.268626] [INFO ] [MAIN ] CHE: Starting full primary refresh. [2025-09-16T17:45:28.376287] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-09. [2025-09-16T17:45:28.632862] [INFO ] [REMOTE ] CHE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:45:28.663686] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-09-16T17:45:28.801417] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-09-16T17:45:29.174919] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-02. [2025-09-16T17:45:29.957022] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-09-16T17:45:29.974856] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-03. [2025-09-16T17:45:30.157569] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T17:45:30.163489] [INFO ] [REMOTE ] CGAU: Full fetch complete (45918 records). [2025-09-16T17:45:30.163626] [DEBUG] [VALIDATOR ] CGAU: Validating primary data. [2025-09-16T17:45:30.582147] [DEBUG] [VALIDATOR ] CGAU: All expected trading days are present. [2025-09-16T17:45:30.582232] [DEBUG] [VALIDATOR ] CGAU: Primary data validation passed. [2025-09-16T17:45:30.582799] [INFO ] [MAIN ] CGAU: Fetched primary data is valid. [2025-09-16T17:45:30.582843] [DEBUG] [LOCAL ] CGAU: Saving primary data. [2025-09-16T17:45:30.583937] [DEBUG] [LOCAL ] CGAU: Trimmed 45918 records to 31389 (keeping 150 days of data) [2025-09-16T17:45:30.766356] [DEBUG] [LOCAL ] CGAU: Primary data saved (31389 records). [2025-09-16T17:45:30.766500] [INFO ] [MAIN ] CGAU: Full history saved. [2025-09-16T17:45:30.766634] [INFO ] [MAIN ] Update Progress: 434/2369 symbols completed. CGAU took 29.4s. Estimated remaining: 3h 22m 41s. Rate: 9.5 symbols/min [2025-09-16T17:45:30.780044] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-04. [2025-09-16T17:45:30.937772] [INFO ] [MAIN ] CHGG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:45:30.952517] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T17:45:30.981255] [DEBUG] [LOCAL ] CHGG: Loaded 35278 existing records from primary storage. [2025-09-16T17:45:30.981408] [INFO ] [MAIN ] CHGG: Loaded 35278 existing records. [2025-09-16T17:45:31.158612] [WARN ] [MAIN ] CHGG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:45:31.158886] [INFO ] [MAIN ] CHGG: Wiping existing primary data before full refresh. [2025-09-16T17:45:31.158933] [INFO ] [LOCAL ] CHGG: Starting deletion of primary data. [2025-09-16T17:45:31.159731] [DEBUG] [LOCAL ] CHGG: Deleted primary data file. [2025-09-16T17:45:31.159781] [INFO ] [MAIN ] CHGG: Starting full primary refresh. [2025-09-16T17:45:31.515294] [INFO ] [REMOTE ] CHGG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:45:31.574949] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-04. [2025-09-16T17:45:32.375357] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-02. [2025-09-16T17:45:32.562594] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-09-16T17:45:32.571586] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T17:45:33.175019] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-07. [2025-09-16T17:45:33.763807] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T17:45:33.974995] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-03. [2025-09-16T17:45:34.325934] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T17:45:34.774948] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-08. [2025-09-16T17:45:35.576215] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-05. [2025-09-16T17:45:35.584791] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T17:45:35.891891] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T17:45:36.374935] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-05. [2025-09-16T17:45:36.461664] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-09-16T17:45:37.174921] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-06. [2025-09-16T17:45:37.875693] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-09-16T17:45:37.981891] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-06. [2025-09-16T17:45:37.983541] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-09-16T17:45:38.774903] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-09. [2025-09-16T17:45:39.493562] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-09-16T17:45:39.574866] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-07. [2025-09-16T17:45:39.942336] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-09-16T17:45:39.945252] [INFO ] [REMOTE ] CHCT: Full fetch complete (26412 records). [2025-09-16T17:45:39.945348] [DEBUG] [VALIDATOR ] CHCT: Validating primary data. [2025-09-16T17:45:40.319529] [DEBUG] [VALIDATOR ] CHCT: All expected trading days are present. [2025-09-16T17:45:40.319626] [DEBUG] [VALIDATOR ] CHCT: Primary data validation passed. [2025-09-16T17:45:40.320098] [INFO ] [MAIN ] CHCT: Fetched primary data is valid. [2025-09-16T17:45:40.320144] [DEBUG] [LOCAL ] CHCT: Saving primary data. [2025-09-16T17:45:40.321032] [DEBUG] [LOCAL ] CHCT: Trimmed 26412 records to 17905 (keeping 150 days of data) [2025-09-16T17:45:40.377259] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-04. [2025-09-16T17:45:40.433111] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-09-16T17:45:40.453214] [DEBUG] [LOCAL ] CHCT: Primary data saved (17905 records). [2025-09-16T17:45:40.453373] [INFO ] [MAIN ] CHCT: Full history saved. [2025-09-16T17:45:40.453465] [INFO ] [MAIN ] Update Progress: 435/2369 symbols completed. CHCT took 31.2s. Estimated remaining: 3h 22m 50s. Rate: 9.5 symbols/min [2025-09-16T17:45:40.624335] [INFO ] [MAIN ] CHH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:45:40.647318] [DEBUG] [LOCAL ] CHH: Loaded 23316 existing records from primary storage. [2025-09-16T17:45:40.647452] [INFO ] [MAIN ] CHH: Loaded 23316 existing records. [2025-09-16T17:45:40.830524] [WARN ] [MAIN ] CHH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:45:40.830868] [INFO ] [MAIN ] CHH: Wiping existing primary data before full refresh. [2025-09-16T17:45:40.830936] [INFO ] [LOCAL ] CHH: Starting deletion of primary data. [2025-09-16T17:45:40.831591] [DEBUG] [LOCAL ] CHH: Deleted primary data file. [2025-09-16T17:45:40.831690] [INFO ] [MAIN ] CHH: Starting full primary refresh. [2025-09-16T17:45:41.176092] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-08. [2025-09-16T17:45:41.198849] [INFO ] [REMOTE ] CHH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:45:41.928993] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T17:45:41.974871] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-09. [2025-09-16T17:45:42.221526] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T17:45:42.777376] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-07. [2025-09-16T17:45:42.779284] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T17:45:42.782297] [INFO ] [REMOTE ] CHE: Full fetch complete (15404 records). [2025-09-16T17:45:42.782420] [DEBUG] [VALIDATOR ] CHE: Validating primary data. [2025-09-16T17:45:43.156265] [DEBUG] [VALIDATOR ] CHE: All expected trading days are present. [2025-09-16T17:45:43.156346] [DEBUG] [VALIDATOR ] CHE: Primary data validation passed. [2025-09-16T17:45:43.156730] [INFO ] [MAIN ] CHE: Fetched primary data is valid. [2025-09-16T17:45:43.156797] [DEBUG] [LOCAL ] CHE: Saving primary data. [2025-09-16T17:45:43.157328] [DEBUG] [LOCAL ] CHE: Trimmed 15404 records to 11229 (keeping 150 days of data) [2025-09-16T17:45:43.227892] [DEBUG] [LOCAL ] CHE: Primary data saved (11229 records). [2025-09-16T17:45:43.228034] [INFO ] [MAIN ] CHE: Full history saved. [2025-09-16T17:45:43.228123] [INFO ] [MAIN ] Update Progress: 436/2369 symbols completed. CHE took 15.3s. Estimated remaining: 3h 22m 28s. Rate: 9.5 symbols/min [2025-09-16T17:45:43.400222] [INFO ] [MAIN ] CHMI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:45:43.400382] [DEBUG] [LOCAL ] CHMI: No existing data file found in primary storage. [2025-09-16T17:45:43.400421] [WARN ] [MAIN ] CHMI: No existing data found, performing full refresh instead. [2025-09-16T17:45:43.400570] [INFO ] [MAIN ] CHMI: Wiping existing primary data before full refresh. [2025-09-16T17:45:43.400612] [INFO ] [LOCAL ] CHMI: Starting deletion of primary data. [2025-09-16T17:45:43.400702] [DEBUG] [LOCAL ] CHMI: No primary data file found to delete. [2025-09-16T17:45:43.400795] [INFO ] [MAIN ] CHMI: Starting full primary refresh. [2025-09-16T17:45:43.582786] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-05. [2025-09-16T17:45:43.771006] [INFO ] [REMOTE ] CHMI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:45:44.374929] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-02. [2025-09-16T17:45:44.609538] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-09-16T17:45:45.181112] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-02. [2025-09-16T17:45:45.201455] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T17:45:45.278843] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-09-16T17:45:45.974914] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-03. [2025-09-16T17:45:46.452794] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T17:45:46.774892] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-03. [2025-09-16T17:45:47.143473] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T17:45:47.574913] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-04. [2025-09-16T17:45:48.374890] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-08. [2025-09-16T17:45:48.472694] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-09-16T17:45:48.883015] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T17:45:49.174954] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-06. [2025-09-16T17:45:49.974921] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-05. [2025-09-16T17:45:50.128979] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T17:45:50.783542] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-04. [2025-09-16T17:45:50.811543] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T17:45:50.836087] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T17:45:51.574924] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-09. [2025-09-16T17:45:51.777213] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-09-16T17:45:52.374923] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-06. [2025-09-16T17:45:52.721809] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T17:45:52.727685] [INFO ] [REMOTE ] CHD: Full fetch complete (55983 records). [2025-09-16T17:45:52.727820] [DEBUG] [VALIDATOR ] CHD: Validating primary data. [2025-09-16T17:45:52.728491] [FATAL] [VALIDATOR ] CHD: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:45:52.728576] [FATAL] [MAIN ] CHD: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:45:52.728635] [WARN ] [MAIN ] CHD: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:45:52.728849] [INFO ] [LOCAL ] CHD: Starting deletion of primary data. [2025-09-16T17:45:52.728912] [DEBUG] [LOCAL ] CHD: No primary data file found to delete. [2025-09-16T17:45:52.728967] [INFO ] [LOCAL ] CHD: Starting deletion of derivative data. [2025-09-16T17:45:52.729333] [DEBUG] [LOCAL ] CHD: Deleted derivative data file. [2025-09-16T17:45:52.731839] [INFO ] [MAIN ] Update Progress: 437/2369 symbols completed. CHD took 32.3s. Estimated remaining: 3h 22m 36s. Rate: 9.5 symbols/min [2025-09-16T17:45:53.091753] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-09-16T17:45:53.174901] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-07. [2025-09-16T17:45:53.975000] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-05. [2025-09-16T17:45:54.662173] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T17:45:54.774892] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-08. [2025-09-16T17:45:54.820501] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-09-16T17:45:55.574908] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-06. [2025-09-16T17:45:56.107383] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-09-16T17:45:56.374909] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-07. [2025-09-16T17:45:56.906776] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T17:45:57.176856] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-09. [2025-09-16T17:45:57.181997] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-09-16T17:45:57.974910] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-07. [2025-09-16T17:45:58.625476] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-09-16T17:45:58.631704] [INFO ] [REMOTE ] CHGG: Full fetch complete (52209 records). [2025-09-16T17:45:58.631837] [DEBUG] [VALIDATOR ] CHGG: Validating primary data. [2025-09-16T17:45:58.783064] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-08. [2025-09-16T17:45:59.020528] [DEBUG] [VALIDATOR ] CHGG: All expected trading days are present. [2025-09-16T17:45:59.020715] [DEBUG] [VALIDATOR ] CHGG: Primary data validation passed. [2025-09-16T17:45:59.021352] [INFO ] [MAIN ] CHGG: Fetched primary data is valid. [2025-09-16T17:45:59.021396] [DEBUG] [LOCAL ] CHGG: Saving primary data. [2025-09-16T17:45:59.022459] [DEBUG] [LOCAL ] CHGG: Trimmed 52209 records to 35600 (keeping 150 days of data) [2025-09-16T17:45:59.136006] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-09-16T17:45:59.263070] [DEBUG] [LOCAL ] CHGG: Primary data saved (35600 records). [2025-09-16T17:45:59.263247] [INFO ] [MAIN ] CHGG: Full history saved. [2025-09-16T17:45:59.263349] [INFO ] [MAIN ] Update Progress: 438/2369 symbols completed. CHGG took 28.5s. Estimated remaining: 3h 22m 31s. Rate: 9.5 symbols/min [2025-09-16T17:45:59.490738] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-09-16T17:45:59.574894] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-09. [2025-09-16T17:46:00.374937] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-08. [2025-09-16T17:46:00.599629] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T17:46:00.603372] [INFO ] [REMOTE ] CHMI: Full fetch complete (23757 records). [2025-09-16T17:46:00.603483] [DEBUG] [VALIDATOR ] CHMI: Validating primary data. [2025-09-16T17:46:00.985184] [DEBUG] [VALIDATOR ] CHMI: All expected trading days are present. [2025-09-16T17:46:00.985277] [DEBUG] [VALIDATOR ] CHMI: Primary data validation passed. [2025-09-16T17:46:00.985724] [INFO ] [MAIN ] CHMI: Fetched primary data is valid. [2025-09-16T17:46:00.985770] [DEBUG] [LOCAL ] CHMI: Saving primary data. [2025-09-16T17:46:00.986538] [DEBUG] [LOCAL ] CHMI: Trimmed 23757 records to 15316 (keeping 150 days of data) [2025-09-16T17:46:01.081398] [DEBUG] [LOCAL ] CHMI: Primary data saved (15316 records). [2025-09-16T17:46:01.081555] [INFO ] [MAIN ] CHMI: Full history saved. [2025-09-16T17:46:01.081672] [INFO ] [MAIN ] Update Progress: 439/2369 symbols completed. CHMI took 17.9s. Estimated remaining: 3h 22m 5s. Rate: 9.6 symbols/min [2025-09-16T17:46:01.408025] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T17:46:01.415151] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-09. [2025-09-16T17:46:02.536469] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-09-16T17:46:02.540563] [INFO ] [REMOTE ] CHH: Full fetch complete (35560 records). [2025-09-16T17:46:02.540773] [DEBUG] [VALIDATOR ] CHH: Validating primary data. [2025-09-16T17:46:02.922448] [DEBUG] [VALIDATOR ] CHH: All expected trading days are present. [2025-09-16T17:46:02.922527] [DEBUG] [VALIDATOR ] CHH: Primary data validation passed. [2025-09-16T17:46:02.923060] [INFO ] [MAIN ] CHH: Fetched primary data is valid. [2025-09-16T17:46:02.923111] [DEBUG] [LOCAL ] CHH: Saving primary data. [2025-09-16T17:46:02.924019] [DEBUG] [LOCAL ] CHH: Trimmed 35560 records to 23518 (keeping 150 days of data) [2025-09-16T17:46:03.080069] [DEBUG] [LOCAL ] CHH: Primary data saved (23518 records). [2025-09-16T17:46:03.080221] [INFO ] [MAIN ] CHH: Full history saved. [2025-09-16T17:46:03.080314] [INFO ] [MAIN ] Update Progress: 440/2369 symbols completed. CHH took 22.6s. Estimated remaining: 3h 21m 40s. Rate: 9.6 symbols/min [2025-09-16T17:46:03.653706] [INFO ] [MAIN ] CHN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:03.744487] [DEBUG] [LOCAL ] CHN: No existing data file found in primary storage. [2025-09-16T17:46:03.826409] [INFO ] [MAIN ] CHPT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:03.826602] [WARN ] [MAIN ] CHN: No existing data found, performing full refresh instead. [2025-09-16T17:46:03.880862] [INFO ] [MAIN ] CHN: Wiping existing primary data before full refresh. [2025-09-16T17:46:03.880782] [INFO ] [MAIN ] CHWY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:03.879632] [INFO ] [MAIN ] CHT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:03.882077] [INFO ] [LOCAL ] CHN: Starting deletion of primary data. [2025-09-16T17:46:03.882840] [DEBUG] [LOCAL ] CHN: No primary data file found to delete. [2025-09-16T17:46:03.882894] [INFO ] [MAIN ] CHN: Starting full primary refresh. [2025-09-16T17:46:04.025604] [DEBUG] [LOCAL ] CHT: Loaded 14218 existing records from primary storage. [2025-09-16T17:46:04.066676] [INFO ] [MAIN ] CHT: Loaded 14218 existing records. [2025-09-16T17:46:04.291110] [DEBUG] [LOCAL ] CHWY: Loaded 39622 existing records from primary storage. [2025-09-16T17:46:04.338568] [INFO ] [MAIN ] CHWY: Loaded 39622 existing records. [2025-09-16T17:46:04.338790] [DEBUG] [LOCAL ] CHPT: Loaded 36128 existing records from primary storage. [2025-09-16T17:46:04.408594] [INFO ] [MAIN ] CHPT: Loaded 36128 existing records. [2025-09-16T17:46:04.672136] [INFO ] [REMOTE ] CHN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:04.742757] [WARN ] [MAIN ] CHT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:46:04.804328] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-02. [2025-09-16T17:46:04.804555] [INFO ] [MAIN ] CHT: Wiping existing primary data before full refresh. [2025-09-16T17:46:04.810293] [WARN ] [MAIN ] CHWY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:46:04.810397] [INFO ] [LOCAL ] CHT: Starting deletion of primary data. [2025-09-16T17:46:04.821971] [INFO ] [MAIN ] CHWY: Wiping existing primary data before full refresh. [2025-09-16T17:46:04.837748] [INFO ] [LOCAL ] CHWY: Starting deletion of primary data. [2025-09-16T17:46:04.827201] [DEBUG] [LOCAL ] CHT: Deleted primary data file. [2025-09-16T17:46:04.888513] [INFO ] [MAIN ] CHT: Starting full primary refresh. [2025-09-16T17:46:04.899288] [DEBUG] [LOCAL ] CHWY: Deleted primary data file. [2025-09-16T17:46:04.905761] [WARN ] [MAIN ] CHPT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:46:04.932053] [INFO ] [MAIN ] CHWY: Starting full primary refresh. [2025-09-16T17:46:04.939172] [INFO ] [MAIN ] CHPT: Wiping existing primary data before full refresh. [2025-09-16T17:46:04.956693] [INFO ] [LOCAL ] CHPT: Starting deletion of primary data. [2025-09-16T17:46:05.045468] [DEBUG] [LOCAL ] CHPT: Deleted primary data file. [2025-09-16T17:46:05.094113] [INFO ] [MAIN ] CHPT: Starting full primary refresh. [2025-09-16T17:46:05.586961] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-09-16T17:46:05.620681] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-03. [2025-09-16T17:46:05.883245] [INFO ] [REMOTE ] CHT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:05.904434] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-02. [2025-09-16T17:46:06.004542] [INFO ] [REMOTE ] CHWY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:06.025931] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-02. [2025-09-16T17:46:06.029618] [INFO ] [REMOTE ] CHPT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:06.030494] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-02. [2025-09-16T17:46:06.035173] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-09-16T17:46:06.036945] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-04. [2025-09-16T17:46:06.513412] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-09-16T17:46:06.774906] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-05. [2025-09-16T17:46:06.995476] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T17:46:07.199040] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T17:46:07.513195] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T17:46:07.574860] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-03. [2025-09-16T17:46:07.973039] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-02 (1.94 seconds). [2025-09-16T17:46:08.374899] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-03. [2025-09-16T17:46:09.147729] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-09-16T17:46:09.184928] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-06. [2025-09-16T17:46:09.190897] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T17:46:09.604329] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-09-16T17:46:09.974920] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-07. [2025-09-16T17:46:10.652756] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-09-16T17:46:10.774909] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-04. [2025-09-16T17:46:11.574900] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-03. [2025-09-16T17:46:11.637675] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-09-16T17:46:12.374902] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-08. [2025-09-16T17:46:12.785238] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-09-16T17:46:13.187544] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-04. [2025-09-16T17:46:13.213156] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T17:46:13.974892] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-09. [2025-09-16T17:46:14.470045] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-09-16T17:46:14.471233] [INFO ] [REMOTE ] CHN: Full fetch complete (5463 records). [2025-09-16T17:46:14.471329] [DEBUG] [VALIDATOR ] CHN: Validating primary data. [2025-09-16T17:46:14.778101] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-05. [2025-09-16T17:46:14.863162] [DEBUG] [VALIDATOR ] CHN: All expected trading days are present. [2025-09-16T17:46:14.863344] [DEBUG] [VALIDATOR ] CHN: Primary data validation passed. [2025-09-16T17:46:14.863591] [INFO ] [MAIN ] CHN: Fetched primary data is valid. [2025-09-16T17:46:14.863808] [DEBUG] [LOCAL ] CHN: Saving primary data. [2025-09-16T17:46:14.864466] [DEBUG] [LOCAL ] CHN: Trimmed 5463 records to 3643 (keeping 150 days of data) [2025-09-16T17:46:14.919093] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T17:46:14.935739] [DEBUG] [LOCAL ] CHN: Primary data saved (3643 records). [2025-09-16T17:46:14.935836] [INFO ] [MAIN ] CHN: Full history saved. [2025-09-16T17:46:14.935917] [INFO ] [MAIN ] Update Progress: 441/2369 symbols completed. CHN took 11.9s. Estimated remaining: 3h 21m 58s. Rate: 9.5 symbols/min [2025-09-16T17:46:15.129845] [INFO ] [MAIN ] CI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:15.178963] [DEBUG] [LOCAL ] CI: Loaded 36930 existing records from primary storage. [2025-09-16T17:46:15.179139] [INFO ] [MAIN ] CI: Loaded 36930 existing records. [2025-09-16T17:46:15.356307] [WARN ] [MAIN ] CI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:46:15.356527] [INFO ] [MAIN ] CI: Wiping existing primary data before full refresh. [2025-09-16T17:46:15.356572] [INFO ] [LOCAL ] CI: Starting deletion of primary data. [2025-09-16T17:46:15.357459] [DEBUG] [LOCAL ] CI: Deleted primary data file. [2025-09-16T17:46:15.357518] [INFO ] [MAIN ] CI: Starting full primary refresh. [2025-09-16T17:46:15.580015] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-05. [2025-09-16T17:46:15.749604] [INFO ] [REMOTE ] CI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:15.776860] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T17:46:16.374897] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-04. [2025-09-16T17:46:17.174880] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-02. [2025-09-16T17:46:17.259062] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T17:46:17.974904] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-06. [2025-09-16T17:46:18.272274] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-09-16T17:46:18.699363] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T17:46:18.774907] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-05. [2025-09-16T17:46:18.977452] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T17:46:19.574902] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-03. [2025-09-16T17:46:20.374881] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-07. [2025-09-16T17:46:20.764176] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-09-16T17:46:21.176655] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-06. [2025-09-16T17:46:21.179449] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T17:46:21.338407] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T17:46:21.974915] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-04. [2025-09-16T17:46:22.774900] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-06. [2025-09-16T17:46:23.391479] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T17:46:23.579967] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-08. [2025-09-16T17:46:23.602006] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-06 (2.43 seconds). [2025-09-16T17:46:24.302149] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-09-16T17:46:24.374970] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-09. [2025-09-16T17:46:25.008988] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-06 (2.23 seconds). [2025-09-16T17:46:25.174980] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-07. [2025-09-16T17:46:25.285865] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-09-16T17:46:25.290225] [INFO ] [REMOTE ] CHT: Full fetch complete (21465 records). [2025-09-16T17:46:25.290409] [DEBUG] [VALIDATOR ] CHT: Validating primary data. [2025-09-16T17:46:25.684011] [DEBUG] [VALIDATOR ] CHT: All expected trading days are present. [2025-09-16T17:46:25.684093] [DEBUG] [VALIDATOR ] CHT: Primary data validation passed. [2025-09-16T17:46:25.684489] [INFO ] [MAIN ] CHT: Fetched primary data is valid. [2025-09-16T17:46:25.684530] [DEBUG] [LOCAL ] CHT: Saving primary data. [2025-09-16T17:46:25.685131] [DEBUG] [LOCAL ] CHT: Trimmed 21465 records to 14367 (keeping 150 days of data) [2025-09-16T17:46:25.775704] [DEBUG] [LOCAL ] CHT: Primary data saved (14367 records). [2025-09-16T17:46:25.775844] [INFO ] [MAIN ] CHT: Full history saved. [2025-09-16T17:46:25.775936] [INFO ] [MAIN ] Update Progress: 442/2369 symbols completed. CHT took 22.7s. Estimated remaining: 3h 22m 12s. Rate: 9.5 symbols/min [2025-09-16T17:46:25.948003] [INFO ] [MAIN ] CIA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:25.961248] [DEBUG] [LOCAL ] CIA: Loaded 12731 existing records from primary storage. [2025-09-16T17:46:25.961368] [INFO ] [MAIN ] CIA: Loaded 12731 existing records. [2025-09-16T17:46:25.982264] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-05. [2025-09-16T17:46:26.146084] [WARN ] [MAIN ] CIA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:46:26.146315] [INFO ] [MAIN ] CIA: Wiping existing primary data before full refresh. [2025-09-16T17:46:26.146360] [INFO ] [LOCAL ] CIA: Starting deletion of primary data. [2025-09-16T17:46:26.146785] [DEBUG] [LOCAL ] CIA: Deleted primary data file. [2025-09-16T17:46:26.146834] [INFO ] [MAIN ] CIA: Starting full primary refresh. [2025-09-16T17:46:26.502429] [INFO ] [REMOTE ] CIA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:26.774919] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-02. [2025-09-16T17:46:27.042256] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T17:46:27.574940] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-08. [2025-09-16T17:46:27.744920] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T17:46:27.819548] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T17:46:28.374901] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-03. [2025-09-16T17:46:29.174919] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-06. [2025-09-16T17:46:29.362866] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-09-16T17:46:29.557543] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-09-16T17:46:29.974900] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-09. [2025-09-16T17:46:30.773045] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-09-16T17:46:30.782430] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-04. [2025-09-16T17:46:31.574954] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-07. [2025-09-16T17:46:31.714505] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T17:46:31.825313] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-09-16T17:46:31.831838] [INFO ] [REMOTE ] CHWY: Full fetch complete (58297 records). [2025-09-16T17:46:31.831972] [DEBUG] [VALIDATOR ] CHWY: Validating primary data. [2025-09-16T17:46:32.233377] [DEBUG] [VALIDATOR ] CHWY: All expected trading days are present. [2025-09-16T17:46:32.233455] [DEBUG] [VALIDATOR ] CHWY: Primary data validation passed. [2025-09-16T17:46:32.234160] [INFO ] [MAIN ] CHWY: Fetched primary data is valid. [2025-09-16T17:46:32.234209] [DEBUG] [LOCAL ] CHWY: Saving primary data. [2025-09-16T17:46:32.235356] [DEBUG] [LOCAL ] CHWY: Trimmed 58297 records to 40012 (keeping 150 days of data) [2025-09-16T17:46:32.413684] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-05. [2025-09-16T17:46:32.491139] [DEBUG] [LOCAL ] CHWY: Primary data saved (40012 records). [2025-09-16T17:46:32.491271] [INFO ] [MAIN ] CHWY: Full history saved. [2025-09-16T17:46:32.492627] [INFO ] [MAIN ] Update Progress: 443/2369 symbols completed. CHWY took 29.4s. Estimated remaining: 3h 22m 7s. Rate: 9.5 symbols/min [2025-09-16T17:46:32.669668] [INFO ] [MAIN ] CIB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:32.694245] [DEBUG] [LOCAL ] CIB: Loaded 21773 existing records from primary storage. [2025-09-16T17:46:32.694395] [INFO ] [MAIN ] CIB: Loaded 21773 existing records. [2025-09-16T17:46:32.883012] [WARN ] [MAIN ] CIB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:46:32.883238] [INFO ] [MAIN ] CIB: Wiping existing primary data before full refresh. [2025-09-16T17:46:32.883283] [INFO ] [LOCAL ] CIB: Starting deletion of primary data. [2025-09-16T17:46:32.883900] [DEBUG] [LOCAL ] CIB: Deleted primary data file. [2025-09-16T17:46:32.883948] [INFO ] [MAIN ] CIB: Starting full primary refresh. [2025-09-16T17:46:33.177664] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-07. [2025-09-16T17:46:33.239563] [INFO ] [REMOTE ] CIB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:33.354264] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-09-16T17:46:33.398859] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T17:46:33.974924] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-08. [2025-09-16T17:46:34.774931] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-02. [2025-09-16T17:46:35.366734] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-09-16T17:46:35.548305] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T17:46:35.574861] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-09. [2025-09-16T17:46:35.863898] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T17:46:36.374942] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-08. [2025-09-16T17:46:37.020918] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-09-16T17:46:37.027179] [INFO ] [REMOTE ] CI: Full fetch complete (54303 records). [2025-09-16T17:46:37.027341] [DEBUG] [VALIDATOR ] CI: Validating primary data. [2025-09-16T17:46:37.180089] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-06. [2025-09-16T17:46:37.424341] [DEBUG] [VALIDATOR ] CI: All expected trading days are present. [2025-09-16T17:46:37.424419] [DEBUG] [VALIDATOR ] CI: Primary data validation passed. [2025-09-16T17:46:37.425051] [INFO ] [MAIN ] CI: Fetched primary data is valid. [2025-09-16T17:46:37.425097] [DEBUG] [LOCAL ] CI: Saving primary data. [2025-09-16T17:46:37.426130] [DEBUG] [LOCAL ] CI: Trimmed 54303 records to 37304 (keeping 150 days of data) [2025-09-16T17:46:37.663520] [DEBUG] [LOCAL ] CI: Primary data saved (37304 records). [2025-09-16T17:46:37.663672] [INFO ] [MAIN ] CI: Full history saved. [2025-09-16T17:46:37.665006] [INFO ] [MAIN ] Update Progress: 444/2369 symbols completed. CI took 22.7s. Estimated remaining: 3h 21m 56s. Rate: 9.5 symbols/min [2025-09-16T17:46:37.841422] [INFO ] [MAIN ] CICB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:37.841620] [DEBUG] [LOCAL ] CICB: No existing data file found in primary storage. [2025-09-16T17:46:37.841713] [WARN ] [MAIN ] CICB: No existing data found, performing full refresh instead. [2025-09-16T17:46:37.841840] [INFO ] [MAIN ] CICB: Wiping existing primary data before full refresh. [2025-09-16T17:46:37.841880] [INFO ] [LOCAL ] CICB: Starting deletion of primary data. [2025-09-16T17:46:37.841922] [DEBUG] [LOCAL ] CICB: No primary data file found to delete. [2025-09-16T17:46:37.841964] [INFO ] [MAIN ] CICB: Starting full primary refresh. [2025-09-16T17:46:37.987393] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-03. [2025-09-16T17:46:38.207834] [INFO ] [REMOTE ] CICB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:38.233706] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-09-16T17:46:38.376765] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T17:46:38.774957] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-09. [2025-09-16T17:46:39.127370] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T17:46:39.574981] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-07. [2025-09-16T17:46:40.176131] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:46:40.180854] [INFO ] [REMOTE ] CHPT: Full fetch complete (54587 records). [2025-09-16T17:46:40.180943] [DEBUG] [VALIDATOR ] CHPT: Validating primary data. [2025-09-16T17:46:40.382710] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-02. [2025-09-16T17:46:40.563492] [DEBUG] [VALIDATOR ] CHPT: All expected trading days are present. [2025-09-16T17:46:40.563578] [DEBUG] [VALIDATOR ] CHPT: Primary data validation passed. [2025-09-16T17:46:40.564239] [INFO ] [MAIN ] CHPT: Fetched primary data is valid. [2025-09-16T17:46:40.564287] [DEBUG] [LOCAL ] CHPT: Saving primary data. [2025-09-16T17:46:40.565589] [DEBUG] [LOCAL ] CHPT: Trimmed 54587 records to 36364 (keeping 150 days of data) [2025-09-16T17:46:40.690128] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T17:46:40.799925] [DEBUG] [LOCAL ] CHPT: Primary data saved (36364 records). [2025-09-16T17:46:40.800072] [INFO ] [MAIN ] CHPT: Full history saved. [2025-09-16T17:46:40.801529] [INFO ] [MAIN ] Update Progress: 445/2369 symbols completed. CHPT took 37.7s. Estimated remaining: 3h 21m 36s. Rate: 9.5 symbols/min [2025-09-16T17:46:40.905639] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-09-16T17:46:40.975979] [INFO ] [MAIN ] CIEN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:41.017674] [DEBUG] [LOCAL ] CIEN: Loaded 38272 existing records from primary storage. [2025-09-16T17:46:41.017819] [INFO ] [MAIN ] CIEN: Loaded 38272 existing records. [2025-09-16T17:46:41.178462] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-08. [2025-09-16T17:46:41.198705] [WARN ] [MAIN ] CIEN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:46:41.198856] [INFO ] [MAIN ] CIEN: Wiping existing primary data before full refresh. [2025-09-16T17:46:41.198900] [INFO ] [LOCAL ] CIEN: Starting deletion of primary data. [2025-09-16T17:46:41.199710] [DEBUG] [LOCAL ] CIEN: Deleted primary data file. [2025-09-16T17:46:41.199762] [INFO ] [MAIN ] CIEN: Starting full primary refresh. [2025-09-16T17:46:41.547871] [INFO ] [REMOTE ] CIEN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:41.974856] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-03. [2025-09-16T17:46:42.280720] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-09-16T17:46:42.378964] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-09-16T17:46:42.774913] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-04. [2025-09-16T17:46:43.574939] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-04. [2025-09-16T17:46:44.027045] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-09-16T17:46:44.088167] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T17:46:44.374975] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-09. [2025-09-16T17:46:45.158739] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-09-16T17:46:45.160609] [INFO ] [REMOTE ] CIA: Full fetch complete (18158 records). [2025-09-16T17:46:45.160702] [DEBUG] [VALIDATOR ] CIA: Validating primary data. [2025-09-16T17:46:45.181489] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-05. [2025-09-16T17:46:45.529349] [DEBUG] [VALIDATOR ] CIA: All expected trading days are present. [2025-09-16T17:46:45.529427] [DEBUG] [VALIDATOR ] CIA: Primary data validation passed. [2025-09-16T17:46:45.529808] [INFO ] [MAIN ] CIA: Fetched primary data is valid. [2025-09-16T17:46:45.529855] [DEBUG] [LOCAL ] CIA: Saving primary data. [2025-09-16T17:46:45.530453] [DEBUG] [LOCAL ] CIA: Trimmed 18158 records to 12780 (keeping 150 days of data) [2025-09-16T17:46:45.583632] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-09-16T17:46:45.603198] [DEBUG] [LOCAL ] CIA: Primary data saved (12780 records). [2025-09-16T17:46:45.603282] [INFO ] [MAIN ] CIA: Full history saved. [2025-09-16T17:46:45.603356] [INFO ] [MAIN ] Update Progress: 446/2369 symbols completed. CIA took 19.8s. Estimated remaining: 3h 21m 23s. Rate: 9.5 symbols/min [2025-09-16T17:46:45.773244] [INFO ] [MAIN ] CIF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:45.778314] [DEBUG] [LOCAL ] CIF: Loaded 2943 existing records from primary storage. [2025-09-16T17:46:45.778413] [INFO ] [MAIN ] CIF: Loaded 2943 existing records. [2025-09-16T17:46:45.953974] [WARN ] [MAIN ] CIF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:46:45.954207] [INFO ] [MAIN ] CIF: Wiping existing primary data before full refresh. [2025-09-16T17:46:45.954251] [INFO ] [LOCAL ] CIF: Starting deletion of primary data. [2025-09-16T17:46:45.954471] [DEBUG] [LOCAL ] CIF: Deleted primary data file. [2025-09-16T17:46:45.954514] [INFO ] [MAIN ] CIF: Starting full primary refresh. [2025-09-16T17:46:45.983448] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-06. [2025-09-16T17:46:46.317313] [INFO ] [REMOTE ] CIF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:46.438805] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-09-16T17:46:46.774871] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-02. [2025-09-16T17:46:47.574958] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-02. [2025-09-16T17:46:48.331999] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T17:46:48.374903] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-05. [2025-09-16T17:46:48.491252] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-09-16T17:46:49.174898] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-03. [2025-09-16T17:46:49.974952] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-03. [2025-09-16T17:46:50.112963] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T17:46:50.461346] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-09-16T17:46:50.774894] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-06. [2025-09-16T17:46:51.007336] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T17:46:51.574944] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-04. [2025-09-16T17:46:51.759631] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T17:46:52.374914] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-07. [2025-09-16T17:46:52.952028] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-09-16T17:46:53.174929] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-04. [2025-09-16T17:46:53.380299] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T17:46:53.653504] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-09-16T17:46:53.974918] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-05. [2025-09-16T17:46:54.379670] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-09-16T17:46:54.774899] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-08. [2025-09-16T17:46:55.098994] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-09-16T17:46:55.574897] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-09. [2025-09-16T17:46:55.999319] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-09-16T17:46:56.000236] [INFO ] [REMOTE ] CICB: Full fetch complete (1438 records). [2025-09-16T17:46:56.000313] [DEBUG] [VALIDATOR ] CICB: Validating primary data. [2025-09-16T17:46:56.190966] [WARN ] [VALIDATOR ] CICB: Validation failed - insufficient historical data. Have 147 unique days, need 150 days [2025-09-16T17:46:56.191184] [FATAL] [MAIN ] CICB: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:46:56.191230] [WARN ] [MAIN ] CICB: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:46:56.191270] [INFO ] [LOCAL ] CICB: Starting deletion of primary data. [2025-09-16T17:46:56.191326] [DEBUG] [LOCAL ] CICB: No primary data file found to delete. [2025-09-16T17:46:56.191364] [INFO ] [LOCAL ] CICB: Starting deletion of derivative data. [2025-09-16T17:46:56.191415] [DEBUG] [LOCAL ] CICB: No derivative data file found to delete. [2025-09-16T17:46:56.191476] [INFO ] [MAIN ] Update Progress: 447/2369 symbols completed. CICB took 18.5s. Estimated remaining: 3h 21m 36s. Rate: 9.5 symbols/min [2025-09-16T17:46:56.366096] [INFO ] [MAIN ] CIG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:46:56.374912] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-06. [2025-09-16T17:46:56.400453] [DEBUG] [LOCAL ] CIG: Loaded 32541 existing records from primary storage. [2025-09-16T17:46:56.400582] [INFO ] [MAIN ] CIG: Loaded 32541 existing records. [2025-09-16T17:46:56.575782] [WARN ] [MAIN ] CIG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:46:56.576059] [INFO ] [MAIN ] CIG: Wiping existing primary data before full refresh. [2025-09-16T17:46:56.576107] [INFO ] [LOCAL ] CIG: Starting deletion of primary data. [2025-09-16T17:46:56.576884] [DEBUG] [LOCAL ] CIG: Deleted primary data file. [2025-09-16T17:46:56.576936] [INFO ] [MAIN ] CIG: Starting full primary refresh. [2025-09-16T17:46:56.933386] [INFO ] [REMOTE ] CIG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:46:56.952897] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-09-16T17:46:57.174898] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-02. [2025-09-16T17:46:57.974902] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-07. [2025-09-16T17:46:58.613674] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T17:46:58.774915] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-05. [2025-09-16T17:46:59.227352] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-09-16T17:46:59.574899] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-08. [2025-09-16T17:47:00.374901] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-03. [2025-09-16T17:47:00.613378] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T17:47:01.174905] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-07. [2025-09-16T17:47:01.228128] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T17:47:01.833337] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-09-16T17:47:01.974898] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-09. [2025-09-16T17:47:02.116490] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T17:47:02.774894] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-04. [2025-09-16T17:47:02.823755] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T17:47:02.828097] [INFO ] [REMOTE ] CIB: Full fetch complete (34168 records). [2025-09-16T17:47:02.828202] [DEBUG] [VALIDATOR ] CIB: Validating primary data. [2025-09-16T17:47:03.214770] [DEBUG] [VALIDATOR ] CIB: All expected trading days are present. [2025-09-16T17:47:03.214994] [DEBUG] [VALIDATOR ] CIB: Primary data validation passed. [2025-09-16T17:47:03.215542] [INFO ] [MAIN ] CIB: Fetched primary data is valid. [2025-09-16T17:47:03.215586] [DEBUG] [LOCAL ] CIB: Saving primary data. [2025-09-16T17:47:03.216607] [DEBUG] [LOCAL ] CIB: Trimmed 34168 records to 21993 (keeping 150 days of data) [2025-09-16T17:47:03.346106] [DEBUG] [LOCAL ] CIB: Primary data saved (21993 records). [2025-09-16T17:47:03.346240] [INFO ] [MAIN ] CIB: Full history saved. [2025-09-16T17:47:03.346323] [INFO ] [MAIN ] Update Progress: 448/2369 symbols completed. CIB took 30.9s. Estimated remaining: 3h 21m 33s. Rate: 9.5 symbols/min [2025-09-16T17:47:03.519199] [INFO ] [MAIN ] CII: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:03.530404] [DEBUG] [LOCAL ] CII: Loaded 10090 existing records from primary storage. [2025-09-16T17:47:03.530518] [INFO ] [MAIN ] CII: Loaded 10090 existing records. [2025-09-16T17:47:03.586894] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-08. [2025-09-16T17:47:03.714279] [WARN ] [MAIN ] CII: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:03.714556] [INFO ] [MAIN ] CII: Wiping existing primary data before full refresh. [2025-09-16T17:47:03.714603] [INFO ] [LOCAL ] CII: Starting deletion of primary data. [2025-09-16T17:47:03.715222] [DEBUG] [LOCAL ] CII: Deleted primary data file. [2025-09-16T17:47:03.715272] [INFO ] [MAIN ] CII: Starting full primary refresh. [2025-09-16T17:47:04.074940] [INFO ] [REMOTE ] CII: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:04.375047] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-02. [2025-09-16T17:47:04.377258] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T17:47:04.411085] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T17:47:05.109351] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-09-16T17:47:05.174853] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-03. [2025-09-16T17:47:05.960244] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T17:47:05.974840] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-04. [2025-09-16T17:47:06.774892] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-05. [2025-09-16T17:47:06.927529] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-09-16T17:47:07.574903] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-06. [2025-09-16T17:47:08.002395] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-09-16T17:47:08.374897] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-06. [2025-09-16T17:47:09.182967] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-05. [2025-09-16T17:47:09.209817] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T17:47:09.827476] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-09-16T17:47:09.974902] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-07. [2025-09-16T17:47:10.774894] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-09. [2025-09-16T17:47:10.946327] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-06 (2.57 seconds). [2025-09-16T17:47:11.244880] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-09-16T17:47:11.245926] [INFO ] [REMOTE ] CIF: Full fetch complete (5117 records). [2025-09-16T17:47:11.246004] [DEBUG] [VALIDATOR ] CIF: Validating primary data. [2025-09-16T17:47:11.580002] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-07. [2025-09-16T17:47:11.623917] [DEBUG] [VALIDATOR ] CIF: All expected trading days are present. [2025-09-16T17:47:11.624015] [DEBUG] [VALIDATOR ] CIF: Primary data validation passed. [2025-09-16T17:47:11.624261] [INFO ] [MAIN ] CIF: Fetched primary data is valid. [2025-09-16T17:47:11.624310] [DEBUG] [LOCAL ] CIF: Saving primary data. [2025-09-16T17:47:11.624749] [DEBUG] [LOCAL ] CIF: Trimmed 5117 records to 2981 (keeping 150 days of data) [2025-09-16T17:47:11.644107] [DEBUG] [LOCAL ] CIF: Primary data saved (2981 records). [2025-09-16T17:47:11.644208] [INFO ] [MAIN ] CIF: Full history saved. [2025-09-16T17:47:11.644305] [INFO ] [MAIN ] Update Progress: 449/2369 symbols completed. CIF took 26.0s. Estimated remaining: 3h 21m 35s. Rate: 9.5 symbols/min [2025-09-16T17:47:11.825159] [INFO ] [MAIN ] CIM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:11.854196] [DEBUG] [LOCAL ] CIM: Loaded 26357 existing records from primary storage. [2025-09-16T17:47:11.854427] [INFO ] [MAIN ] CIM: Loaded 26357 existing records. [2025-09-16T17:47:12.054564] [WARN ] [MAIN ] CIM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:12.068485] [INFO ] [MAIN ] CIM: Wiping existing primary data before full refresh. [2025-09-16T17:47:12.068630] [INFO ] [LOCAL ] CIM: Starting deletion of primary data. [2025-09-16T17:47:12.070364] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-09-16T17:47:12.070405] [DEBUG] [LOCAL ] CIM: Deleted primary data file. [2025-09-16T17:47:12.080892] [INFO ] [MAIN ] CIM: Starting full primary refresh. [2025-09-16T17:47:12.380047] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-06. [2025-09-16T17:47:12.458736] [INFO ] [REMOTE ] CIM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:13.174905] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-02. [2025-09-16T17:47:13.377794] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T17:47:13.554028] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-09-16T17:47:13.974901] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-07. [2025-09-16T17:47:14.575931] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-09-16T17:47:14.774887] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-08. [2025-09-16T17:47:14.945106] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T17:47:15.574902] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-08. [2025-09-16T17:47:16.330244] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-09-16T17:47:16.377435] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-08. [2025-09-16T17:47:16.383521] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-09-16T17:47:17.174973] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-09. [2025-09-16T17:47:17.861437] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T17:47:17.974899] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-09. [2025-09-16T17:47:18.774900] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-09. [2025-09-16T17:47:18.935819] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T17:47:18.940986] [INFO ] [REMOTE ] CIG: Full fetch complete (46968 records). [2025-09-16T17:47:18.941094] [DEBUG] [VALIDATOR ] CIG: Validating primary data. [2025-09-16T17:47:19.315917] [DEBUG] [VALIDATOR ] CIG: All expected trading days are present. [2025-09-16T17:47:19.316015] [DEBUG] [VALIDATOR ] CIG: Primary data validation passed. [2025-09-16T17:47:19.316746] [INFO ] [MAIN ] CIG: Fetched primary data is valid. [2025-09-16T17:47:19.316821] [DEBUG] [LOCAL ] CIG: Saving primary data. [2025-09-16T17:47:19.317943] [DEBUG] [LOCAL ] CIG: Trimmed 46968 records to 32909 (keeping 150 days of data) [2025-09-16T17:47:19.515908] [DEBUG] [LOCAL ] CIG: Primary data saved (32909 records). [2025-09-16T17:47:19.516083] [INFO ] [MAIN ] CIG: Full history saved. [2025-09-16T17:47:19.517337] [INFO ] [MAIN ] Update Progress: 450/2369 symbols completed. CIG took 23.3s. Estimated remaining: 3h 21m 36s. Rate: 9.5 symbols/min [2025-09-16T17:47:19.583249] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-03. [2025-09-16T17:47:19.700546] [INFO ] [MAIN ] CIMN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:19.708468] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T17:47:19.714302] [INFO ] [REMOTE ] CII: Full fetch complete (16884 records). [2025-09-16T17:47:19.714152] [DEBUG] [LOCAL ] CIMN: Loaded 1932 existing records from primary storage. [2025-09-16T17:47:19.714522] [DEBUG] [VALIDATOR ] CII: Validating primary data. [2025-09-16T17:47:19.714609] [INFO ] [MAIN ] CIMN: Loaded 1932 existing records. [2025-09-16T17:47:19.942652] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-09-16T17:47:20.002857] [INFO ] [REMOTE ] CIEN: Full fetch complete (56666 records). [2025-09-16T17:47:20.115278] [WARN ] [MAIN ] CIMN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:20.050690] [DEBUG] [VALIDATOR ] CIEN: Validating primary data. [2025-09-16T17:47:20.178730] [INFO ] [MAIN ] CIMN: Wiping existing primary data before full refresh. [2025-09-16T17:47:20.219695] [INFO ] [LOCAL ] CIMN: Starting deletion of primary data. [2025-09-16T17:47:20.256443] [DEBUG] [LOCAL ] CIMN: Deleted primary data file. [2025-09-16T17:47:20.288129] [INFO ] [MAIN ] CIMN: Starting full primary refresh. [2025-09-16T17:47:20.652488] [DEBUG] [VALIDATOR ] CII: All expected trading days are present. [2025-09-16T17:47:20.728982] [DEBUG] [VALIDATOR ] CII: Primary data validation passed. [2025-09-16T17:47:20.831168] [INFO ] [MAIN ] CII: Fetched primary data is valid. [2025-09-16T17:47:20.852330] [DEBUG] [LOCAL ] CII: Saving primary data. [2025-09-16T17:47:20.951216] [DEBUG] [LOCAL ] CII: Trimmed 16884 records to 10219 (keeping 150 days of data) [2025-09-16T17:47:21.167017] [INFO ] [REMOTE ] CIMN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:21.170887] [DEBUG] [VALIDATOR ] CIEN: All expected trading days are present. [2025-09-16T17:47:21.171152] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-02. [2025-09-16T17:47:21.173813] [DEBUG] [VALIDATOR ] CIEN: Primary data validation passed. [2025-09-16T17:47:21.174745] [INFO ] [MAIN ] CIEN: Fetched primary data is valid. [2025-09-16T17:47:21.175446] [DEBUG] [LOCAL ] CIEN: Saving primary data. [2025-09-16T17:47:21.188609] [DEBUG] [LOCAL ] CIEN: Trimmed 56666 records to 38661 (keeping 150 days of data) [2025-09-16T17:47:21.305180] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T17:47:21.307389] [DEBUG] [LOCAL ] CII: Primary data saved (10219 records). [2025-09-16T17:47:21.307884] [INFO ] [MAIN ] CII: Full history saved. [2025-09-16T17:47:21.316209] [INFO ] [MAIN ] Update Progress: 451/2369 symbols completed. CII took 18.0s. Estimated remaining: 3h 21m 10s. Rate: 9.5 symbols/min [2025-09-16T17:47:21.317376] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-04. [2025-09-16T17:47:21.506252] [INFO ] [MAIN ] CIMO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:21.514673] [DEBUG] [LOCAL ] CIMO: Loaded 2027 existing records from primary storage. [2025-09-16T17:47:21.517336] [INFO ] [MAIN ] CIMO: Loaded 2027 existing records. [2025-09-16T17:47:21.755081] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-09-16T17:47:21.808691] [WARN ] [MAIN ] CIMO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:21.814746] [INFO ] [MAIN ] CIMO: Wiping existing primary data before full refresh. [2025-09-16T17:47:21.816761] [INFO ] [LOCAL ] CIMO: Starting deletion of primary data. [2025-09-16T17:47:21.819448] [DEBUG] [LOCAL ] CIMO: Deleted primary data file. [2025-09-16T17:47:21.820000] [INFO ] [MAIN ] CIMO: Starting full primary refresh. [2025-09-16T17:47:21.989782] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-03. [2025-09-16T17:47:22.121717] [DEBUG] [LOCAL ] CIEN: Primary data saved (38661 records). [2025-09-16T17:47:22.121868] [INFO ] [MAIN ] CIEN: Full history saved. [2025-09-16T17:47:22.121971] [INFO ] [MAIN ] Update Progress: 452/2369 symbols completed. CIEN took 41.3s. Estimated remaining: 3h 20m 41s. Rate: 9.6 symbols/min [2025-09-16T17:47:22.306128] [INFO ] [REMOTE ] CIMO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:22.380521] [INFO ] [MAIN ] CINT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:22.394137] [DEBUG] [LOCAL ] CINT: Loaded 13303 existing records from primary storage. [2025-09-16T17:47:22.394251] [INFO ] [MAIN ] CINT: Loaded 13303 existing records. [2025-09-16T17:47:22.559995] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-09-16T17:47:22.589471] [WARN ] [MAIN ] CINT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:22.590972] [INFO ] [MAIN ] CINT: Wiping existing primary data before full refresh. [2025-09-16T17:47:22.591051] [INFO ] [LOCAL ] CINT: Starting deletion of primary data. [2025-09-16T17:47:22.591449] [DEBUG] [LOCAL ] CINT: Deleted primary data file. [2025-09-16T17:47:22.591497] [INFO ] [MAIN ] CINT: Starting full primary refresh. [2025-09-16T17:47:22.781979] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-04. [2025-09-16T17:47:22.961961] [INFO ] [REMOTE ] CINT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:22.987440] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T17:47:23.296228] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-09-16T17:47:23.574897] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-02. [2025-09-16T17:47:24.374905] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-02. [2025-09-16T17:47:24.398868] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T17:47:24.951457] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-09-16T17:47:25.174920] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-03. [2025-09-16T17:47:25.877080] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-09-16T17:47:25.974934] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-03. [2025-09-16T17:47:26.420059] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-09-16T17:47:26.774896] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-05. [2025-09-16T17:47:27.574902] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-04. [2025-09-16T17:47:28.269210] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-09-16T17:47:28.293586] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T17:47:28.374855] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-06. [2025-09-16T17:47:29.174927] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-05. [2025-09-16T17:47:29.594062] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T17:47:29.625036] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T17:47:29.974942] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-06. [2025-09-16T17:47:30.372956] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-09-16T17:47:30.774917] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-05. [2025-09-16T17:47:31.308231] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-09-16T17:47:31.574941] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-07. [2025-09-16T17:47:32.065927] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-09-16T17:47:32.374902] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-08. [2025-09-16T17:47:32.930202] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-09-16T17:47:33.174951] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-06. [2025-09-16T17:47:33.670490] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-09-16T17:47:33.974959] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-04. [2025-09-16T17:47:34.774939] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-09. [2025-09-16T17:47:35.276355] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-09-16T17:47:35.560522] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T17:47:35.561711] [INFO ] [REMOTE ] CIMO: Full fetch complete (3167 records). [2025-09-16T17:47:35.561782] [DEBUG] [VALIDATOR ] CIMO: Validating primary data. [2025-09-16T17:47:35.583901] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-07. [2025-09-16T17:47:35.939085] [DEBUG] [VALIDATOR ] CIMO: All expected trading days are present. [2025-09-16T17:47:35.939202] [DEBUG] [VALIDATOR ] CIMO: Primary data validation passed. [2025-09-16T17:47:35.939328] [INFO ] [MAIN ] CIMO: Fetched primary data is valid. [2025-09-16T17:47:35.939369] [DEBUG] [LOCAL ] CIMO: Saving primary data. [2025-09-16T17:47:35.939788] [DEBUG] [LOCAL ] CIMO: Trimmed 3167 records to 2059 (keeping 150 days of data) [2025-09-16T17:47:35.953179] [DEBUG] [LOCAL ] CIMO: Primary data saved (2059 records). [2025-09-16T17:47:35.953306] [INFO ] [MAIN ] CIMO: Full history saved. [2025-09-16T17:47:35.953395] [INFO ] [MAIN ] Update Progress: 453/2369 symbols completed. CIMO took 14.6s. Estimated remaining: 3h 21m 6s. Rate: 9.5 symbols/min [2025-09-16T17:47:36.125997] [INFO ] [MAIN ] CIO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:36.140568] [DEBUG] [LOCAL ] CIO: Loaded 14449 existing records from primary storage. [2025-09-16T17:47:36.140770] [INFO ] [MAIN ] CIO: Loaded 14449 existing records. [2025-09-16T17:47:36.322346] [WARN ] [MAIN ] CIO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:36.322593] [INFO ] [MAIN ] CIO: Wiping existing primary data before full refresh. [2025-09-16T17:47:36.322639] [INFO ] [LOCAL ] CIO: Starting deletion of primary data. [2025-09-16T17:47:36.323115] [DEBUG] [LOCAL ] CIO: Deleted primary data file. [2025-09-16T17:47:36.323192] [INFO ] [MAIN ] CIO: Starting full primary refresh. [2025-09-16T17:47:36.380585] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-05. [2025-09-16T17:47:36.693178] [INFO ] [REMOTE ] CIO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:37.041995] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T17:47:37.174858] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-02. [2025-09-16T17:47:37.300952] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-09-16T17:47:37.974870] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-07. [2025-09-16T17:47:38.236705] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T17:47:38.387092] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-09-16T17:47:38.774893] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-08. [2025-09-16T17:47:39.199412] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-09-16T17:47:39.574886] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-08. [2025-09-16T17:47:40.374907] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-09. [2025-09-16T17:47:40.866079] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-09-16T17:47:40.866899] [INFO ] [REMOTE ] CIMN: Full fetch complete (2854 records). [2025-09-16T17:47:40.866965] [DEBUG] [VALIDATOR ] CIMN: Validating primary data. [2025-09-16T17:47:41.178030] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-03. [2025-09-16T17:47:41.231046] [DEBUG] [VALIDATOR ] CIMN: All expected trading days are present. [2025-09-16T17:47:41.231116] [DEBUG] [VALIDATOR ] CIMN: Primary data validation passed. [2025-09-16T17:47:41.231334] [INFO ] [MAIN ] CIMN: Fetched primary data is valid. [2025-09-16T17:47:41.231413] [DEBUG] [LOCAL ] CIMN: Saving primary data. [2025-09-16T17:47:41.231912] [DEBUG] [LOCAL ] CIMN: Trimmed 2854 records to 1937 (keeping 150 days of data) [2025-09-16T17:47:41.263756] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T17:47:41.273914] [DEBUG] [LOCAL ] CIMN: Primary data saved (1937 records). [2025-09-16T17:47:41.273997] [INFO ] [MAIN ] CIMN: Full history saved. [2025-09-16T17:47:41.274083] [INFO ] [MAIN ] Update Progress: 454/2369 symbols completed. CIMN took 21.8s. Estimated remaining: 3h 20m 56s. Rate: 9.5 symbols/min [2025-09-16T17:47:41.441972] [INFO ] [MAIN ] CION: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:41.463846] [DEBUG] [LOCAL ] CION: Loaded 20264 existing records from primary storage. [2025-09-16T17:47:41.463969] [INFO ] [MAIN ] CION: Loaded 20264 existing records. [2025-09-16T17:47:41.639470] [WARN ] [MAIN ] CION: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:41.639700] [INFO ] [MAIN ] CION: Wiping existing primary data before full refresh. [2025-09-16T17:47:41.639777] [INFO ] [LOCAL ] CION: Starting deletion of primary data. [2025-09-16T17:47:41.640312] [DEBUG] [LOCAL ] CION: Deleted primary data file. [2025-09-16T17:47:41.640366] [INFO ] [MAIN ] CION: Starting full primary refresh. [2025-09-16T17:47:41.986474] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-06. [2025-09-16T17:47:41.990439] [INFO ] [REMOTE ] CION: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:42.160275] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T17:47:42.774900] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-02. [2025-09-16T17:47:42.918288] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-09-16T17:47:43.574903] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-04. [2025-09-16T17:47:43.974496] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-09-16T17:47:44.238952] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-09-16T17:47:44.374940] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-09. [2025-09-16T17:47:45.174951] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-07. [2025-09-16T17:47:45.429876] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T17:47:45.434308] [INFO ] [REMOTE ] CIM: Full fetch complete (40778 records). [2025-09-16T17:47:45.434396] [DEBUG] [VALIDATOR ] CIM: Validating primary data. [2025-09-16T17:47:45.837575] [DEBUG] [VALIDATOR ] CIM: All expected trading days are present. [2025-09-16T17:47:45.837694] [DEBUG] [VALIDATOR ] CIM: Primary data validation passed. [2025-09-16T17:47:45.838253] [INFO ] [MAIN ] CIM: Fetched primary data is valid. [2025-09-16T17:47:45.838300] [DEBUG] [LOCAL ] CIM: Saving primary data. [2025-09-16T17:47:45.839828] [DEBUG] [LOCAL ] CIM: Trimmed 40778 records to 26648 (keeping 150 days of data) [2025-09-16T17:47:45.984029] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-05. [2025-09-16T17:47:46.010679] [DEBUG] [LOCAL ] CIM: Primary data saved (26648 records). [2025-09-16T17:47:46.010834] [INFO ] [MAIN ] CIM: Full history saved. [2025-09-16T17:47:46.010929] [INFO ] [MAIN ] Update Progress: 455/2369 symbols completed. CIM took 34.4s. Estimated remaining: 3h 20m 43s. Rate: 9.5 symbols/min [2025-09-16T17:47:46.188728] [INFO ] [MAIN ] CIVI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:46.201048] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-09-16T17:47:46.246902] [DEBUG] [LOCAL ] CIVI: Loaded 38791 existing records from primary storage. [2025-09-16T17:47:46.247097] [INFO ] [MAIN ] CIVI: Loaded 38791 existing records. [2025-09-16T17:47:46.426058] [WARN ] [MAIN ] CIVI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:46.426355] [INFO ] [MAIN ] CIVI: Wiping existing primary data before full refresh. [2025-09-16T17:47:46.426411] [INFO ] [LOCAL ] CIVI: Starting deletion of primary data. [2025-09-16T17:47:46.427383] [DEBUG] [LOCAL ] CIVI: Deleted primary data file. [2025-09-16T17:47:46.427435] [INFO ] [MAIN ] CIVI: Starting full primary refresh. [2025-09-16T17:47:46.784012] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-03. [2025-09-16T17:47:46.788967] [INFO ] [REMOTE ] CIVI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:46.946803] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-09-16T17:47:47.575020] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-02. [2025-09-16T17:47:47.985073] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T17:47:48.374956] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-08. [2025-09-16T17:47:49.174947] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-06. [2025-09-16T17:47:49.194552] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-09-16T17:47:49.358607] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T17:47:49.974866] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-04. [2025-09-16T17:47:50.055420] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-09-16T17:47:50.774917] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-07. [2025-09-16T17:47:51.554667] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T17:47:51.574862] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-05. [2025-09-16T17:47:51.724966] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-09-16T17:47:52.374889] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-08. [2025-09-16T17:47:52.838882] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-09-16T17:47:53.072199] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-09-16T17:47:53.174856] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-09. [2025-09-16T17:47:53.974933] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-06. [2025-09-16T17:47:54.253501] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T17:47:54.256594] [INFO ] [REMOTE ] CIO: Full fetch complete (21245 records). [2025-09-16T17:47:54.256705] [DEBUG] [VALIDATOR ] CIO: Validating primary data. [2025-09-16T17:47:54.636023] [DEBUG] [VALIDATOR ] CIO: All expected trading days are present. [2025-09-16T17:47:54.636129] [DEBUG] [VALIDATOR ] CIO: Primary data validation passed. [2025-09-16T17:47:54.636556] [INFO ] [MAIN ] CIO: Fetched primary data is valid. [2025-09-16T17:47:54.636607] [DEBUG] [LOCAL ] CIO: Saving primary data. [2025-09-16T17:47:54.637792] [DEBUG] [LOCAL ] CIO: Trimmed 21245 records to 14659 (keeping 150 days of data) [2025-09-16T17:47:54.723088] [DEBUG] [LOCAL ] CIO: Primary data saved (14659 records). [2025-09-16T17:47:54.723248] [INFO ] [MAIN ] CIO: Full history saved. [2025-09-16T17:47:54.723364] [INFO ] [MAIN ] Update Progress: 456/2369 symbols completed. CIO took 18.8s. Estimated remaining: 3h 20m 47s. Rate: 9.5 symbols/min [2025-09-16T17:47:54.787224] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-09. [2025-09-16T17:47:54.900392] [INFO ] [MAIN ] CL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:54.945446] [DEBUG] [LOCAL ] CL: Loaded 39609 existing records from primary storage. [2025-09-16T17:47:54.945695] [INFO ] [MAIN ] CL: Loaded 39609 existing records. [2025-09-16T17:47:55.123753] [WARN ] [MAIN ] CL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:55.124184] [INFO ] [MAIN ] CL: Wiping existing primary data before full refresh. [2025-09-16T17:47:55.124270] [INFO ] [LOCAL ] CL: Starting deletion of primary data. [2025-09-16T17:47:55.132898] [DEBUG] [LOCAL ] CL: Deleted primary data file. [2025-09-16T17:47:55.144242] [INFO ] [MAIN ] CL: Starting full primary refresh. [2025-09-16T17:47:55.238539] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T17:47:55.514487] [INFO ] [REMOTE ] CL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:55.574877] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-02. [2025-09-16T17:47:55.779881] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T17:47:55.782831] [INFO ] [REMOTE ] CINT: Full fetch complete (19910 records). [2025-09-16T17:47:55.782909] [DEBUG] [VALIDATOR ] CINT: Validating primary data. [2025-09-16T17:47:56.150323] [DEBUG] [VALIDATOR ] CINT: All expected trading days are present. [2025-09-16T17:47:56.150400] [DEBUG] [VALIDATOR ] CINT: Primary data validation passed. [2025-09-16T17:47:56.150827] [INFO ] [MAIN ] CINT: Fetched primary data is valid. [2025-09-16T17:47:56.150899] [DEBUG] [LOCAL ] CINT: Saving primary data. [2025-09-16T17:47:56.151596] [DEBUG] [LOCAL ] CINT: Trimmed 19910 records to 13656 (keeping 150 days of data) [2025-09-16T17:47:56.232670] [DEBUG] [LOCAL ] CINT: Primary data saved (13656 records). [2025-09-16T17:47:56.232830] [INFO ] [MAIN ] CINT: Full history saved. [2025-09-16T17:47:56.232946] [INFO ] [MAIN ] Update Progress: 457/2369 symbols completed. CINT took 34.1s. Estimated remaining: 3h 20m 21s. Rate: 9.5 symbols/min [2025-09-16T17:47:56.380542] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-03. [2025-09-16T17:47:56.404776] [INFO ] [MAIN ] CLB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:47:56.429629] [DEBUG] [LOCAL ] CLB: Loaded 24105 existing records from primary storage. [2025-09-16T17:47:56.429793] [INFO ] [MAIN ] CLB: Loaded 24105 existing records. [2025-09-16T17:47:56.604828] [WARN ] [MAIN ] CLB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:47:56.605083] [INFO ] [MAIN ] CLB: Wiping existing primary data before full refresh. [2025-09-16T17:47:56.605128] [INFO ] [LOCAL ] CLB: Starting deletion of primary data. [2025-09-16T17:47:56.605888] [DEBUG] [LOCAL ] CLB: Deleted primary data file. [2025-09-16T17:47:56.605947] [INFO ] [MAIN ] CLB: Starting full primary refresh. [2025-09-16T17:47:56.972736] [INFO ] [REMOTE ] CLB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:47:57.062933] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T17:47:57.174898] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-03. [2025-09-16T17:47:57.974918] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-02. [2025-09-16T17:47:58.051557] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T17:47:58.774937] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-07. [2025-09-16T17:47:58.869897] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-09-16T17:47:59.223191] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T17:47:59.574878] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-03. [2025-09-16T17:47:59.945899] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T17:48:00.374901] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-08. [2025-09-16T17:48:01.013318] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T17:48:01.174902] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-04. [2025-09-16T17:48:01.918763] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T17:48:01.974946] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-04. [2025-09-16T17:48:02.637706] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T17:48:02.774915] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-05. [2025-09-16T17:48:03.574926] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-04. [2025-09-16T17:48:03.853950] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T17:48:03.925748] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T17:48:04.374942] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-06. [2025-09-16T17:48:05.174862] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-05. [2025-09-16T17:48:05.544948] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-09-16T17:48:05.660509] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T17:48:05.974903] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-07. [2025-09-16T17:48:06.774908] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-09. [2025-09-16T17:48:07.007740] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-09-16T17:48:07.392024] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-09-16T17:48:07.574945] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-05. [2025-09-16T17:48:08.052054] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-09-16T17:48:08.055911] [INFO ] [REMOTE ] CION: Full fetch complete (31485 records). [2025-09-16T17:48:08.056007] [DEBUG] [VALIDATOR ] CION: Validating primary data. [2025-09-16T17:48:08.380141] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-06. [2025-09-16T17:48:08.441456] [DEBUG] [VALIDATOR ] CION: All expected trading days are present. [2025-09-16T17:48:08.441548] [DEBUG] [VALIDATOR ] CION: Primary data validation passed. [2025-09-16T17:48:08.442195] [INFO ] [MAIN ] CION: Fetched primary data is valid. [2025-09-16T17:48:08.442262] [DEBUG] [LOCAL ] CION: Saving primary data. [2025-09-16T17:48:08.443262] [DEBUG] [LOCAL ] CION: Trimmed 31485 records to 20480 (keeping 150 days of data) [2025-09-16T17:48:08.574873] [DEBUG] [LOCAL ] CION: Primary data saved (20480 records). [2025-09-16T17:48:08.575002] [INFO ] [MAIN ] CION: Full history saved. [2025-09-16T17:48:08.575112] [INFO ] [MAIN ] Update Progress: 458/2369 symbols completed. CION took 27.3s. Estimated remaining: 3h 20m 40s. Rate: 9.5 symbols/min [2025-09-16T17:48:08.745472] [INFO ] [MAIN ] CLCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:48:08.754094] [DEBUG] [LOCAL ] CLCO: Loaded 7925 existing records from primary storage. [2025-09-16T17:48:08.754193] [INFO ] [MAIN ] CLCO: Loaded 7925 existing records. [2025-09-16T17:48:08.931911] [WARN ] [MAIN ] CLCO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:48:08.932190] [INFO ] [MAIN ] CLCO: Wiping existing primary data before full refresh. [2025-09-16T17:48:08.932237] [INFO ] [LOCAL ] CLCO: Starting deletion of primary data. [2025-09-16T17:48:08.932583] [DEBUG] [LOCAL ] CLCO: Deleted primary data file. [2025-09-16T17:48:08.932639] [INFO ] [MAIN ] CLCO: Starting full primary refresh. [2025-09-16T17:48:09.184958] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-08. [2025-09-16T17:48:09.330046] [INFO ] [REMOTE ] CLCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:48:09.423192] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-09-16T17:48:09.974915] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-06. [2025-09-16T17:48:10.231416] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-09-16T17:48:10.597389] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-09-16T17:48:10.774893] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-07. [2025-09-16T17:48:11.200909] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T17:48:11.574950] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-07. [2025-09-16T17:48:12.116957] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-09-16T17:48:12.374901] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-08. [2025-09-16T17:48:13.174913] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-02. [2025-09-16T17:48:13.575607] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-09-16T17:48:13.974906] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-08. [2025-09-16T17:48:14.017287] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T17:48:14.317005] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-09-16T17:48:14.774887] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-03. [2025-09-16T17:48:15.574891] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-09. [2025-09-16T17:48:15.909932] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-09-16T17:48:15.934875] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T17:48:16.374970] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-09. [2025-09-16T17:48:17.071804] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T17:48:17.078162] [INFO ] [REMOTE ] CIVI: Full fetch complete (56727 records). [2025-09-16T17:48:17.078316] [DEBUG] [VALIDATOR ] CIVI: Validating primary data. [2025-09-16T17:48:17.079121] [FATAL] [VALIDATOR ] CIVI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:48:17.079207] [FATAL] [MAIN ] CIVI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:48:17.079247] [WARN ] [MAIN ] CIVI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:48:17.079299] [INFO ] [LOCAL ] CIVI: Starting deletion of primary data. [2025-09-16T17:48:17.079372] [DEBUG] [LOCAL ] CIVI: No primary data file found to delete. [2025-09-16T17:48:17.079413] [INFO ] [LOCAL ] CIVI: Starting deletion of derivative data. [2025-09-16T17:48:17.079762] [DEBUG] [LOCAL ] CIVI: Deleted derivative data file. [2025-09-16T17:48:17.079850] [INFO ] [MAIN ] Update Progress: 459/2369 symbols completed. CIVI took 31.1s. Estimated remaining: 3h 20m 43s. Rate: 9.5 symbols/min [2025-09-16T17:48:17.180158] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-04. [2025-09-16T17:48:17.265859] [INFO ] [MAIN ] CLDT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:48:17.284516] [DEBUG] [LOCAL ] CLDT: Loaded 17486 existing records from primary storage. [2025-09-16T17:48:17.284704] [INFO ] [MAIN ] CLDT: Loaded 17486 existing records. [2025-09-16T17:48:17.473859] [WARN ] [MAIN ] CLDT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:48:17.474095] [INFO ] [MAIN ] CLDT: Wiping existing primary data before full refresh. [2025-09-16T17:48:17.474141] [INFO ] [LOCAL ] CLDT: Starting deletion of primary data. [2025-09-16T17:48:17.474618] [DEBUG] [LOCAL ] CLDT: Deleted primary data file. [2025-09-16T17:48:17.474695] [INFO ] [MAIN ] CLDT: Starting full primary refresh. [2025-09-16T17:48:17.848688] [INFO ] [REMOTE ] CLDT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:48:17.861917] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T17:48:17.868061] [INFO ] [REMOTE ] CLB: Full fetch complete (35847 records). [2025-09-16T17:48:17.868255] [DEBUG] [VALIDATOR ] CLB: Validating primary data. [2025-09-16T17:48:17.982718] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-02. [2025-09-16T17:48:18.266412] [DEBUG] [VALIDATOR ] CLB: All expected trading days are present. [2025-09-16T17:48:18.266503] [DEBUG] [VALIDATOR ] CLB: Primary data validation passed. [2025-09-16T17:48:18.267110] [INFO ] [MAIN ] CLB: Fetched primary data is valid. [2025-09-16T17:48:18.267155] [DEBUG] [LOCAL ] CLB: Saving primary data. [2025-09-16T17:48:18.268081] [DEBUG] [LOCAL ] CLB: Trimmed 35847 records to 24336 (keeping 150 days of data) [2025-09-16T17:48:18.371192] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:48:18.435088] [DEBUG] [LOCAL ] CLB: Primary data saved (24336 records). [2025-09-16T17:48:18.435227] [INFO ] [MAIN ] CLB: Full history saved. [2025-09-16T17:48:18.435325] [INFO ] [MAIN ] Update Progress: 460/2369 symbols completed. CLB took 22.2s. Estimated remaining: 3h 20m 16s. Rate: 9.5 symbols/min [2025-09-16T17:48:18.606047] [INFO ] [MAIN ] CLF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:48:18.645179] [DEBUG] [LOCAL ] CLF: Loaded 39702 existing records from primary storage. [2025-09-16T17:48:18.645330] [INFO ] [MAIN ] CLF: Loaded 39702 existing records. [2025-09-16T17:48:18.788625] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-09. [2025-09-16T17:48:18.827731] [WARN ] [MAIN ] CLF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:48:18.827983] [INFO ] [MAIN ] CLF: Wiping existing primary data before full refresh. [2025-09-16T17:48:18.828030] [INFO ] [LOCAL ] CLF: Starting deletion of primary data. [2025-09-16T17:48:18.829338] [DEBUG] [LOCAL ] CLF: Deleted primary data file. [2025-09-16T17:48:18.829387] [INFO ] [MAIN ] CLF: Starting full primary refresh. [2025-09-16T17:48:19.184554] [INFO ] [REMOTE ] CLF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:48:19.256163] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T17:48:19.574934] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-03. [2025-09-16T17:48:20.374907] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-05. [2025-09-16T17:48:20.680419] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-09-16T17:48:20.902794] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-09 (2.11 seconds). [2025-09-16T17:48:20.909162] [INFO ] [REMOTE ] CL: Full fetch complete (58319 records). [2025-09-16T17:48:20.909276] [DEBUG] [VALIDATOR ] CL: Validating primary data. [2025-09-16T17:48:21.177564] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-04. [2025-09-16T17:48:21.234580] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T17:48:21.310028] [DEBUG] [VALIDATOR ] CL: All expected trading days are present. [2025-09-16T17:48:21.310120] [DEBUG] [VALIDATOR ] CL: Primary data validation passed. [2025-09-16T17:48:21.310916] [INFO ] [MAIN ] CL: Fetched primary data is valid. [2025-09-16T17:48:21.310995] [DEBUG] [LOCAL ] CL: Saving primary data. [2025-09-16T17:48:21.312287] [DEBUG] [LOCAL ] CL: Trimmed 58319 records to 39999 (keeping 150 days of data) [2025-09-16T17:48:21.561625] [DEBUG] [LOCAL ] CL: Primary data saved (39999 records). [2025-09-16T17:48:21.561799] [INFO ] [MAIN ] CL: Full history saved. [2025-09-16T17:48:21.563229] [INFO ] [MAIN ] Update Progress: 461/2369 symbols completed. CL took 26.8s. Estimated remaining: 3h 19m 56s. Rate: 9.5 symbols/min [2025-09-16T17:48:21.741766] [INFO ] [MAIN ] CLH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:48:21.764895] [DEBUG] [LOCAL ] CLH: Loaded 19570 existing records from primary storage. [2025-09-16T17:48:21.765046] [INFO ] [MAIN ] CLH: Loaded 19570 existing records. [2025-09-16T17:48:21.945553] [WARN ] [MAIN ] CLH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:48:21.945848] [INFO ] [MAIN ] CLH: Wiping existing primary data before full refresh. [2025-09-16T17:48:21.945895] [INFO ] [LOCAL ] CLH: Starting deletion of primary data. [2025-09-16T17:48:21.946402] [DEBUG] [LOCAL ] CLH: Deleted primary data file. [2025-09-16T17:48:21.946447] [INFO ] [MAIN ] CLH: Starting full primary refresh. [2025-09-16T17:48:21.979988] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-02. [2025-09-16T17:48:22.302472] [INFO ] [REMOTE ] CLH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:48:22.656580] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T17:48:22.774890] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-06. [2025-09-16T17:48:23.410137] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-09-16T17:48:23.574955] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-02. [2025-09-16T17:48:24.032107] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-09-16T17:48:24.374934] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-03. [2025-09-16T17:48:24.773510] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-09-16T17:48:25.174915] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-05. [2025-09-16T17:48:25.974948] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-03. [2025-09-16T17:48:26.105752] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T17:48:26.400516] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-09-16T17:48:26.774888] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-07. [2025-09-16T17:48:27.124679] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T17:48:27.574931] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-06. [2025-09-16T17:48:27.597067] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-09-16T17:48:28.374941] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-04. [2025-09-16T17:48:28.664156] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-09-16T17:48:29.174962] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-04. [2025-09-16T17:48:29.763285] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T17:48:29.974978] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-08. [2025-09-16T17:48:30.504271] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-09-16T17:48:30.774885] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-09. [2025-09-16T17:48:31.240627] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-09-16T17:48:31.574908] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-05. [2025-09-16T17:48:31.591825] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-09-16T17:48:31.593360] [INFO ] [REMOTE ] CLCO: Full fetch complete (15521 records). [2025-09-16T17:48:31.593440] [DEBUG] [VALIDATOR ] CLCO: Validating primary data. [2025-09-16T17:48:31.964854] [DEBUG] [VALIDATOR ] CLCO: All expected trading days are present. [2025-09-16T17:48:31.964945] [DEBUG] [VALIDATOR ] CLCO: Primary data validation passed. [2025-09-16T17:48:31.965350] [INFO ] [MAIN ] CLCO: Fetched primary data is valid. [2025-09-16T17:48:31.965401] [DEBUG] [LOCAL ] CLCO: Saving primary data. [2025-09-16T17:48:31.966014] [DEBUG] [LOCAL ] CLCO: Trimmed 15521 records to 7998 (keeping 150 days of data) [2025-09-16T17:48:32.012924] [DEBUG] [LOCAL ] CLCO: Primary data saved (7998 records). [2025-09-16T17:48:32.013070] [INFO ] [MAIN ] CLCO: Full history saved. [2025-09-16T17:48:32.013163] [INFO ] [MAIN ] Update Progress: 462/2369 symbols completed. CLCO took 23.4s. Estimated remaining: 3h 20m 7s. Rate: 9.5 symbols/min [2025-09-16T17:48:32.191678] [INFO ] [MAIN ] CLPR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:48:32.200626] [DEBUG] [LOCAL ] CLPR: Loaded 7760 existing records from primary storage. [2025-09-16T17:48:32.200750] [INFO ] [MAIN ] CLPR: Loaded 7760 existing records. [2025-09-16T17:48:32.380873] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-05. [2025-09-16T17:48:32.386577] [WARN ] [MAIN ] CLPR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:48:32.386928] [INFO ] [MAIN ] CLPR: Wiping existing primary data before full refresh. [2025-09-16T17:48:32.387032] [INFO ] [LOCAL ] CLPR: Starting deletion of primary data. [2025-09-16T17:48:32.387371] [DEBUG] [LOCAL ] CLPR: Deleted primary data file. [2025-09-16T17:48:32.387592] [INFO ] [MAIN ] CLPR: Starting full primary refresh. [2025-09-16T17:48:32.573736] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T17:48:32.780106] [INFO ] [REMOTE ] CLPR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:48:33.174908] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-02. [2025-09-16T17:48:33.871454] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-09-16T17:48:33.974920] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-03. [2025-09-16T17:48:34.415308] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-09-16T17:48:34.774913] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-07. [2025-09-16T17:48:34.924755] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-09-16T17:48:35.486016] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-09-16T17:48:35.574988] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-06. [2025-09-16T17:48:36.374940] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-06. [2025-09-16T17:48:36.685150] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-09-16T17:48:37.174916] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-08. [2025-09-16T17:48:37.974911] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-04. [2025-09-16T17:48:38.204745] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T17:48:38.289202] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-09-16T17:48:38.566178] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-09-16T17:48:38.774893] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-07. [2025-09-16T17:48:39.574917] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-09. [2025-09-16T17:48:40.374881] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-07. [2025-09-16T17:48:40.827793] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-09-16T17:48:40.830883] [INFO ] [REMOTE ] CLDT: Full fetch complete (28602 records). [2025-09-16T17:48:40.830983] [DEBUG] [VALIDATOR ] CLDT: Validating primary data. [2025-09-16T17:48:41.195445] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-05. [2025-09-16T17:48:41.265435] [DEBUG] [VALIDATOR ] CLDT: All expected trading days are present. [2025-09-16T17:48:41.266212] [DEBUG] [VALIDATOR ] CLDT: Primary data validation passed. [2025-09-16T17:48:41.269959] [INFO ] [MAIN ] CLDT: Fetched primary data is valid. [2025-09-16T17:48:41.269315] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-07 (2.49 seconds). [2025-09-16T17:48:41.270151] [DEBUG] [LOCAL ] CLDT: Saving primary data. [2025-09-16T17:48:41.286075] [DEBUG] [LOCAL ] CLDT: Trimmed 28602 records to 17684 (keeping 150 days of data) [2025-09-16T17:48:41.391570] [DEBUG] [LOCAL ] CLDT: Primary data saved (17684 records). [2025-09-16T17:48:41.391755] [INFO ] [MAIN ] CLDT: Full history saved. [2025-09-16T17:48:41.391845] [INFO ] [MAIN ] Update Progress: 463/2369 symbols completed. CLDT took 24.3s. Estimated remaining: 3h 20m 14s. Rate: 9.5 symbols/min [2025-09-16T17:48:41.574261] [INFO ] [MAIN ] CLS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:48:41.615200] [DEBUG] [LOCAL ] CLS: Loaded 39616 existing records from primary storage. [2025-09-16T17:48:41.615419] [INFO ] [MAIN ] CLS: Loaded 39616 existing records. [2025-09-16T17:48:41.745165] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T17:48:41.832547] [WARN ] [MAIN ] CLS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:48:41.833012] [INFO ] [MAIN ] CLS: Wiping existing primary data before full refresh. [2025-09-16T17:48:41.833064] [INFO ] [LOCAL ] CLS: Starting deletion of primary data. [2025-09-16T17:48:41.834014] [DEBUG] [LOCAL ] CLS: Deleted primary data file. [2025-09-16T17:48:41.834069] [INFO ] [MAIN ] CLS: Starting full primary refresh. [2025-09-16T17:48:41.977458] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-08. [2025-09-16T17:48:42.179358] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-09-16T17:48:42.201945] [INFO ] [REMOTE ] CLS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:48:42.774881] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-06. [2025-09-16T17:48:43.328696] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-09-16T17:48:43.359881] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T17:48:43.574917] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-08. [2025-09-16T17:48:44.374941] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-09. [2025-09-16T17:48:45.174922] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-02. [2025-09-16T17:48:45.510977] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T17:48:45.514802] [INFO ] [REMOTE ] CLH: Full fetch complete (31781 records). [2025-09-16T17:48:45.514959] [DEBUG] [VALIDATOR ] CLH: Validating primary data. [2025-09-16T17:48:45.650726] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-09-16T17:48:45.929089] [DEBUG] [VALIDATOR ] CLH: All expected trading days are present. [2025-09-16T17:48:45.929180] [DEBUG] [VALIDATOR ] CLH: Primary data validation passed. [2025-09-16T17:48:45.929678] [INFO ] [MAIN ] CLH: Fetched primary data is valid. [2025-09-16T17:48:45.929724] [DEBUG] [LOCAL ] CLH: Saving primary data. [2025-09-16T17:48:45.930614] [DEBUG] [LOCAL ] CLH: Trimmed 31781 records to 19781 (keeping 150 days of data) [2025-09-16T17:48:45.983977] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-09. [2025-09-16T17:48:46.056370] [DEBUG] [LOCAL ] CLH: Primary data saved (19781 records). [2025-09-16T17:48:46.056521] [INFO ] [MAIN ] CLH: Full history saved. [2025-09-16T17:48:46.056610] [INFO ] [MAIN ] Update Progress: 464/2369 symbols completed. CLH took 24.5s. Estimated remaining: 3h 20m 1s. Rate: 9.5 symbols/min [2025-09-16T17:48:46.228065] [INFO ] [MAIN ] CLVT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:48:46.269400] [DEBUG] [LOCAL ] CLVT: Loaded 38400 existing records from primary storage. [2025-09-16T17:48:46.269565] [INFO ] [MAIN ] CLVT: Loaded 38400 existing records. [2025-09-16T17:48:46.446829] [WARN ] [MAIN ] CLVT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:48:46.447059] [INFO ] [MAIN ] CLVT: Wiping existing primary data before full refresh. [2025-09-16T17:48:46.447105] [INFO ] [LOCAL ] CLVT: Starting deletion of primary data. [2025-09-16T17:48:46.447952] [DEBUG] [LOCAL ] CLVT: Deleted primary data file. [2025-09-16T17:48:46.448003] [INFO ] [MAIN ] CLVT: Starting full primary refresh. [2025-09-16T17:48:46.787270] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-07. [2025-09-16T17:48:46.791557] [INFO ] [REMOTE ] CLVT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:48:47.436389] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-02 (2.26 seconds). [2025-09-16T17:48:47.574861] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-03. [2025-09-16T17:48:47.744024] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-09-16T17:48:48.050157] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-09-16T17:48:48.056473] [INFO ] [REMOTE ] CLF: Full fetch complete (58422 records). [2025-09-16T17:48:48.056571] [DEBUG] [VALIDATOR ] CLF: Validating primary data. [2025-09-16T17:48:48.387775] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-08. [2025-09-16T17:48:48.437860] [DEBUG] [VALIDATOR ] CLF: All expected trading days are present. [2025-09-16T17:48:48.437935] [DEBUG] [VALIDATOR ] CLF: Primary data validation passed. [2025-09-16T17:48:48.438603] [INFO ] [MAIN ] CLF: Fetched primary data is valid. [2025-09-16T17:48:48.438662] [DEBUG] [LOCAL ] CLF: Saving primary data. [2025-09-16T17:48:48.439818] [DEBUG] [LOCAL ] CLF: Trimmed 58422 records to 40092 (keeping 150 days of data) [2025-09-16T17:48:48.678110] [DEBUG] [LOCAL ] CLF: Primary data saved (40092 records). [2025-09-16T17:48:48.678240] [INFO ] [MAIN ] CLF: Full history saved. [2025-09-16T17:48:48.678321] [INFO ] [MAIN ] Update Progress: 465/2369 symbols completed. CLF took 30.2s. Estimated remaining: 3h 19m 39s. Rate: 9.5 symbols/min [2025-09-16T17:48:48.849549] [INFO ] [MAIN ] CLW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:48:48.865165] [DEBUG] [LOCAL ] CLW: Loaded 15618 existing records from primary storage. [2025-09-16T17:48:48.865302] [INFO ] [MAIN ] CLW: Loaded 15618 existing records. [2025-09-16T17:48:49.042780] [WARN ] [MAIN ] CLW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:48:49.043005] [INFO ] [MAIN ] CLW: Wiping existing primary data before full refresh. [2025-09-16T17:48:49.043050] [INFO ] [LOCAL ] CLW: Starting deletion of primary data. [2025-09-16T17:48:49.043454] [DEBUG] [LOCAL ] CLW: Deleted primary data file. [2025-09-16T17:48:49.043499] [INFO ] [MAIN ] CLW: Starting full primary refresh. [2025-09-16T17:48:49.191486] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-02. [2025-09-16T17:48:49.402533] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T17:48:49.422087] [INFO ] [REMOTE ] CLW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:48:49.507671] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-09-16T17:48:49.974896] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-02. [2025-09-16T17:48:50.664618] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T17:48:50.774863] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-03. [2025-09-16T17:48:51.169383] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T17:48:51.574885] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-03. [2025-09-16T17:48:52.374937] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-09. [2025-09-16T17:48:52.603732] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T17:48:52.836340] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T17:48:53.091565] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-09-16T17:48:53.092963] [INFO ] [REMOTE ] CLPR: Full fetch complete (12528 records). [2025-09-16T17:48:53.093033] [DEBUG] [VALIDATOR ] CLPR: Validating primary data. [2025-09-16T17:48:53.093443] [FATAL] [VALIDATOR ] CLPR: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:48:53.093518] [FATAL] [MAIN ] CLPR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:48:53.093556] [WARN ] [MAIN ] CLPR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:48:53.093596] [INFO ] [LOCAL ] CLPR: Starting deletion of primary data. [2025-09-16T17:48:53.093715] [DEBUG] [LOCAL ] CLPR: No primary data file found to delete. [2025-09-16T17:48:53.093769] [INFO ] [LOCAL ] CLPR: Starting deletion of derivative data. [2025-09-16T17:48:53.094087] [DEBUG] [LOCAL ] CLPR: Deleted derivative data file. [2025-09-16T17:48:53.094167] [INFO ] [MAIN ] Update Progress: 466/2369 symbols completed. CLPR took 21.1s. Estimated remaining: 3h 19m 25s. Rate: 9.5 symbols/min [2025-09-16T17:48:53.180017] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-04. [2025-09-16T17:48:53.269270] [INFO ] [MAIN ] CLX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:48:53.311057] [DEBUG] [LOCAL ] CLX: Loaded 38387 existing records from primary storage. [2025-09-16T17:48:53.311196] [INFO ] [MAIN ] CLX: Loaded 38387 existing records. [2025-09-16T17:48:53.492320] [WARN ] [MAIN ] CLX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:48:53.492671] [INFO ] [MAIN ] CLX: Wiping existing primary data before full refresh. [2025-09-16T17:48:53.492720] [INFO ] [LOCAL ] CLX: Starting deletion of primary data. [2025-09-16T17:48:53.493616] [DEBUG] [LOCAL ] CLX: Deleted primary data file. [2025-09-16T17:48:53.493695] [INFO ] [MAIN ] CLX: Starting full primary refresh. [2025-09-16T17:48:53.864159] [INFO ] [REMOTE ] CLX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:48:53.974843] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-02. [2025-09-16T17:48:54.774881] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-04. [2025-09-16T17:48:55.113412] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-09-16T17:48:55.367263] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T17:48:55.574861] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-03. [2025-09-16T17:48:56.374918] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-04. [2025-09-16T17:48:57.046804] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-04 (2.27 seconds). [2025-09-16T17:48:57.174864] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-05. [2025-09-16T17:48:57.217196] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T17:48:57.520170] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T17:48:57.974957] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-04. [2025-09-16T17:48:58.774908] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-05. [2025-09-16T17:48:59.540425] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-05 (2.37 seconds). [2025-09-16T17:48:59.574895] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-06. [2025-09-16T17:48:59.699761] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T17:49:00.374929] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-05. [2025-09-16T17:49:00.568723] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T17:49:01.174916] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-05. [2025-09-16T17:49:01.802508] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-06 (2.23 seconds). [2025-09-16T17:49:01.974920] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-07. [2025-09-16T17:49:02.128523] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-09-16T17:49:02.360890] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-09-16T17:49:02.774881] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-06. [2025-09-16T17:49:03.574920] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-06. [2025-09-16T17:49:03.977602] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-09-16T17:49:04.374954] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-06. [2025-09-16T17:49:04.778812] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-09-16T17:49:05.076085] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-09-16T17:49:05.174901] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-08. [2025-09-16T17:49:05.266358] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T17:49:05.974913] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-07. [2025-09-16T17:49:06.774891] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-07. [2025-09-16T17:49:07.563362] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-08 (2.39 seconds). [2025-09-16T17:49:07.575129] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-09. [2025-09-16T17:49:07.828056] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T17:49:08.374899] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-07. [2025-09-16T17:49:08.461620] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T17:49:09.174905] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-08. [2025-09-16T17:49:09.368814] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-09-16T17:49:09.486287] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-09-16T17:49:09.492694] [INFO ] [REMOTE ] CLS: Full fetch complete (58322 records). [2025-09-16T17:49:09.492810] [DEBUG] [VALIDATOR ] CLS: Validating primary data. [2025-09-16T17:49:09.889688] [DEBUG] [VALIDATOR ] CLS: All expected trading days are present. [2025-09-16T17:49:09.889790] [DEBUG] [VALIDATOR ] CLS: Primary data validation passed. [2025-09-16T17:49:09.890596] [INFO ] [MAIN ] CLS: Fetched primary data is valid. [2025-09-16T17:49:09.890667] [DEBUG] [LOCAL ] CLS: Saving primary data. [2025-09-16T17:49:09.891982] [DEBUG] [LOCAL ] CLS: Trimmed 58322 records to 40005 (keeping 150 days of data) [2025-09-16T17:49:09.982281] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-08. [2025-09-16T17:49:10.143840] [DEBUG] [LOCAL ] CLS: Primary data saved (40005 records). [2025-09-16T17:49:10.143987] [INFO ] [MAIN ] CLS: Full history saved. [2025-09-16T17:49:10.144146] [INFO ] [MAIN ] Update Progress: 467/2369 symbols completed. CLS took 28.8s. Estimated remaining: 3h 20m 3s. Rate: 9.5 symbols/min [2025-09-16T17:49:10.319284] [INFO ] [MAIN ] CM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:49:10.319455] [DEBUG] [LOCAL ] CM: No existing data file found in primary storage. [2025-09-16T17:49:10.319496] [WARN ] [MAIN ] CM: No existing data found, performing full refresh instead. [2025-09-16T17:49:10.319620] [INFO ] [MAIN ] CM: Wiping existing primary data before full refresh. [2025-09-16T17:49:10.319681] [INFO ] [LOCAL ] CM: Starting deletion of primary data. [2025-09-16T17:49:10.319727] [DEBUG] [LOCAL ] CM: No primary data file found to delete. [2025-09-16T17:49:10.319761] [INFO ] [MAIN ] CM: Starting full primary refresh. [2025-09-16T17:49:10.677198] [INFO ] [REMOTE ] CM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:49:10.774857] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-02. [2025-09-16T17:49:10.898282] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-09-16T17:49:11.005950] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-09-16T17:49:11.574896] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-09. [2025-09-16T17:49:12.374870] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-08. [2025-09-16T17:49:12.462337] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T17:49:12.690506] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-09-16T17:49:12.693538] [INFO ] [REMOTE ] CLW: Full fetch complete (24645 records). [2025-09-16T17:49:12.693627] [DEBUG] [VALIDATOR ] CLW: Validating primary data. [2025-09-16T17:49:13.063823] [DEBUG] [VALIDATOR ] CLW: All expected trading days are present. [2025-09-16T17:49:13.063900] [DEBUG] [VALIDATOR ] CLW: Primary data validation passed. [2025-09-16T17:49:13.064349] [INFO ] [MAIN ] CLW: Fetched primary data is valid. [2025-09-16T17:49:13.064392] [DEBUG] [LOCAL ] CLW: Saving primary data. [2025-09-16T17:49:13.065095] [DEBUG] [LOCAL ] CLW: Trimmed 24645 records to 15781 (keeping 150 days of data) [2025-09-16T17:49:13.156829] [DEBUG] [LOCAL ] CLW: Primary data saved (15781 records). [2025-09-16T17:49:13.156978] [INFO ] [MAIN ] CLW: Full history saved. [2025-09-16T17:49:13.157064] [INFO ] [MAIN ] Update Progress: 468/2369 symbols completed. CLW took 24.5s. Estimated remaining: 3h 19m 43s. Rate: 9.5 symbols/min [2025-09-16T17:49:13.180008] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-09. [2025-09-16T17:49:13.339814] [INFO ] [MAIN ] CMA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:49:13.376780] [DEBUG] [LOCAL ] CMA: Loaded 36840 existing records from primary storage. [2025-09-16T17:49:13.376938] [INFO ] [MAIN ] CMA: Loaded 36840 existing records. [2025-09-16T17:49:13.559203] [WARN ] [MAIN ] CMA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:49:13.559415] [INFO ] [MAIN ] CMA: Wiping existing primary data before full refresh. [2025-09-16T17:49:13.559461] [INFO ] [LOCAL ] CMA: Starting deletion of primary data. [2025-09-16T17:49:13.560298] [DEBUG] [LOCAL ] CMA: Deleted primary data file. [2025-09-16T17:49:13.560360] [INFO ] [MAIN ] CMA: Starting full primary refresh. [2025-09-16T17:49:13.918383] [INFO ] [REMOTE ] CMA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:49:13.974861] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-02. [2025-09-16T17:49:14.245931] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-09-16T17:49:14.770531] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T17:49:14.776429] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-09. [2025-09-16T17:49:14.778273] [INFO ] [REMOTE ] CLX: Full fetch complete (55705 records). [2025-09-16T17:49:14.778446] [DEBUG] [VALIDATOR ] CLX: Validating primary data. [2025-09-16T17:49:15.174424] [DEBUG] [VALIDATOR ] CLX: All expected trading days are present. [2025-09-16T17:49:15.174525] [DEBUG] [VALIDATOR ] CLX: Primary data validation passed. [2025-09-16T17:49:15.175271] [INFO ] [MAIN ] CLX: Fetched primary data is valid. [2025-09-16T17:49:15.175345] [DEBUG] [LOCAL ] CLX: Saving primary data. [2025-09-16T17:49:15.176633] [DEBUG] [LOCAL ] CLX: Trimmed 55705 records to 38771 (keeping 150 days of data) [2025-09-16T17:49:15.424531] [DEBUG] [LOCAL ] CLX: Primary data saved (38771 records). [2025-09-16T17:49:15.424744] [INFO ] [MAIN ] CLX: Full history saved. [2025-09-16T17:49:15.426997] [INFO ] [MAIN ] Update Progress: 469/2369 symbols completed. CLX took 22.3s. Estimated remaining: 3h 19m 20s. Rate: 9.5 symbols/min [2025-09-16T17:49:15.582574] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-03. [2025-09-16T17:49:15.607860] [INFO ] [MAIN ] CMBT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:49:15.626438] [DEBUG] [LOCAL ] CMBT: Loaded 16946 existing records from primary storage. [2025-09-16T17:49:15.626588] [INFO ] [MAIN ] CMBT: Loaded 16946 existing records. [2025-09-16T17:49:15.813597] [WARN ] [MAIN ] CMBT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:49:15.813980] [INFO ] [MAIN ] CMBT: Wiping existing primary data before full refresh. [2025-09-16T17:49:15.814110] [INFO ] [LOCAL ] CMBT: Starting deletion of primary data. [2025-09-16T17:49:15.814578] [DEBUG] [LOCAL ] CMBT: Deleted primary data file. [2025-09-16T17:49:15.814627] [INFO ] [MAIN ] CMBT: Starting full primary refresh. [2025-09-16T17:49:16.082052] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-02 (2.11 seconds). [2025-09-16T17:49:16.241603] [INFO ] [REMOTE ] CMBT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:49:16.258814] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T17:49:16.265044] [INFO ] [REMOTE ] CLVT: Full fetch complete (56663 records). [2025-09-16T17:49:16.265180] [DEBUG] [VALIDATOR ] CLVT: Validating primary data. [2025-09-16T17:49:16.379964] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-02. [2025-09-16T17:49:16.642664] [DEBUG] [VALIDATOR ] CLVT: All expected trading days are present. [2025-09-16T17:49:16.642744] [DEBUG] [VALIDATOR ] CLVT: Primary data validation passed. [2025-09-16T17:49:16.643407] [INFO ] [MAIN ] CLVT: Fetched primary data is valid. [2025-09-16T17:49:16.643450] [DEBUG] [LOCAL ] CLVT: Saving primary data. [2025-09-16T17:49:16.644599] [DEBUG] [LOCAL ] CLVT: Trimmed 56663 records to 38762 (keeping 150 days of data) [2025-09-16T17:49:16.861926] [DEBUG] [LOCAL ] CLVT: Primary data saved (38762 records). [2025-09-16T17:49:16.862089] [INFO ] [MAIN ] CLVT: Full history saved. [2025-09-16T17:49:16.863145] [INFO ] [MAIN ] Update Progress: 470/2369 symbols completed. CLVT took 30.8s. Estimated remaining: 3h 18m 55s. Rate: 9.5 symbols/min [2025-09-16T17:49:17.036814] [INFO ] [MAIN ] CMC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:49:17.072102] [DEBUG] [LOCAL ] CMC: Loaded 32872 existing records from primary storage. [2025-09-16T17:49:17.072281] [INFO ] [MAIN ] CMC: Loaded 32872 existing records. [2025-09-16T17:49:17.175096] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-03. [2025-09-16T17:49:17.256342] [WARN ] [MAIN ] CMC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:49:17.256811] [INFO ] [MAIN ] CMC: Wiping existing primary data before full refresh. [2025-09-16T17:49:17.257030] [INFO ] [LOCAL ] CMC: Starting deletion of primary data. [2025-09-16T17:49:17.257853] [DEBUG] [LOCAL ] CMC: Deleted primary data file. [2025-09-16T17:49:17.257949] [INFO ] [MAIN ] CMC: Starting full primary refresh. [2025-09-16T17:49:17.412375] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T17:49:17.689522] [INFO ] [REMOTE ] CMC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:49:17.690845] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T17:49:17.974926] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-02. [2025-09-16T17:49:18.774865] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-03. [2025-09-16T17:49:18.941162] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T17:49:19.437285] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-09-16T17:49:19.574909] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-04. [2025-09-16T17:49:19.729791] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T17:49:20.374900] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-03. [2025-09-16T17:49:20.614364] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T17:49:21.174914] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-04. [2025-09-16T17:49:21.671035] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T17:49:21.974987] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-05. [2025-09-16T17:49:22.786415] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-04. [2025-09-16T17:49:22.796068] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T17:49:23.043112] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-09-16T17:49:23.574951] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-06. [2025-09-16T17:49:24.289015] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T17:49:24.352274] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T17:49:24.374896] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-07. [2025-09-16T17:49:25.134850] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-09-16T17:49:25.174950] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-04. [2025-09-16T17:49:25.974957] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-05. [2025-09-16T17:49:26.771515] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T17:49:26.781200] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-05. [2025-09-16T17:49:27.574943] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-08. [2025-09-16T17:49:27.809719] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-09-16T17:49:28.374922] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-05. [2025-09-16T17:49:28.421387] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T17:49:29.150024] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T17:49:29.174853] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-09. [2025-09-16T17:49:29.974879] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-06. [2025-09-16T17:49:30.022292] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T17:49:30.697964] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T17:49:30.704103] [INFO ] [REMOTE ] CMBT: Full fetch complete (22064 records). [2025-09-16T17:49:30.704214] [DEBUG] [VALIDATOR ] CMBT: Validating primary data. [2025-09-16T17:49:30.784170] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-06. [2025-09-16T17:49:31.082781] [DEBUG] [VALIDATOR ] CMBT: All expected trading days are present. [2025-09-16T17:49:31.082903] [DEBUG] [VALIDATOR ] CMBT: Primary data validation passed. [2025-09-16T17:49:31.083496] [INFO ] [MAIN ] CMBT: Fetched primary data is valid. [2025-09-16T17:49:31.083541] [DEBUG] [LOCAL ] CMBT: Saving primary data. [2025-09-16T17:49:31.084280] [DEBUG] [LOCAL ] CMBT: Trimmed 22064 records to 17313 (keeping 150 days of data) [2025-09-16T17:49:31.187500] [DEBUG] [LOCAL ] CMBT: Primary data saved (17313 records). [2025-09-16T17:49:31.187705] [INFO ] [MAIN ] CMBT: Full history saved. [2025-09-16T17:49:31.187829] [INFO ] [MAIN ] Update Progress: 471/2369 symbols completed. CMBT took 15.8s. Estimated remaining: 3h 19m 21s. Rate: 9.5 symbols/min [2025-09-16T17:49:31.366567] [INFO ] [MAIN ] CMCM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:49:31.371554] [DEBUG] [LOCAL ] CMCM: Loaded 3391 existing records from primary storage. [2025-09-16T17:49:31.371632] [INFO ] [MAIN ] CMCM: Loaded 3391 existing records. [2025-09-16T17:49:31.561522] [WARN ] [MAIN ] CMCM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:49:31.575836] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-06. [2025-09-16T17:49:31.580528] [INFO ] [MAIN ] CMCM: Wiping existing primary data before full refresh. [2025-09-16T17:49:31.584309] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-09-16T17:49:31.581152] [INFO ] [LOCAL ] CMCM: Starting deletion of primary data. [2025-09-16T17:49:31.596580] [DEBUG] [LOCAL ] CMCM: Deleted primary data file. [2025-09-16T17:49:31.596852] [INFO ] [MAIN ] CMCM: Starting full primary refresh. [2025-09-16T17:49:31.977967] [INFO ] [REMOTE ] CMCM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:49:32.374929] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-02. [2025-09-16T17:49:32.585208] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-09-16T17:49:32.912909] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-09-16T17:49:32.958136] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T17:49:33.174923] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-07. [2025-09-16T17:49:33.975002] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-07. [2025-09-16T17:49:34.674377] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T17:49:34.774909] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-08. [2025-09-16T17:49:35.351546] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-09-16T17:49:35.574902] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-08. [2025-09-16T17:49:36.169446] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-09-16T17:49:36.374910] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-07. [2025-09-16T17:49:36.993370] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T17:49:37.174880] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-09. [2025-09-16T17:49:37.974903] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-09. [2025-09-16T17:49:38.161934] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-09-16T17:49:38.576992] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:49:38.582981] [INFO ] [REMOTE ] CMA: Full fetch complete (54773 records). [2025-09-16T17:49:38.583103] [DEBUG] [VALIDATOR ] CMA: Validating primary data. [2025-09-16T17:49:38.783884] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-03. [2025-09-16T17:49:38.982028] [DEBUG] [VALIDATOR ] CMA: All expected trading days are present. [2025-09-16T17:49:38.982150] [DEBUG] [VALIDATOR ] CMA: Primary data validation passed. [2025-09-16T17:49:38.982845] [INFO ] [MAIN ] CMA: Fetched primary data is valid. [2025-09-16T17:49:38.982898] [DEBUG] [LOCAL ] CMA: Saving primary data. [2025-09-16T17:49:38.984270] [DEBUG] [LOCAL ] CMA: Trimmed 54773 records to 37171 (keeping 150 days of data) [2025-09-16T17:49:39.178937] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-09-16T17:49:39.215591] [INFO ] [REMOTE ] CMC: Full fetch complete (49757 records). [2025-09-16T17:49:39.219879] [DEBUG] [VALIDATOR ] CMC: Validating primary data. [2025-09-16T17:49:39.444817] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-09-16T17:49:39.580167] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-08. [2025-09-16T17:49:39.609383] [DEBUG] [LOCAL ] CMA: Primary data saved (37171 records). [2025-09-16T17:49:39.622080] [INFO ] [MAIN ] CMA: Full history saved. [2025-09-16T17:49:39.622165] [DEBUG] [VALIDATOR ] CMC: All expected trading days are present. [2025-09-16T17:49:39.622289] [INFO ] [MAIN ] Update Progress: 472/2369 symbols completed. CMA took 26.5s. Estimated remaining: 3h 19m 23s. Rate: 9.5 symbols/min [2025-09-16T17:49:39.622347] [DEBUG] [VALIDATOR ] CMC: Primary data validation passed. [2025-09-16T17:49:39.623186] [INFO ] [MAIN ] CMC: Fetched primary data is valid. [2025-09-16T17:49:39.653696] [DEBUG] [LOCAL ] CMC: Saving primary data. [2025-09-16T17:49:39.734866] [DEBUG] [LOCAL ] CMC: Trimmed 49757 records to 33219 (keeping 150 days of data) [2025-09-16T17:49:39.798456] [INFO ] [MAIN ] CMDB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:49:39.798665] [DEBUG] [LOCAL ] CMDB: No existing data file found in primary storage. [2025-09-16T17:49:39.798706] [WARN ] [MAIN ] CMDB: No existing data found, performing full refresh instead. [2025-09-16T17:49:39.851683] [INFO ] [MAIN ] CMDB: Wiping existing primary data before full refresh. [2025-09-16T17:49:39.860981] [INFO ] [LOCAL ] CMDB: Starting deletion of primary data. [2025-09-16T17:49:39.861415] [DEBUG] [LOCAL ] CMDB: No primary data file found to delete. [2025-09-16T17:49:39.887591] [INFO ] [MAIN ] CMDB: Starting full primary refresh. [2025-09-16T17:49:40.254160] [INFO ] [REMOTE ] CMDB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:49:40.350416] [DEBUG] [LOCAL ] CMC: Primary data saved (33219 records). [2025-09-16T17:49:40.350565] [INFO ] [MAIN ] CMC: Full history saved. [2025-09-16T17:49:40.351302] [INFO ] [MAIN ] Update Progress: 473/2369 symbols completed. CMC took 23.5s. Estimated remaining: 3h 18m 54s. Rate: 9.5 symbols/min [2025-09-16T17:49:40.382264] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-02. [2025-09-16T17:49:40.521284] [INFO ] [MAIN ] CMG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:49:40.561263] [DEBUG] [LOCAL ] CMG: Loaded 39687 existing records from primary storage. [2025-09-16T17:49:40.561424] [INFO ] [MAIN ] CMG: Loaded 39687 existing records. [2025-09-16T17:49:40.662231] [DEBUG] [REMOTE ] CMDB: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:49:40.662749] [WARN ] [REMOTE ] CMDB: No data returned for month 2025-02 [2025-09-16T17:49:40.738012] [WARN ] [MAIN ] CMG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:49:40.738201] [INFO ] [MAIN ] CMG: Wiping existing primary data before full refresh. [2025-09-16T17:49:40.738256] [INFO ] [LOCAL ] CMG: Starting deletion of primary data. [2025-09-16T17:49:40.739183] [DEBUG] [LOCAL ] CMG: Deleted primary data file. [2025-09-16T17:49:40.739233] [INFO ] [MAIN ] CMG: Starting full primary refresh. [2025-09-16T17:49:41.124142] [INFO ] [REMOTE ] CMG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:49:41.174843] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-02. [2025-09-16T17:49:41.229244] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T17:49:41.974885] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-03. [2025-09-16T17:49:42.167162] [DEBUG] [REMOTE ] CMDB: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:49:42.167409] [WARN ] [REMOTE ] CMDB: No data returned for month 2025-03 [2025-09-16T17:49:42.774915] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-04. [2025-09-16T17:49:43.227140] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-09-16T17:49:43.377866] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-09-16T17:49:43.574928] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-09. [2025-09-16T17:49:44.374945] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-05. [2025-09-16T17:49:44.801002] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-09-16T17:49:45.134870] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T17:49:45.140549] [INFO ] [REMOTE ] CM: Full fetch complete (52654 records). [2025-09-16T17:49:45.140699] [DEBUG] [VALIDATOR ] CM: Validating primary data. [2025-09-16T17:49:45.180015] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-03. [2025-09-16T17:49:45.516316] [DEBUG] [VALIDATOR ] CM: All expected trading days are present. [2025-09-16T17:49:45.516401] [DEBUG] [VALIDATOR ] CM: Primary data validation passed. [2025-09-16T17:49:45.517044] [INFO ] [MAIN ] CM: Fetched primary data is valid. [2025-09-16T17:49:45.517088] [DEBUG] [LOCAL ] CM: Saving primary data. [2025-09-16T17:49:45.518161] [DEBUG] [LOCAL ] CM: Trimmed 52654 records to 36130 (keeping 150 days of data) [2025-09-16T17:49:45.743097] [DEBUG] [LOCAL ] CM: Primary data saved (36130 records). [2025-09-16T17:49:45.743301] [INFO ] [MAIN ] CM: Full history saved. [2025-09-16T17:49:45.743404] [INFO ] [MAIN ] Update Progress: 474/2369 symbols completed. CM took 35.6s. Estimated remaining: 3h 18m 44s. Rate: 9.5 symbols/min [2025-09-16T17:49:45.912842] [INFO ] [MAIN ] CMI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:49:45.944904] [DEBUG] [LOCAL ] CMI: Loaded 30182 existing records from primary storage. [2025-09-16T17:49:45.945040] [INFO ] [MAIN ] CMI: Loaded 30182 existing records. [2025-09-16T17:49:45.980022] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-04. [2025-09-16T17:49:46.124944] [WARN ] [MAIN ] CMI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:49:46.125177] [INFO ] [MAIN ] CMI: Wiping existing primary data before full refresh. [2025-09-16T17:49:46.125232] [INFO ] [LOCAL ] CMI: Starting deletion of primary data. [2025-09-16T17:49:46.125883] [DEBUG] [LOCAL ] CMI: Deleted primary data file. [2025-09-16T17:49:46.125933] [INFO ] [MAIN ] CMI: Starting full primary refresh. [2025-09-16T17:49:46.275061] [DEBUG] [REMOTE ] CMDB: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:49:46.347181] [WARN ] [REMOTE ] CMDB: No data returned for month 2025-04 [2025-09-16T17:49:46.485790] [INFO ] [REMOTE ] CMI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:49:46.774876] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-05. [2025-09-16T17:49:47.239967] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-03 (2.06 seconds). [2025-09-16T17:49:47.574908] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-04. [2025-09-16T17:49:47.868246] [INFO ] [REMOTE ] CMDB: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-09-16T17:49:48.374941] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-02. [2025-09-16T17:49:49.174918] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-06. [2025-09-16T17:49:49.450113] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T17:49:49.540147] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-09-16T17:49:49.974870] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-05. [2025-09-16T17:49:50.059398] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T17:49:50.774918] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-07. [2025-09-16T17:49:51.400457] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-09-16T17:49:51.574863] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-03. [2025-09-16T17:49:51.998248] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-05 (2.02 seconds). [2025-09-16T17:49:52.374909] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-08. [2025-09-16T17:49:52.997874] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-09-16T17:49:53.174891] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-06. [2025-09-16T17:49:53.231735] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-09-16T17:49:53.974896] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-04. [2025-09-16T17:49:54.774910] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-09. [2025-09-16T17:49:55.432366] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-06 (2.26 seconds). [2025-09-16T17:49:55.514634] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-09-16T17:49:55.516491] [INFO ] [REMOTE ] CMCM: Full fetch complete (5307 records). [2025-09-16T17:49:55.516561] [DEBUG] [VALIDATOR ] CMCM: Validating primary data. [2025-09-16T17:49:55.580110] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-06. [2025-09-16T17:49:55.894341] [DEBUG] [VALIDATOR ] CMCM: All expected trading days are present. [2025-09-16T17:49:55.894470] [DEBUG] [VALIDATOR ] CMCM: Primary data validation passed. [2025-09-16T17:49:55.894729] [INFO ] [MAIN ] CMCM: Fetched primary data is valid. [2025-09-16T17:49:55.895004] [DEBUG] [LOCAL ] CMCM: Saving primary data. [2025-09-16T17:49:55.895626] [DEBUG] [LOCAL ] CMCM: Trimmed 5307 records to 3467 (keeping 150 days of data) [2025-09-16T17:49:55.941278] [DEBUG] [LOCAL ] CMCM: Primary data saved (3467 records). [2025-09-16T17:49:55.942234] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-09-16T17:49:55.942463] [INFO ] [MAIN ] CMCM: Full history saved. [2025-09-16T17:49:55.951456] [INFO ] [MAIN ] Update Progress: 475/2369 symbols completed. CMCM took 24.8s. Estimated remaining: 3h 18m 54s. Rate: 9.5 symbols/min [2025-09-16T17:49:56.124329] [INFO ] [MAIN ] CMP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:49:56.153179] [DEBUG] [LOCAL ] CMP: Loaded 27389 existing records from primary storage. [2025-09-16T17:49:56.153342] [INFO ] [MAIN ] CMP: Loaded 27389 existing records. [2025-09-16T17:49:56.329218] [WARN ] [MAIN ] CMP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:49:56.329460] [INFO ] [MAIN ] CMP: Wiping existing primary data before full refresh. [2025-09-16T17:49:56.329512] [INFO ] [LOCAL ] CMP: Starting deletion of primary data. [2025-09-16T17:49:56.330072] [DEBUG] [LOCAL ] CMP: Deleted primary data file. [2025-09-16T17:49:56.330131] [INFO ] [MAIN ] CMP: Starting full primary refresh. [2025-09-16T17:49:56.378534] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-05. [2025-09-16T17:49:56.538879] [INFO ] [REMOTE ] CMDB: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-09-16T17:49:56.720760] [INFO ] [REMOTE ] CMP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:49:57.174906] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-02. [2025-09-16T17:49:57.658148] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-09-16T17:49:57.974872] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-06. [2025-09-16T17:49:58.736797] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T17:49:58.774845] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-03. [2025-09-16T17:49:59.476857] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T17:49:59.574930] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-07. [2025-09-16T17:50:00.168421] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T17:50:00.374909] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-04. [2025-09-16T17:50:01.151310] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-09-16T17:50:01.174870] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-08. [2025-09-16T17:50:01.974957] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-07. [2025-09-16T17:50:02.169525] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T17:50:02.774926] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-05. [2025-09-16T17:50:03.273383] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-08 (2.10 seconds). [2025-09-16T17:50:03.574983] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-07. [2025-09-16T17:50:03.920772] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-09-16T17:50:04.341994] [INFO ] [REMOTE ] CMDB: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T17:50:04.369890] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T17:50:04.378384] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-08. [2025-09-16T17:50:05.174902] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-08. [2025-09-16T17:50:05.785304] [INFO ] [REMOTE ] CMDB: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-09-16T17:50:05.974887] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-06. [2025-09-16T17:50:06.774885] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-09. [2025-09-16T17:50:06.868177] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-08 (2.49 seconds). [2025-09-16T17:50:07.404795] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T17:50:07.459941] [INFO ] [REMOTE ] CMDB: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-09-16T17:50:07.462577] [INFO ] [REMOTE ] CMDB: Full fetch complete (10335 records). [2025-09-16T17:50:07.462723] [DEBUG] [VALIDATOR ] CMDB: Validating primary data. [2025-09-16T17:50:07.580105] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-09. [2025-09-16T17:50:07.667573] [WARN ] [VALIDATOR ] CMDB: Validation failed - insufficient historical data. Have 91 unique days, need 150 days [2025-09-16T17:50:07.667936] [FATAL] [MAIN ] CMDB: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:50:07.668032] [WARN ] [MAIN ] CMDB: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:50:07.668105] [INFO ] [LOCAL ] CMDB: Starting deletion of primary data. [2025-09-16T17:50:07.668190] [DEBUG] [LOCAL ] CMDB: No primary data file found to delete. [2025-09-16T17:50:07.668230] [INFO ] [LOCAL ] CMDB: Starting deletion of derivative data. [2025-09-16T17:50:07.668286] [DEBUG] [LOCAL ] CMDB: No derivative data file found to delete. [2025-09-16T17:50:07.668371] [INFO ] [MAIN ] Update Progress: 476/2369 symbols completed. CMDB took 28.0s. Estimated remaining: 3h 19m 9s. Rate: 9.5 symbols/min [2025-09-16T17:50:07.849666] [INFO ] [MAIN ] CMRE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:07.881409] [DEBUG] [LOCAL ] CMRE: Loaded 26054 existing records from primary storage. [2025-09-16T17:50:07.881617] [INFO ] [MAIN ] CMRE: Loaded 26054 existing records. [2025-09-16T17:50:08.070979] [WARN ] [MAIN ] CMRE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:50:08.071341] [INFO ] [MAIN ] CMRE: Wiping existing primary data before full refresh. [2025-09-16T17:50:08.071393] [INFO ] [LOCAL ] CMRE: Starting deletion of primary data. [2025-09-16T17:50:08.072044] [DEBUG] [LOCAL ] CMRE: Deleted primary data file. [2025-09-16T17:50:08.072099] [INFO ] [MAIN ] CMRE: Starting full primary refresh. [2025-09-16T17:50:08.380058] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-07. [2025-09-16T17:50:08.441853] [INFO ] [REMOTE ] CMRE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:50:09.174904] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-02. [2025-09-16T17:50:09.955382] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-09-16T17:50:09.974840] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-08. [2025-09-16T17:50:10.038494] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-09 (2.46 seconds). [2025-09-16T17:50:10.045350] [INFO ] [REMOTE ] CMG: Full fetch complete (58398 records). [2025-09-16T17:50:10.045462] [DEBUG] [VALIDATOR ] CMG: Validating primary data. [2025-09-16T17:50:10.443550] [DEBUG] [VALIDATOR ] CMG: All expected trading days are present. [2025-09-16T17:50:10.443674] [DEBUG] [VALIDATOR ] CMG: Primary data validation passed. [2025-09-16T17:50:10.444389] [INFO ] [MAIN ] CMG: Fetched primary data is valid. [2025-09-16T17:50:10.444446] [DEBUG] [LOCAL ] CMG: Saving primary data. [2025-09-16T17:50:10.463749] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T17:50:10.463885] [DEBUG] [LOCAL ] CMG: Trimmed 58398 records to 40077 (keeping 150 days of data) [2025-09-16T17:50:10.721783] [DEBUG] [LOCAL ] CMG: Primary data saved (40077 records). [2025-09-16T17:50:10.721931] [INFO ] [MAIN ] CMG: Full history saved. [2025-09-16T17:50:10.722546] [INFO ] [MAIN ] Update Progress: 477/2369 symbols completed. CMG took 30.4s. Estimated remaining: 3h 18m 50s. Rate: 9.5 symbols/min [2025-09-16T17:50:10.774936] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-03. [2025-09-16T17:50:11.104269] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T17:50:11.574966] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-09. [2025-09-16T17:50:12.246376] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-09-16T17:50:12.374902] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-04. [2025-09-16T17:50:13.174915] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-09. [2025-09-16T17:50:13.713052] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-09-16T17:50:13.974895] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-05. [2025-09-16T17:50:14.523870] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T17:50:14.529728] [INFO ] [REMOTE ] CMP: Full fetch complete (40315 records). [2025-09-16T17:50:14.529931] [DEBUG] [VALIDATOR ] CMP: Validating primary data. [2025-09-16T17:50:15.138356] [DEBUG] [VALIDATOR ] CMP: All expected trading days are present. [2025-09-16T17:50:15.138625] [DEBUG] [VALIDATOR ] CMP: Primary data validation passed. [2025-09-16T17:50:15.139993] [INFO ] [MAIN ] CMP: Fetched primary data is valid. [2025-09-16T17:50:15.140107] [DEBUG] [LOCAL ] CMP: Saving primary data. [2025-09-16T17:50:15.177206] [DEBUG] [LOCAL ] CMP: Trimmed 40315 records to 27600 (keeping 150 days of data) [2025-09-16T17:50:15.179715] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T17:50:15.191692] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-06. [2025-09-16T17:50:15.375465] [DEBUG] [LOCAL ] CMP: Primary data saved (27600 records). [2025-09-16T17:50:15.375660] [INFO ] [MAIN ] CMP: Full history saved. [2025-09-16T17:50:15.375753] [INFO ] [MAIN ] Update Progress: 478/2369 symbols completed. CMP took 19.4s. Estimated remaining: 3h 18m 37s. Rate: 9.5 symbols/min [2025-09-16T17:50:16.260627] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-09 (4.69 seconds). [2025-09-16T17:50:16.268244] [INFO ] [REMOTE ] CMI: Full fetch complete (45444 records). [2025-09-16T17:50:16.268360] [DEBUG] [VALIDATOR ] CMI: Validating primary data. [2025-09-16T17:50:16.628108] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T17:50:16.686296] [DEBUG] [VALIDATOR ] CMI: All expected trading days are present. [2025-09-16T17:50:16.686897] [DEBUG] [VALIDATOR ] CMI: Primary data validation passed. [2025-09-16T17:50:16.687495] [INFO ] [MAIN ] CMI: Fetched primary data is valid. [2025-09-16T17:50:16.687550] [DEBUG] [LOCAL ] CMI: Saving primary data. [2025-09-16T17:50:16.686817] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-07. [2025-09-16T17:50:16.689471] [DEBUG] [LOCAL ] CMI: Trimmed 45444 records to 30499 (keeping 150 days of data) [2025-09-16T17:50:16.889279] [DEBUG] [LOCAL ] CMI: Primary data saved (30499 records). [2025-09-16T17:50:16.889429] [INFO ] [MAIN ] CMI: Full history saved. [2025-09-16T17:50:16.889514] [INFO ] [MAIN ] Update Progress: 479/2369 symbols completed. CMI took 31.1s. Estimated remaining: 3h 18m 12s. Rate: 9.5 symbols/min [2025-09-16T17:50:18.049600] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-09-16T17:50:18.058201] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-08. [2025-09-16T17:50:19.760519] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-09-16T17:50:19.769179] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-09. [2025-09-16T17:50:21.148618] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T17:50:21.153356] [INFO ] [REMOTE ] CMRE: Full fetch complete (38782 records). [2025-09-16T17:50:21.153479] [DEBUG] [VALIDATOR ] CMRE: Validating primary data. [2025-09-16T17:50:21.549838] [DEBUG] [VALIDATOR ] CMRE: All expected trading days are present. [2025-09-16T17:50:21.549940] [DEBUG] [VALIDATOR ] CMRE: Primary data validation passed. [2025-09-16T17:50:21.550496] [INFO ] [MAIN ] CMRE: Fetched primary data is valid. [2025-09-16T17:50:21.550544] [DEBUG] [LOCAL ] CMRE: Saving primary data. [2025-09-16T17:50:21.551708] [DEBUG] [LOCAL ] CMRE: Trimmed 38782 records to 26276 (keeping 150 days of data) [2025-09-16T17:50:21.712118] [DEBUG] [LOCAL ] CMRE: Primary data saved (26276 records). [2025-09-16T17:50:21.712300] [INFO ] [MAIN ] CMRE: Full history saved. [2025-09-16T17:50:21.712387] [INFO ] [MAIN ] Update Progress: 480/2369 symbols completed. CMRE took 14.0s. Estimated remaining: 3h 17m 60s. Rate: 9.5 symbols/min [2025-09-16T17:50:22.346812] [INFO ] [MAIN ] CMSA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:22.360066] [INFO ] [MAIN ] CMS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:22.407187] [INFO ] [MAIN ] CMSC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:22.408758] [INFO ] [MAIN ] CMSD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:22.413086] [DEBUG] [LOCAL ] CMSA: Loaded 2500 existing records from primary storage. [2025-09-16T17:50:22.415569] [INFO ] [MAIN ] CMSA: Loaded 2500 existing records. [2025-09-16T17:50:22.453457] [DEBUG] [LOCAL ] CMSC: Loaded 3425 existing records from primary storage. [2025-09-16T17:50:22.470149] [DEBUG] [LOCAL ] CMSD: Loaded 5630 existing records from primary storage. [2025-09-16T17:50:22.478188] [INFO ] [MAIN ] CMSC: Loaded 3425 existing records. [2025-09-16T17:50:22.492626] [INFO ] [MAIN ] CMSD: Loaded 5630 existing records. [2025-09-16T17:50:22.850011] [WARN ] [MAIN ] CMSA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:50:22.890387] [DEBUG] [LOCAL ] CMS: Loaded 38924 existing records from primary storage. [2025-09-16T17:50:22.906446] [INFO ] [MAIN ] CMSA: Wiping existing primary data before full refresh. [2025-09-16T17:50:22.911583] [WARN ] [MAIN ] CMSD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:50:22.932390] [INFO ] [MAIN ] CMS: Loaded 38924 existing records. [2025-09-16T17:50:22.922043] [INFO ] [LOCAL ] CMSA: Starting deletion of primary data. [2025-09-16T17:50:22.962709] [INFO ] [MAIN ] CMSD: Wiping existing primary data before full refresh. [2025-09-16T17:50:22.989661] [DEBUG] [LOCAL ] CMSA: Deleted primary data file. [2025-09-16T17:50:23.017683] [INFO ] [LOCAL ] CMSD: Starting deletion of primary data. [2025-09-16T17:50:23.019149] [INFO ] [MAIN ] CMSA: Starting full primary refresh. [2025-09-16T17:50:23.028065] [WARN ] [MAIN ] CMSC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:50:23.088883] [DEBUG] [LOCAL ] CMSD: Deleted primary data file. [2025-09-16T17:50:23.121186] [INFO ] [MAIN ] CMSC: Wiping existing primary data before full refresh. [2025-09-16T17:50:23.145354] [INFO ] [MAIN ] CMSD: Starting full primary refresh. [2025-09-16T17:50:23.173135] [INFO ] [LOCAL ] CMSC: Starting deletion of primary data. [2025-09-16T17:50:23.237847] [DEBUG] [LOCAL ] CMSC: Deleted primary data file. [2025-09-16T17:50:23.272531] [INFO ] [MAIN ] CMSC: Starting full primary refresh. [2025-09-16T17:50:23.463053] [WARN ] [MAIN ] CMS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:50:23.602670] [INFO ] [MAIN ] CMS: Wiping existing primary data before full refresh. [2025-09-16T17:50:23.637346] [INFO ] [LOCAL ] CMS: Starting deletion of primary data. [2025-09-16T17:50:23.688061] [DEBUG] [LOCAL ] CMS: Deleted primary data file. [2025-09-16T17:50:23.811602] [INFO ] [MAIN ] CMS: Starting full primary refresh. [2025-09-16T17:50:24.122289] [INFO ] [REMOTE ] CMSA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:50:24.189406] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-02. [2025-09-16T17:50:24.279106] [INFO ] [REMOTE ] CMSD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:50:24.306776] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-02. [2025-09-16T17:50:24.367164] [INFO ] [REMOTE ] CMSC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:50:24.383783] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-02. [2025-09-16T17:50:24.611339] [INFO ] [REMOTE ] CMS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:50:24.611632] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-02. [2025-09-16T17:50:24.773201] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-09-16T17:50:24.774459] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-03. [2025-09-16T17:50:25.070905] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T17:50:25.072460] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-03. [2025-09-16T17:50:25.080393] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-09-16T17:50:25.082185] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-04. [2025-09-16T17:50:25.131853] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-09-16T17:50:25.133661] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-03. [2025-09-16T17:50:25.598929] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-09-16T17:50:25.600473] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-05. [2025-09-16T17:50:25.606294] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-09-16T17:50:25.608396] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-04. [2025-09-16T17:50:25.648380] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-09-16T17:50:25.930942] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-09-16T17:50:25.974860] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-06. [2025-09-16T17:50:26.033928] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-09-16T17:50:26.033713] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T17:50:26.306548] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-09-16T17:50:26.774908] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-04. [2025-09-16T17:50:27.478620] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-09-16T17:50:27.574920] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-03. [2025-09-16T17:50:28.374883] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-05. [2025-09-16T17:50:28.777408] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-09-16T17:50:29.128834] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T17:50:29.174877] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-04. [2025-09-16T17:50:29.974928] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-05. [2025-09-16T17:50:30.472966] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-09-16T17:50:30.774862] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-05. [2025-09-16T17:50:30.869200] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-09-16T17:50:31.574890] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-06. [2025-09-16T17:50:32.019325] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-09-16T17:50:32.374917] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-07. [2025-09-16T17:50:32.916549] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-09-16T17:50:33.174902] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-07. [2025-09-16T17:50:33.826999] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-05 (3.05 seconds). [2025-09-16T17:50:33.974934] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-08. [2025-09-16T17:50:34.340149] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-09-16T17:50:34.433210] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T17:50:34.774911] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-09. [2025-09-16T17:50:35.472559] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-09-16T17:50:35.473852] [INFO ] [REMOTE ] CMSA: Full fetch complete (3539 records). [2025-09-16T17:50:35.473939] [DEBUG] [VALIDATOR ] CMSA: Validating primary data. [2025-09-16T17:50:35.580067] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-06. [2025-09-16T17:50:35.863820] [DEBUG] [VALIDATOR ] CMSA: All expected trading days are present. [2025-09-16T17:50:35.863909] [DEBUG] [VALIDATOR ] CMSA: Primary data validation passed. [2025-09-16T17:50:35.864077] [INFO ] [MAIN ] CMSA: Fetched primary data is valid. [2025-09-16T17:50:35.864118] [DEBUG] [LOCAL ] CMSA: Saving primary data. [2025-09-16T17:50:35.864489] [DEBUG] [LOCAL ] CMSA: Trimmed 3539 records to 2523 (keeping 150 days of data) [2025-09-16T17:50:35.880352] [DEBUG] [LOCAL ] CMSA: Primary data saved (2523 records). [2025-09-16T17:50:35.880424] [INFO ] [MAIN ] CMSA: Full history saved. [2025-09-16T17:50:35.880496] [INFO ] [MAIN ] Update Progress: 481/2369 symbols completed. CMSA took 14.2s. Estimated remaining: 3h 18m 24s. Rate: 9.5 symbols/min [2025-09-16T17:50:36.058704] [INFO ] [MAIN ] CMTG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:36.083529] [DEBUG] [LOCAL ] CMTG: Loaded 26381 existing records from primary storage. [2025-09-16T17:50:36.083687] [INFO ] [MAIN ] CMTG: Loaded 26381 existing records. [2025-09-16T17:50:36.304200] [WARN ] [MAIN ] CMTG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:50:36.304437] [INFO ] [MAIN ] CMTG: Wiping existing primary data before full refresh. [2025-09-16T17:50:36.304485] [INFO ] [LOCAL ] CMTG: Starting deletion of primary data. [2025-09-16T17:50:36.305074] [DEBUG] [LOCAL ] CMTG: Deleted primary data file. [2025-09-16T17:50:36.305137] [INFO ] [MAIN ] CMTG: Starting full primary refresh. [2025-09-16T17:50:36.382379] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-06. [2025-09-16T17:50:36.678054] [INFO ] [REMOTE ] CMTG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:50:36.994698] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T17:50:37.174905] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-08. [2025-09-16T17:50:37.231010] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-09-16T17:50:37.662518] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-09-16T17:50:37.974902] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-07. [2025-09-16T17:50:38.774929] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-09. [2025-09-16T17:50:39.314250] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-09-16T17:50:39.315584] [INFO ] [REMOTE ] CMSD: Full fetch complete (8724 records). [2025-09-16T17:50:39.315679] [DEBUG] [VALIDATOR ] CMSD: Validating primary data. [2025-09-16T17:50:39.576879] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-02. [2025-09-16T17:50:39.710316] [DEBUG] [VALIDATOR ] CMSD: All expected trading days are present. [2025-09-16T17:50:39.710492] [DEBUG] [VALIDATOR ] CMSD: Primary data validation passed. [2025-09-16T17:50:39.710829] [INFO ] [MAIN ] CMSD: Fetched primary data is valid. [2025-09-16T17:50:39.712687] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T17:50:39.711092] [DEBUG] [LOCAL ] CMSD: Saving primary data. [2025-09-16T17:50:39.724015] [DEBUG] [LOCAL ] CMSD: Trimmed 8724 records to 5674 (keeping 150 days of data) [2025-09-16T17:50:39.758486] [DEBUG] [LOCAL ] CMSD: Primary data saved (5674 records). [2025-09-16T17:50:39.758573] [INFO ] [MAIN ] CMSD: Full history saved. [2025-09-16T17:50:39.758666] [INFO ] [MAIN ] Update Progress: 482/2369 symbols completed. CMSD took 18.0s. Estimated remaining: 3h 18m 8s. Rate: 9.5 symbols/min [2025-09-16T17:50:39.933104] [INFO ] [MAIN ] CMU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:39.938543] [DEBUG] [LOCAL ] CMU: Loaded 4387 existing records from primary storage. [2025-09-16T17:50:39.938664] [INFO ] [MAIN ] CMU: Loaded 4387 existing records. [2025-09-16T17:50:40.113749] [WARN ] [MAIN ] CMU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:50:40.114006] [INFO ] [MAIN ] CMU: Wiping existing primary data before full refresh. [2025-09-16T17:50:40.114051] [INFO ] [LOCAL ] CMU: Starting deletion of primary data. [2025-09-16T17:50:40.114299] [DEBUG] [LOCAL ] CMU: Deleted primary data file. [2025-09-16T17:50:40.114343] [INFO ] [MAIN ] CMU: Starting full primary refresh. [2025-09-16T17:50:40.380231] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-08. [2025-09-16T17:50:40.467095] [INFO ] [REMOTE ] CMU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:50:40.870573] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T17:50:41.174897] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-02. [2025-09-16T17:50:41.676280] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-09-16T17:50:41.678289] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-09-16T17:50:41.974944] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-09. [2025-09-16T17:50:42.774921] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-07. [2025-09-16T17:50:43.241118] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T17:50:43.247503] [INFO ] [REMOTE ] CMS: Full fetch complete (57265 records). [2025-09-16T17:50:43.247618] [DEBUG] [VALIDATOR ] CMS: Validating primary data. [2025-09-16T17:50:43.360341] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-09-16T17:50:43.588192] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-03. [2025-09-16T17:50:43.638748] [DEBUG] [VALIDATOR ] CMS: All expected trading days are present. [2025-09-16T17:50:43.638824] [DEBUG] [VALIDATOR ] CMS: Primary data validation passed. [2025-09-16T17:50:43.639540] [INFO ] [MAIN ] CMS: Fetched primary data is valid. [2025-09-16T17:50:43.639584] [DEBUG] [LOCAL ] CMS: Saving primary data. [2025-09-16T17:50:43.640811] [DEBUG] [LOCAL ] CMS: Trimmed 57265 records to 39283 (keeping 150 days of data) [2025-09-16T17:50:43.884717] [DEBUG] [LOCAL ] CMS: Primary data saved (39283 records). [2025-09-16T17:50:43.884864] [INFO ] [MAIN ] CMS: Full history saved. [2025-09-16T17:50:43.884951] [INFO ] [MAIN ] Update Progress: 483/2369 symbols completed. CMS took 22.2s. Estimated remaining: 3h 17m 54s. Rate: 9.5 symbols/min [2025-09-16T17:50:44.056997] [INFO ] [MAIN ] CNA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:44.079877] [DEBUG] [LOCAL ] CNA: Loaded 24485 existing records from primary storage. [2025-09-16T17:50:44.080027] [INFO ] [MAIN ] CNA: Loaded 24485 existing records. [2025-09-16T17:50:44.259910] [WARN ] [MAIN ] CNA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:50:44.260209] [INFO ] [MAIN ] CNA: Wiping existing primary data before full refresh. [2025-09-16T17:50:44.260274] [INFO ] [LOCAL ] CNA: Starting deletion of primary data. [2025-09-16T17:50:44.260892] [DEBUG] [LOCAL ] CNA: Deleted primary data file. [2025-09-16T17:50:44.260941] [INFO ] [MAIN ] CNA: Starting full primary refresh. [2025-09-16T17:50:44.379961] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-08. [2025-09-16T17:50:44.617768] [INFO ] [REMOTE ] CNA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:50:44.792944] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-09-16T17:50:44.939366] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T17:50:45.174928] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-04. [2025-09-16T17:50:45.974903] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-03. [2025-09-16T17:50:46.359909] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T17:50:46.774916] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-02. [2025-09-16T17:50:46.850150] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-09-16T17:50:47.574902] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-04. [2025-09-16T17:50:47.740902] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-09-16T17:50:48.121051] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T17:50:48.374970] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-05. [2025-09-16T17:50:49.174953] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-05. [2025-09-16T17:50:49.733629] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-09-16T17:50:49.776894] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T17:50:49.974927] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-06. [2025-09-16T17:50:50.774926] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-09. [2025-09-16T17:50:51.422813] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-09-16T17:50:51.423992] [INFO ] [REMOTE ] CMSC: Full fetch complete (5380 records). [2025-09-16T17:50:51.424142] [DEBUG] [VALIDATOR ] CMSC: Validating primary data. [2025-09-16T17:50:51.579722] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-06. [2025-09-16T17:50:51.824523] [DEBUG] [VALIDATOR ] CMSC: All expected trading days are present. [2025-09-16T17:50:51.825368] [DEBUG] [VALIDATOR ] CMSC: Primary data validation passed. [2025-09-16T17:50:51.825595] [INFO ] [MAIN ] CMSC: Fetched primary data is valid. [2025-09-16T17:50:51.825670] [DEBUG] [LOCAL ] CMSC: Saving primary data. [2025-09-16T17:50:51.827240] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-09-16T17:50:51.836310] [DEBUG] [LOCAL ] CMSC: Trimmed 5380 records to 3464 (keeping 150 days of data) [2025-09-16T17:50:51.858184] [DEBUG] [LOCAL ] CMSC: Primary data saved (3464 records). [2025-09-16T17:50:51.858272] [INFO ] [MAIN ] CMSC: Full history saved. [2025-09-16T17:50:51.858353] [INFO ] [MAIN ] Update Progress: 484/2369 symbols completed. CMSC took 30.1s. Estimated remaining: 3h 17m 54s. Rate: 9.5 symbols/min [2025-09-16T17:50:52.027132] [INFO ] [MAIN ] CNC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:52.070669] [DEBUG] [LOCAL ] CNC: Loaded 39748 existing records from primary storage. [2025-09-16T17:50:52.070816] [INFO ] [MAIN ] CNC: Loaded 39748 existing records. [2025-09-16T17:50:52.260303] [WARN ] [MAIN ] CNC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:50:52.260731] [INFO ] [MAIN ] CNC: Wiping existing primary data before full refresh. [2025-09-16T17:50:52.260797] [INFO ] [LOCAL ] CNC: Starting deletion of primary data. [2025-09-16T17:50:52.261795] [DEBUG] [LOCAL ] CNC: Deleted primary data file. [2025-09-16T17:50:52.261852] [INFO ] [MAIN ] CNC: Starting full primary refresh. [2025-09-16T17:50:52.370819] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-09-16T17:50:52.378166] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-03. [2025-09-16T17:50:52.640296] [INFO ] [REMOTE ] CNC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:50:53.174949] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-07. [2025-09-16T17:50:53.533239] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T17:50:53.742572] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-09-16T17:50:53.974972] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-02. [2025-09-16T17:50:54.774945] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-07. [2025-09-16T17:50:55.574970] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-08. [2025-09-16T17:50:56.084387] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-09-16T17:50:56.250169] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T17:50:56.374915] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-08. [2025-09-16T17:50:56.376423] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-02 (2.40 seconds). [2025-09-16T17:50:57.174904] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-04. [2025-09-16T17:50:57.456834] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T17:50:57.974930] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-09. [2025-09-16T17:50:58.636373] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T17:50:58.774901] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-03. [2025-09-16T17:50:59.063666] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T17:50:59.067978] [INFO ] [REMOTE ] CMTG: Full fetch complete (41440 records). [2025-09-16T17:50:59.068138] [DEBUG] [VALIDATOR ] CMTG: Validating primary data. [2025-09-16T17:50:59.446713] [DEBUG] [VALIDATOR ] CMTG: All expected trading days are present. [2025-09-16T17:50:59.446801] [DEBUG] [VALIDATOR ] CMTG: Primary data validation passed. [2025-09-16T17:50:59.447348] [INFO ] [MAIN ] CMTG: Fetched primary data is valid. [2025-09-16T17:50:59.447389] [DEBUG] [LOCAL ] CMTG: Saving primary data. [2025-09-16T17:50:59.448271] [DEBUG] [LOCAL ] CMTG: Trimmed 41440 records to 26595 (keeping 150 days of data) [2025-09-16T17:50:59.575528] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-05. [2025-09-16T17:50:59.601965] [DEBUG] [LOCAL ] CMTG: Primary data saved (26595 records). [2025-09-16T17:50:59.602115] [INFO ] [MAIN ] CMTG: Full history saved. [2025-09-16T17:50:59.602206] [INFO ] [MAIN ] Update Progress: 485/2369 symbols completed. CMTG took 23.7s. Estimated remaining: 3h 17m 53s. Rate: 9.5 symbols/min [2025-09-16T17:50:59.787242] [INFO ] [MAIN ] CNF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:50:59.796534] [DEBUG] [LOCAL ] CNF: Loaded 8697 existing records from primary storage. [2025-09-16T17:50:59.796632] [INFO ] [MAIN ] CNF: Loaded 8697 existing records. [2025-09-16T17:51:00.002888] [WARN ] [MAIN ] CNF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:51:00.003621] [INFO ] [MAIN ] CNF: Wiping existing primary data before full refresh. [2025-09-16T17:51:00.003706] [INFO ] [LOCAL ] CNF: Starting deletion of primary data. [2025-09-16T17:51:00.004302] [DEBUG] [LOCAL ] CNF: Deleted primary data file. [2025-09-16T17:51:00.004348] [INFO ] [MAIN ] CNF: Starting full primary refresh. [2025-09-16T17:51:00.352596] [INFO ] [REMOTE ] CNF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:51:00.374850] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-02. [2025-09-16T17:51:00.686132] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-09-16T17:51:01.175918] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-04. [2025-09-16T17:51:01.181259] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T17:51:01.217013] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T17:51:01.974922] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-09. [2025-09-16T17:51:02.632903] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-09-16T17:51:02.634110] [INFO ] [REMOTE ] CMU: Full fetch complete (7577 records). [2025-09-16T17:51:02.634181] [DEBUG] [VALIDATOR ] CMU: Validating primary data. [2025-09-16T17:51:02.782705] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-06. [2025-09-16T17:51:03.011061] [DEBUG] [VALIDATOR ] CMU: All expected trading days are present. [2025-09-16T17:51:03.011179] [DEBUG] [VALIDATOR ] CMU: Primary data validation passed. [2025-09-16T17:51:03.011379] [INFO ] [MAIN ] CMU: Fetched primary data is valid. [2025-09-16T17:51:03.011428] [DEBUG] [LOCAL ] CMU: Saving primary data. [2025-09-16T17:51:03.012072] [DEBUG] [LOCAL ] CMU: Trimmed 7577 records to 4419 (keeping 150 days of data) [2025-09-16T17:51:03.038279] [DEBUG] [LOCAL ] CMU: Primary data saved (4419 records). [2025-09-16T17:51:03.038379] [INFO ] [MAIN ] CMU: Full history saved. [2025-09-16T17:51:03.038458] [INFO ] [MAIN ] Update Progress: 486/2369 symbols completed. CMU took 23.3s. Estimated remaining: 3h 17m 36s. Rate: 9.5 symbols/min [2025-09-16T17:51:03.210013] [INFO ] [MAIN ] CNH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:51:03.247331] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-09-16T17:51:03.293509] [DEBUG] [LOCAL ] CNH: Loaded 39528 existing records from primary storage. [2025-09-16T17:51:03.293680] [INFO ] [MAIN ] CNH: Loaded 39528 existing records. [2025-09-16T17:51:03.474587] [WARN ] [MAIN ] CNH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:51:03.474879] [INFO ] [MAIN ] CNH: Wiping existing primary data before full refresh. [2025-09-16T17:51:03.474933] [INFO ] [LOCAL ] CNH: Starting deletion of primary data. [2025-09-16T17:51:03.475964] [DEBUG] [LOCAL ] CNH: Deleted primary data file. [2025-09-16T17:51:03.476014] [INFO ] [MAIN ] CNH: Starting full primary refresh. [2025-09-16T17:51:03.577174] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-03. [2025-09-16T17:51:03.829091] [INFO ] [REMOTE ] CNH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:51:04.082797] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:51:04.374903] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-02. [2025-09-16T17:51:04.425514] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-09-16T17:51:05.174897] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-05. [2025-09-16T17:51:05.975003] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-04. [2025-09-16T17:51:06.487799] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-02 (2.11 seconds). [2025-09-16T17:51:06.595602] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-09-16T17:51:06.774927] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-03. [2025-09-16T17:51:07.144453] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-09-16T17:51:07.574979] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-06. [2025-09-16T17:51:08.374916] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-05. [2025-09-16T17:51:08.509150] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T17:51:09.062547] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-09-16T17:51:09.174891] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-04. [2025-09-16T17:51:09.437802] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-09-16T17:51:09.974900] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-06. [2025-09-16T17:51:10.692819] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T17:51:10.712890] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-09-16T17:51:10.774895] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-07. [2025-09-16T17:51:11.574886] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-07. [2025-09-16T17:51:11.657717] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-09-16T17:51:12.374906] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-07. [2025-09-16T17:51:13.174904] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-08. [2025-09-16T17:51:13.207537] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-09-16T17:51:13.974871] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-08. [2025-09-16T17:51:14.151720] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-09-16T17:51:14.614020] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-07 (2.24 seconds). [2025-09-16T17:51:14.774885] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-09. [2025-09-16T17:51:15.518607] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T17:51:15.574858] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-09. [2025-09-16T17:51:15.663491] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T17:51:15.665724] [INFO ] [REMOTE ] CNF: Full fetch complete (10864 records). [2025-09-16T17:51:15.665797] [DEBUG] [VALIDATOR ] CNF: Validating primary data. [2025-09-16T17:51:16.046202] [DEBUG] [VALIDATOR ] CNF: All expected trading days are present. [2025-09-16T17:51:16.046284] [DEBUG] [VALIDATOR ] CNF: Primary data validation passed. [2025-09-16T17:51:16.046553] [INFO ] [MAIN ] CNF: Fetched primary data is valid. [2025-09-16T17:51:16.046599] [DEBUG] [LOCAL ] CNF: Saving primary data. [2025-09-16T17:51:16.047117] [DEBUG] [LOCAL ] CNF: Trimmed 10864 records to 8738 (keeping 150 days of data) [2025-09-16T17:51:16.094996] [DEBUG] [LOCAL ] CNF: Primary data saved (8738 records). [2025-09-16T17:51:16.095126] [INFO ] [MAIN ] CNF: Full history saved. [2025-09-16T17:51:16.095217] [INFO ] [MAIN ] Update Progress: 487/2369 symbols completed. CNF took 16.5s. Estimated remaining: 3h 17m 56s. Rate: 9.5 symbols/min [2025-09-16T17:51:16.265901] [INFO ] [MAIN ] CNI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:51:16.305636] [DEBUG] [LOCAL ] CNI: Loaded 38141 existing records from primary storage. [2025-09-16T17:51:16.305810] [INFO ] [MAIN ] CNI: Loaded 38141 existing records. [2025-09-16T17:51:16.380182] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-05. [2025-09-16T17:51:16.483704] [WARN ] [MAIN ] CNI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:51:16.483978] [INFO ] [MAIN ] CNI: Wiping existing primary data before full refresh. [2025-09-16T17:51:16.484024] [INFO ] [LOCAL ] CNI: Starting deletion of primary data. [2025-09-16T17:51:16.484926] [DEBUG] [LOCAL ] CNI: Deleted primary data file. [2025-09-16T17:51:16.484976] [INFO ] [MAIN ] CNI: Starting full primary refresh. [2025-09-16T17:51:16.871668] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T17:51:16.875707] [INFO ] [REMOTE ] CNI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:51:16.880381] [INFO ] [REMOTE ] CNA: Full fetch complete (35934 records). [2025-09-16T17:51:16.880499] [DEBUG] [VALIDATOR ] CNA: Validating primary data. [2025-09-16T17:51:17.180040] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-02. [2025-09-16T17:51:17.256432] [DEBUG] [VALIDATOR ] CNA: All expected trading days are present. [2025-09-16T17:51:17.256511] [DEBUG] [VALIDATOR ] CNA: Primary data validation passed. [2025-09-16T17:51:17.257020] [INFO ] [MAIN ] CNA: Fetched primary data is valid. [2025-09-16T17:51:17.257073] [DEBUG] [LOCAL ] CNA: Saving primary data. [2025-09-16T17:51:17.257962] [DEBUG] [LOCAL ] CNA: Trimmed 35934 records to 24774 (keeping 150 days of data) [2025-09-16T17:51:17.411827] [DEBUG] [LOCAL ] CNA: Primary data saved (24774 records). [2025-09-16T17:51:17.411977] [INFO ] [MAIN ] CNA: Full history saved. [2025-09-16T17:51:17.412071] [INFO ] [MAIN ] Update Progress: 488/2369 symbols completed. CNA took 33.5s. Estimated remaining: 3h 17m 30s. Rate: 9.5 symbols/min [2025-09-16T17:51:17.584165] [INFO ] [MAIN ] CNK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:51:17.621976] [DEBUG] [LOCAL ] CNK: Loaded 38995 existing records from primary storage. [2025-09-16T17:51:17.622137] [INFO ] [MAIN ] CNK: Loaded 38995 existing records. [2025-09-16T17:51:17.800777] [WARN ] [MAIN ] CNK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:51:17.801041] [INFO ] [MAIN ] CNK: Wiping existing primary data before full refresh. [2025-09-16T17:51:17.801100] [INFO ] [LOCAL ] CNK: Starting deletion of primary data. [2025-09-16T17:51:17.802084] [DEBUG] [LOCAL ] CNK: Deleted primary data file. [2025-09-16T17:51:17.802147] [INFO ] [MAIN ] CNK: Starting full primary refresh. [2025-09-16T17:51:18.008543] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-08. [2025-09-16T17:51:18.162354] [INFO ] [REMOTE ] CNK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:51:18.301744] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-09-16T17:51:18.780041] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-06. [2025-09-16T17:51:18.792924] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T17:51:19.574921] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-03. [2025-09-16T17:51:20.374916] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-02. [2025-09-16T17:51:20.518104] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T17:51:20.618017] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-08 (2.61 seconds). [2025-09-16T17:51:21.174900] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-09. [2025-09-16T17:51:21.350234] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T17:51:21.851136] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T17:51:21.974924] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-07. [2025-09-16T17:51:22.774936] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-04. [2025-09-16T17:51:22.994887] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-09-16T17:51:23.003867] [INFO ] [REMOTE ] CNC: Full fetch complete (58396 records). [2025-09-16T17:51:23.004021] [DEBUG] [VALIDATOR ] CNC: Validating primary data. [2025-09-16T17:51:23.393856] [DEBUG] [VALIDATOR ] CNC: All expected trading days are present. [2025-09-16T17:51:23.393954] [DEBUG] [VALIDATOR ] CNC: Primary data validation passed. [2025-09-16T17:51:23.394747] [INFO ] [MAIN ] CNC: Fetched primary data is valid. [2025-09-16T17:51:23.394802] [DEBUG] [LOCAL ] CNC: Saving primary data. [2025-09-16T17:51:23.396094] [DEBUG] [LOCAL ] CNC: Trimmed 58396 records to 40138 (keeping 150 days of data) [2025-09-16T17:51:23.581587] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-03. [2025-09-16T17:51:23.647729] [DEBUG] [LOCAL ] CNC: Primary data saved (40138 records). [2025-09-16T17:51:23.647869] [INFO ] [MAIN ] CNC: Full history saved. [2025-09-16T17:51:23.647983] [INFO ] [MAIN ] Update Progress: 489/2369 symbols completed. CNC took 31.8s. Estimated remaining: 3h 17m 23s. Rate: 9.5 symbols/min [2025-09-16T17:51:23.820821] [INFO ] [MAIN ] CNM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:51:23.856732] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T17:51:23.894150] [DEBUG] [LOCAL ] CNM: Loaded 37287 existing records from primary storage. [2025-09-16T17:51:23.894296] [INFO ] [MAIN ] CNM: Loaded 37287 existing records. [2025-09-16T17:51:24.098010] [WARN ] [MAIN ] CNM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:51:24.098258] [INFO ] [MAIN ] CNM: Wiping existing primary data before full refresh. [2025-09-16T17:51:24.098306] [INFO ] [LOCAL ] CNM: Starting deletion of primary data. [2025-09-16T17:51:24.099148] [DEBUG] [LOCAL ] CNM: Deleted primary data file. [2025-09-16T17:51:24.099199] [INFO ] [MAIN ] CNM: Starting full primary refresh. [2025-09-16T17:51:24.380752] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-08. [2025-09-16T17:51:24.459003] [INFO ] [REMOTE ] CNM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:51:24.532873] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T17:51:25.174891] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-05. [2025-09-16T17:51:25.241464] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-09-16T17:51:25.974878] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-04. [2025-09-16T17:51:26.043328] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T17:51:26.685460] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-09-16T17:51:26.774917] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-09. [2025-09-16T17:51:27.574886] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-06. [2025-09-16T17:51:27.789631] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T17:51:28.374916] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-05. [2025-09-16T17:51:29.064564] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-09 (2.29 seconds). [2025-09-16T17:51:29.071076] [INFO ] [REMOTE ] CNH: Full fetch complete (58240 records). [2025-09-16T17:51:29.071221] [DEBUG] [VALIDATOR ] CNH: Validating primary data. [2025-09-16T17:51:29.180014] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-02. [2025-09-16T17:51:29.453006] [DEBUG] [VALIDATOR ] CNH: All expected trading days are present. [2025-09-16T17:51:29.453081] [DEBUG] [VALIDATOR ] CNH: Primary data validation passed. [2025-09-16T17:51:29.453759] [INFO ] [MAIN ] CNH: Fetched primary data is valid. [2025-09-16T17:51:29.453831] [DEBUG] [LOCAL ] CNH: Saving primary data. [2025-09-16T17:51:29.455030] [DEBUG] [LOCAL ] CNH: Trimmed 58240 records to 39916 (keeping 150 days of data) [2025-09-16T17:51:29.696654] [DEBUG] [LOCAL ] CNH: Primary data saved (39916 records). [2025-09-16T17:51:29.696804] [INFO ] [MAIN ] CNH: Full history saved. [2025-09-16T17:51:29.696901] [INFO ] [MAIN ] Update Progress: 490/2369 symbols completed. CNH took 26.7s. Estimated remaining: 3h 17m 16s. Rate: 9.5 symbols/min [2025-09-16T17:51:29.871603] [INFO ] [MAIN ] CNMD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:51:29.904569] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-06 (2.33 seconds). [2025-09-16T17:51:29.929675] [DEBUG] [LOCAL ] CNMD: Loaded 23436 existing records from primary storage. [2025-09-16T17:51:29.929815] [INFO ] [MAIN ] CNMD: Loaded 23436 existing records. [2025-09-16T17:51:29.985791] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-07. [2025-09-16T17:51:30.113758] [WARN ] [MAIN ] CNMD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:51:30.114123] [INFO ] [MAIN ] CNMD: Wiping existing primary data before full refresh. [2025-09-16T17:51:30.114196] [INFO ] [LOCAL ] CNMD: Starting deletion of primary data. [2025-09-16T17:51:30.114964] [DEBUG] [LOCAL ] CNMD: Deleted primary data file. [2025-09-16T17:51:30.115028] [INFO ] [MAIN ] CNMD: Starting full primary refresh. [2025-09-16T17:51:30.302693] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-09-16T17:51:30.516693] [INFO ] [REMOTE ] CNMD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:51:30.568529] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T17:51:30.774920] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-06. [2025-09-16T17:51:31.574924] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-03. [2025-09-16T17:51:31.680051] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T17:51:32.374960] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-02. [2025-09-16T17:51:32.443737] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-09-16T17:51:33.136460] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T17:51:33.174926] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-04. [2025-09-16T17:51:33.432006] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T17:51:33.974903] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-03. [2025-09-16T17:51:34.774912] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-07. [2025-09-16T17:51:34.907503] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T17:51:35.341982] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T17:51:35.574898] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-04. [2025-09-16T17:51:36.374880] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-08. [2025-09-16T17:51:36.461193] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T17:51:36.935988] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-09-16T17:51:37.174948] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-05. [2025-09-16T17:51:37.715662] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T17:51:37.974902] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-08. [2025-09-16T17:51:38.535418] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-09-16T17:51:38.774918] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-09. [2025-09-16T17:51:39.356494] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T17:51:39.574922] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-06. [2025-09-16T17:51:40.080294] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T17:51:40.086471] [INFO ] [REMOTE ] CNI: Full fetch complete (56115 records). [2025-09-16T17:51:40.086566] [DEBUG] [VALIDATOR ] CNI: Validating primary data. [2025-09-16T17:51:40.379369] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-09. [2025-09-16T17:51:40.468337] [DEBUG] [VALIDATOR ] CNI: All expected trading days are present. [2025-09-16T17:51:40.468413] [DEBUG] [VALIDATOR ] CNI: Primary data validation passed. [2025-09-16T17:51:40.469081] [INFO ] [MAIN ] CNI: Fetched primary data is valid. [2025-09-16T17:51:40.469127] [DEBUG] [LOCAL ] CNI: Saving primary data. [2025-09-16T17:51:40.470253] [DEBUG] [LOCAL ] CNI: Trimmed 56115 records to 38509 (keeping 150 days of data) [2025-09-16T17:51:40.711624] [DEBUG] [LOCAL ] CNI: Primary data saved (38509 records). [2025-09-16T17:51:40.711771] [INFO ] [MAIN ] CNI: Full history saved. [2025-09-16T17:51:40.713502] [INFO ] [MAIN ] Update Progress: 491/2369 symbols completed. CNI took 24.6s. Estimated remaining: 3h 17m 28s. Rate: 9.5 symbols/min [2025-09-16T17:51:40.854462] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T17:51:40.909954] [INFO ] [MAIN ] CNNE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:51:40.937719] [DEBUG] [LOCAL ] CNNE: Loaded 26310 existing records from primary storage. [2025-09-16T17:51:40.937850] [INFO ] [MAIN ] CNNE: Loaded 26310 existing records. [2025-09-16T17:51:41.118300] [WARN ] [MAIN ] CNNE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:51:41.118587] [INFO ] [MAIN ] CNNE: Wiping existing primary data before full refresh. [2025-09-16T17:51:41.118637] [INFO ] [LOCAL ] CNNE: Starting deletion of primary data. [2025-09-16T17:51:41.119341] [DEBUG] [LOCAL ] CNNE: Deleted primary data file. [2025-09-16T17:51:41.119400] [INFO ] [MAIN ] CNNE: Starting full primary refresh. [2025-09-16T17:51:41.183591] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-07. [2025-09-16T17:51:41.476717] [INFO ] [REMOTE ] CNNE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:51:41.974917] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-02. [2025-09-16T17:51:41.994916] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T17:51:42.001790] [INFO ] [REMOTE ] CNK: Full fetch complete (57472 records). [2025-09-16T17:51:42.001991] [DEBUG] [VALIDATOR ] CNK: Validating primary data. [2025-09-16T17:51:42.401484] [DEBUG] [VALIDATOR ] CNK: All expected trading days are present. [2025-09-16T17:51:42.401665] [DEBUG] [VALIDATOR ] CNK: Primary data validation passed. [2025-09-16T17:51:42.402412] [INFO ] [MAIN ] CNK: Fetched primary data is valid. [2025-09-16T17:51:42.402501] [DEBUG] [LOCAL ] CNK: Saving primary data. [2025-09-16T17:51:42.405303] [DEBUG] [LOCAL ] CNK: Trimmed 57472 records to 39385 (keeping 150 days of data) [2025-09-16T17:51:42.517837] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-09-16T17:51:42.676840] [DEBUG] [LOCAL ] CNK: Primary data saved (39385 records). [2025-09-16T17:51:42.676991] [INFO ] [MAIN ] CNK: Full history saved. [2025-09-16T17:51:42.677713] [INFO ] [MAIN ] Update Progress: 492/2369 symbols completed. CNK took 25.3s. Estimated remaining: 3h 17m 5s. Rate: 9.5 symbols/min [2025-09-16T17:51:42.779026] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-05. [2025-09-16T17:51:42.849478] [INFO ] [MAIN ] CNO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:51:42.881742] [DEBUG] [LOCAL ] CNO: Loaded 29693 existing records from primary storage. [2025-09-16T17:51:42.881877] [INFO ] [MAIN ] CNO: Loaded 29693 existing records. [2025-09-16T17:51:43.061058] [WARN ] [MAIN ] CNO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:51:43.061419] [INFO ] [MAIN ] CNO: Wiping existing primary data before full refresh. [2025-09-16T17:51:43.061512] [INFO ] [LOCAL ] CNO: Starting deletion of primary data. [2025-09-16T17:51:43.067911] [DEBUG] [LOCAL ] CNO: Deleted primary data file. [2025-09-16T17:51:43.077489] [INFO ] [MAIN ] CNO: Starting full primary refresh. [2025-09-16T17:51:43.078027] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T17:51:43.441035] [INFO ] [REMOTE ] CNO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:51:43.574884] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-03. [2025-09-16T17:51:44.088365] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T17:51:44.374946] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-02. [2025-09-16T17:51:44.806252] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-09-16T17:51:45.174933] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-04. [2025-09-16T17:51:45.447795] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-09-16T17:51:45.974925] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-03. [2025-09-16T17:51:46.572487] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T17:51:46.774917] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-05. [2025-09-16T17:51:47.241275] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T17:51:47.574929] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-08. [2025-09-16T17:51:47.747610] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T17:51:48.374906] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-06. [2025-09-16T17:51:48.670138] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-09-16T17:51:49.174935] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-09. [2025-09-16T17:51:49.780315] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T17:51:49.974898] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-07. [2025-09-16T17:51:50.574553] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:51:50.578279] [INFO ] [REMOTE ] CNMD: Full fetch complete (35745 records). [2025-09-16T17:51:50.578431] [DEBUG] [VALIDATOR ] CNMD: Validating primary data. [2025-09-16T17:51:50.784084] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-06. [2025-09-16T17:51:50.981796] [DEBUG] [VALIDATOR ] CNMD: All expected trading days are present. [2025-09-16T17:51:50.981939] [DEBUG] [VALIDATOR ] CNMD: Primary data validation passed. [2025-09-16T17:51:50.982703] [INFO ] [MAIN ] CNMD: Fetched primary data is valid. [2025-09-16T17:51:50.982770] [DEBUG] [LOCAL ] CNMD: Saving primary data. [2025-09-16T17:51:50.984169] [DEBUG] [LOCAL ] CNMD: Trimmed 35745 records to 23618 (keeping 150 days of data) [2025-09-16T17:51:51.142528] [DEBUG] [LOCAL ] CNMD: Primary data saved (23618 records). [2025-09-16T17:51:51.142679] [INFO ] [MAIN ] CNMD: Full history saved. [2025-09-16T17:51:51.142790] [INFO ] [MAIN ] Update Progress: 493/2369 symbols completed. CNMD took 21.4s. Estimated remaining: 3h 17m 7s. Rate: 9.5 symbols/min [2025-09-16T17:51:51.313852] [INFO ] [MAIN ] CNP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:51:51.350414] [DEBUG] [LOCAL ] CNP: Loaded 39456 existing records from primary storage. [2025-09-16T17:51:51.350570] [INFO ] [MAIN ] CNP: Loaded 39456 existing records. [2025-09-16T17:51:51.481166] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T17:51:51.568776] [WARN ] [MAIN ] CNP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:51:51.569028] [INFO ] [MAIN ] CNP: Wiping existing primary data before full refresh. [2025-09-16T17:51:51.569085] [INFO ] [LOCAL ] CNP: Starting deletion of primary data. [2025-09-16T17:51:51.569953] [DEBUG] [LOCAL ] CNP: Deleted primary data file. [2025-09-16T17:51:51.570002] [INFO ] [MAIN ] CNP: Starting full primary refresh. [2025-09-16T17:51:51.581016] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-04. [2025-09-16T17:51:51.933012] [INFO ] [REMOTE ] CNP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:51:51.956764] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-09-16T17:51:52.374921] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-02. [2025-09-16T17:51:53.039570] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T17:51:53.174884] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-05. [2025-09-16T17:51:53.686151] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T17:51:53.974917] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-03. [2025-09-16T17:51:54.234061] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-09-16T17:51:54.774907] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-06. [2025-09-16T17:51:55.398428] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T17:51:55.574934] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-07. [2025-09-16T17:51:55.878571] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-09-16T17:51:56.374903] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-07. [2025-09-16T17:51:56.712134] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T17:51:57.174901] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-08. [2025-09-16T17:51:57.491440] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T17:51:57.974904] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-04. [2025-09-16T17:51:58.758505] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T17:51:58.774853] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-09. [2025-09-16T17:51:59.574861] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-08. [2025-09-16T17:51:59.678061] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-09-16T17:52:00.245413] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T17:52:00.251694] [INFO ] [REMOTE ] CNM: Full fetch complete (55118 records). [2025-09-16T17:52:00.251841] [DEBUG] [VALIDATOR ] CNM: Validating primary data. [2025-09-16T17:52:00.379983] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-05. [2025-09-16T17:52:00.653937] [DEBUG] [VALIDATOR ] CNM: All expected trading days are present. [2025-09-16T17:52:00.654022] [DEBUG] [VALIDATOR ] CNM: Primary data validation passed. [2025-09-16T17:52:00.654692] [INFO ] [MAIN ] CNM: Fetched primary data is valid. [2025-09-16T17:52:00.654763] [DEBUG] [LOCAL ] CNM: Saving primary data. [2025-09-16T17:52:00.655950] [DEBUG] [LOCAL ] CNM: Trimmed 55118 records to 37677 (keeping 150 days of data) [2025-09-16T17:52:00.881527] [DEBUG] [LOCAL ] CNM: Primary data saved (37677 records). [2025-09-16T17:52:00.881685] [INFO ] [MAIN ] CNM: Full history saved. [2025-09-16T17:52:00.881797] [INFO ] [MAIN ] Update Progress: 494/2369 symbols completed. CNM took 37.2s. Estimated remaining: 3h 17m 14s. Rate: 9.5 symbols/min [2025-09-16T17:52:01.055289] [INFO ] [MAIN ] CNQ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:01.095134] [DEBUG] [LOCAL ] CNQ: Loaded 39551 existing records from primary storage. [2025-09-16T17:52:01.095303] [INFO ] [MAIN ] CNQ: Loaded 39551 existing records. [2025-09-16T17:52:01.186039] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-08. [2025-09-16T17:52:01.274166] [WARN ] [MAIN ] CNQ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:01.274380] [INFO ] [MAIN ] CNQ: Wiping existing primary data before full refresh. [2025-09-16T17:52:01.274426] [INFO ] [LOCAL ] CNQ: Starting deletion of primary data. [2025-09-16T17:52:01.275212] [DEBUG] [LOCAL ] CNQ: Deleted primary data file. [2025-09-16T17:52:01.275271] [INFO ] [MAIN ] CNQ: Starting full primary refresh. [2025-09-16T17:52:01.632165] [INFO ] [REMOTE ] CNQ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:01.690364] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-09-16T17:52:01.974909] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-02. [2025-09-16T17:52:02.422516] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T17:52:02.658412] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-05 (2.28 seconds). [2025-09-16T17:52:02.774889] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-06. [2025-09-16T17:52:03.574870] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-09. [2025-09-16T17:52:03.612461] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T17:52:04.339784] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T17:52:04.374856] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-07. [2025-09-16T17:52:04.944087] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T17:52:04.948908] [INFO ] [REMOTE ] CNNE: Full fetch complete (39808 records). [2025-09-16T17:52:04.949011] [DEBUG] [VALIDATOR ] CNNE: Validating primary data. [2025-09-16T17:52:05.180086] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-09. [2025-09-16T17:52:05.322179] [DEBUG] [VALIDATOR ] CNNE: All expected trading days are present. [2025-09-16T17:52:05.322275] [DEBUG] [VALIDATOR ] CNNE: Primary data validation passed. [2025-09-16T17:52:05.322962] [INFO ] [MAIN ] CNNE: Fetched primary data is valid. [2025-09-16T17:52:05.323019] [DEBUG] [LOCAL ] CNNE: Saving primary data. [2025-09-16T17:52:05.324118] [DEBUG] [LOCAL ] CNNE: Trimmed 39808 records to 26562 (keeping 150 days of data) [2025-09-16T17:52:05.493515] [DEBUG] [LOCAL ] CNNE: Primary data saved (26562 records). [2025-09-16T17:52:05.493707] [INFO ] [MAIN ] CNNE: Full history saved. [2025-09-16T17:52:05.493849] [INFO ] [MAIN ] Update Progress: 495/2369 symbols completed. CNNE took 24.8s. Estimated remaining: 3h 17m 1s. Rate: 9.5 symbols/min [2025-09-16T17:52:05.673711] [INFO ] [MAIN ] CNR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:05.707504] [DEBUG] [LOCAL ] CNR: Loaded 32097 existing records from primary storage. [2025-09-16T17:52:05.707669] [INFO ] [MAIN ] CNR: Loaded 32097 existing records. [2025-09-16T17:52:05.896818] [WARN ] [MAIN ] CNR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:05.897056] [INFO ] [MAIN ] CNR: Wiping existing primary data before full refresh. [2025-09-16T17:52:05.897116] [INFO ] [LOCAL ] CNR: Starting deletion of primary data. [2025-09-16T17:52:05.897835] [DEBUG] [LOCAL ] CNR: Deleted primary data file. [2025-09-16T17:52:05.897889] [INFO ] [MAIN ] CNR: Starting full primary refresh. [2025-09-16T17:52:05.984262] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-03. [2025-09-16T17:52:06.261802] [INFO ] [REMOTE ] CNR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:06.296463] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T17:52:06.534908] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T17:52:06.541000] [INFO ] [REMOTE ] CNO: Full fetch complete (45738 records). [2025-09-16T17:52:06.541096] [DEBUG] [VALIDATOR ] CNO: Validating primary data. [2025-09-16T17:52:06.782396] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-02. [2025-09-16T17:52:06.922161] [DEBUG] [VALIDATOR ] CNO: All expected trading days are present. [2025-09-16T17:52:06.922265] [DEBUG] [VALIDATOR ] CNO: Primary data validation passed. [2025-09-16T17:52:06.923024] [INFO ] [MAIN ] CNO: Fetched primary data is valid. [2025-09-16T17:52:06.923073] [DEBUG] [LOCAL ] CNO: Saving primary data. [2025-09-16T17:52:06.924304] [DEBUG] [LOCAL ] CNO: Trimmed 45738 records to 29999 (keeping 150 days of data) [2025-09-16T17:52:07.126495] [DEBUG] [LOCAL ] CNO: Primary data saved (29999 records). [2025-09-16T17:52:07.126671] [INFO ] [MAIN ] CNO: Full history saved. [2025-09-16T17:52:07.127357] [INFO ] [MAIN ] Update Progress: 496/2369 symbols completed. CNO took 24.4s. Estimated remaining: 3h 16m 37s. Rate: 9.5 symbols/min [2025-09-16T17:52:07.304716] [INFO ] [MAIN ] CNS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:07.324550] [DEBUG] [LOCAL ] CNS: Loaded 14550 existing records from primary storage. [2025-09-16T17:52:07.324715] [INFO ] [MAIN ] CNS: Loaded 14550 existing records. [2025-09-16T17:52:07.510170] [WARN ] [MAIN ] CNS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:07.510501] [INFO ] [MAIN ] CNS: Wiping existing primary data before full refresh. [2025-09-16T17:52:07.510553] [INFO ] [LOCAL ] CNS: Starting deletion of primary data. [2025-09-16T17:52:07.510993] [DEBUG] [LOCAL ] CNS: Deleted primary data file. [2025-09-16T17:52:07.511040] [INFO ] [MAIN ] CNS: Starting full primary refresh. [2025-09-16T17:52:07.581719] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-08. [2025-09-16T17:52:07.879975] [INFO ] [REMOTE ] CNS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:07.926549] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-09-16T17:52:08.374909] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-02. [2025-09-16T17:52:08.410304] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T17:52:09.174873] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-03. [2025-09-16T17:52:09.428593] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-09-16T17:52:09.533080] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-09-16T17:52:09.974906] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-04. [2025-09-16T17:52:10.665509] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-09-16T17:52:10.774881] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-04. [2025-09-16T17:52:11.574938] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-09. [2025-09-16T17:52:11.839308] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-09-16T17:52:12.227886] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T17:52:12.374897] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-05. [2025-09-16T17:52:13.161419] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T17:52:13.167793] [INFO ] [REMOTE ] CNP: Full fetch complete (58127 records). [2025-09-16T17:52:13.167912] [DEBUG] [VALIDATOR ] CNP: Validating primary data. [2025-09-16T17:52:13.179983] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-03. [2025-09-16T17:52:13.546733] [DEBUG] [VALIDATOR ] CNP: All expected trading days are present. [2025-09-16T17:52:13.546813] [DEBUG] [VALIDATOR ] CNP: Primary data validation passed. [2025-09-16T17:52:13.547471] [INFO ] [MAIN ] CNP: Fetched primary data is valid. [2025-09-16T17:52:13.547513] [DEBUG] [LOCAL ] CNP: Saving primary data. [2025-09-16T17:52:13.548704] [DEBUG] [LOCAL ] CNP: Trimmed 58127 records to 39844 (keeping 150 days of data) [2025-09-16T17:52:13.798537] [DEBUG] [LOCAL ] CNP: Primary data saved (39844 records). [2025-09-16T17:52:13.798716] [INFO ] [MAIN ] CNP: Full history saved. [2025-09-16T17:52:13.798804] [INFO ] [MAIN ] Update Progress: 497/2369 symbols completed. CNP took 22.7s. Estimated remaining: 3h 16m 32s. Rate: 9.5 symbols/min [2025-09-16T17:52:13.968743] [INFO ] [MAIN ] CNX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:13.974911] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-05. [2025-09-16T17:52:14.006362] [DEBUG] [LOCAL ] CNX: Loaded 37796 existing records from primary storage. [2025-09-16T17:52:14.006511] [INFO ] [MAIN ] CNX: Loaded 37796 existing records. [2025-09-16T17:52:14.186559] [WARN ] [MAIN ] CNX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:14.187196] [INFO ] [MAIN ] CNX: Wiping existing primary data before full refresh. [2025-09-16T17:52:14.187313] [INFO ] [LOCAL ] CNX: Starting deletion of primary data. [2025-09-16T17:52:14.188205] [DEBUG] [LOCAL ] CNX: Deleted primary data file. [2025-09-16T17:52:14.188267] [INFO ] [MAIN ] CNX: Starting full primary refresh. [2025-09-16T17:52:14.405561] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-09-16T17:52:14.614475] [INFO ] [REMOTE ] CNX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:14.630625] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T17:52:14.774903] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-02. [2025-09-16T17:52:15.383837] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-09-16T17:52:15.574898] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-06. [2025-09-16T17:52:16.374893] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-04. [2025-09-16T17:52:16.494312] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-09-16T17:52:16.918620] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-09-16T17:52:17.174901] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-03. [2025-09-16T17:52:17.522667] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T17:52:17.974881] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-05. [2025-09-16T17:52:18.766019] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-09-16T17:52:18.779521] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-07. [2025-09-16T17:52:18.947690] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T17:52:19.574978] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-04. [2025-09-16T17:52:20.343978] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-09-16T17:52:20.374890] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-08. [2025-09-16T17:52:20.769204] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T17:52:21.174915] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-06. [2025-09-16T17:52:21.605849] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-09-16T17:52:21.949692] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T17:52:21.974869] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-07. [2025-09-16T17:52:22.774930] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-05. [2025-09-16T17:52:22.903192] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-09-16T17:52:23.574932] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-08. [2025-09-16T17:52:24.103937] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T17:52:24.374915] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-06. [2025-09-16T17:52:24.615046] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-09-16T17:52:25.174925] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-06. [2025-09-16T17:52:25.974910] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-09. [2025-09-16T17:52:26.077103] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T17:52:26.774912] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-07. [2025-09-16T17:52:27.004186] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T17:52:27.440834] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T17:52:27.446345] [INFO ] [REMOTE ] CNR: Full fetch complete (48399 records). [2025-09-16T17:52:27.446517] [DEBUG] [VALIDATOR ] CNR: Validating primary data. [2025-09-16T17:52:27.588500] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-09. [2025-09-16T17:52:27.830974] [DEBUG] [VALIDATOR ] CNR: All expected trading days are present. [2025-09-16T17:52:27.831054] [DEBUG] [VALIDATOR ] CNR: Primary data validation passed. [2025-09-16T17:52:27.831657] [INFO ] [MAIN ] CNR: Fetched primary data is valid. [2025-09-16T17:52:27.831703] [DEBUG] [LOCAL ] CNR: Saving primary data. [2025-09-16T17:52:27.832679] [DEBUG] [LOCAL ] CNR: Trimmed 48399 records to 32395 (keeping 150 days of data) [2025-09-16T17:52:28.057986] [DEBUG] [LOCAL ] CNR: Primary data saved (32395 records). [2025-09-16T17:52:28.058146] [INFO ] [MAIN ] CNR: Full history saved. [2025-09-16T17:52:28.060259] [INFO ] [MAIN ] Update Progress: 498/2369 symbols completed. CNR took 22.6s. Estimated remaining: 3h 16m 56s. Rate: 9.5 symbols/min [2025-09-16T17:52:28.234679] [INFO ] [MAIN ] CODI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:28.266071] [DEBUG] [LOCAL ] CODI: Loaded 31124 existing records from primary storage. [2025-09-16T17:52:28.266268] [INFO ] [MAIN ] CODI: Loaded 31124 existing records. [2025-09-16T17:52:28.378381] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-07. [2025-09-16T17:52:28.446901] [WARN ] [MAIN ] CODI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:28.447264] [INFO ] [MAIN ] CODI: Wiping existing primary data before full refresh. [2025-09-16T17:52:28.447370] [INFO ] [LOCAL ] CODI: Starting deletion of primary data. [2025-09-16T17:52:28.448039] [DEBUG] [LOCAL ] CODI: Deleted primary data file. [2025-09-16T17:52:28.448090] [INFO ] [MAIN ] CODI: Starting full primary refresh. [2025-09-16T17:52:28.771178] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-09-16T17:52:28.847379] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T17:52:28.867906] [INFO ] [REMOTE ] CNS: Full fetch complete (22783 records). [2025-09-16T17:52:28.870760] [INFO ] [REMOTE ] CODI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:28.870960] [DEBUG] [VALIDATOR ] CNS: Validating primary data. [2025-09-16T17:52:29.180038] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-08. [2025-09-16T17:52:29.240906] [DEBUG] [VALIDATOR ] CNS: All expected trading days are present. [2025-09-16T17:52:29.240988] [DEBUG] [VALIDATOR ] CNS: Primary data validation passed. [2025-09-16T17:52:29.241438] [INFO ] [MAIN ] CNS: Fetched primary data is valid. [2025-09-16T17:52:29.241480] [DEBUG] [LOCAL ] CNS: Saving primary data. [2025-09-16T17:52:29.242169] [DEBUG] [LOCAL ] CNS: Trimmed 22783 records to 14689 (keeping 150 days of data) [2025-09-16T17:52:29.332299] [DEBUG] [LOCAL ] CNS: Primary data saved (14689 records). [2025-09-16T17:52:29.332452] [INFO ] [MAIN ] CNS: Full history saved. [2025-09-16T17:52:29.332542] [INFO ] [MAIN ] Update Progress: 499/2369 symbols completed. CNS took 22.2s. Estimated remaining: 3h 16m 31s. Rate: 9.5 symbols/min [2025-09-16T17:52:29.507903] [INFO ] [MAIN ] COF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:29.546446] [DEBUG] [LOCAL ] COF: Loaded 39597 existing records from primary storage. [2025-09-16T17:52:29.546586] [INFO ] [MAIN ] COF: Loaded 39597 existing records. [2025-09-16T17:52:29.726307] [WARN ] [MAIN ] COF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:29.726806] [INFO ] [MAIN ] COF: Wiping existing primary data before full refresh. [2025-09-16T17:52:29.726855] [INFO ] [LOCAL ] COF: Starting deletion of primary data. [2025-09-16T17:52:29.728057] [DEBUG] [LOCAL ] COF: Deleted primary data file. [2025-09-16T17:52:29.728108] [INFO ] [MAIN ] COF: Starting full primary refresh. [2025-09-16T17:52:29.977143] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-02. [2025-09-16T17:52:30.081487] [INFO ] [REMOTE ] COF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:30.140927] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T17:52:30.729905] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T17:52:30.774921] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-08. [2025-09-16T17:52:30.839853] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T17:52:31.574907] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-09. [2025-09-16T17:52:32.374896] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-03. [2025-09-16T17:52:32.504371] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-09-16T17:52:32.978093] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:52:32.984357] [INFO ] [REMOTE ] CNX: Full fetch complete (55955 records). [2025-09-16T17:52:32.984521] [DEBUG] [VALIDATOR ] CNX: Validating primary data. [2025-09-16T17:52:33.180413] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-02. [2025-09-16T17:52:33.380994] [DEBUG] [VALIDATOR ] CNX: All expected trading days are present. [2025-09-16T17:52:33.381085] [DEBUG] [VALIDATOR ] CNX: Primary data validation passed. [2025-09-16T17:52:33.381953] [INFO ] [MAIN ] CNX: Fetched primary data is valid. [2025-09-16T17:52:33.382017] [DEBUG] [LOCAL ] CNX: Saving primary data. [2025-09-16T17:52:33.383274] [DEBUG] [LOCAL ] CNX: Trimmed 55955 records to 38164 (keeping 150 days of data) [2025-09-16T17:52:33.614113] [DEBUG] [LOCAL ] CNX: Primary data saved (38164 records). [2025-09-16T17:52:33.614285] [INFO ] [MAIN ] CNX: Full history saved. [2025-09-16T17:52:33.614380] [INFO ] [MAIN ] Update Progress: 500/2369 symbols completed. CNX took 19.8s. Estimated remaining: 3h 16m 17s. Rate: 9.5 symbols/min [2025-09-16T17:52:33.792707] [INFO ] [MAIN ] COHR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:33.811078] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T17:52:33.853318] [DEBUG] [LOCAL ] COHR: Loaded 39393 existing records from primary storage. [2025-09-16T17:52:33.853498] [INFO ] [MAIN ] COHR: Loaded 39393 existing records. [2025-09-16T17:52:33.979974] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-04. [2025-09-16T17:52:34.037303] [WARN ] [MAIN ] COHR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:34.037578] [INFO ] [MAIN ] COHR: Wiping existing primary data before full refresh. [2025-09-16T17:52:34.037629] [INFO ] [LOCAL ] COHR: Starting deletion of primary data. [2025-09-16T17:52:34.038537] [DEBUG] [LOCAL ] COHR: Deleted primary data file. [2025-09-16T17:52:34.038608] [INFO ] [MAIN ] COHR: Starting full primary refresh. [2025-09-16T17:52:34.395961] [INFO ] [REMOTE ] COHR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:34.774906] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-09. [2025-09-16T17:52:35.053446] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-09-16T17:52:35.195900] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-09-16T17:52:35.574914] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-05. [2025-09-16T17:52:36.329929] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T17:52:36.336454] [INFO ] [REMOTE ] CNQ: Full fetch complete (58257 records). [2025-09-16T17:52:36.336559] [DEBUG] [VALIDATOR ] CNQ: Validating primary data. [2025-09-16T17:52:36.380283] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-03. [2025-09-16T17:52:36.717137] [DEBUG] [VALIDATOR ] CNQ: All expected trading days are present. [2025-09-16T17:52:36.717221] [DEBUG] [VALIDATOR ] CNQ: Primary data validation passed. [2025-09-16T17:52:36.717907] [INFO ] [MAIN ] CNQ: Fetched primary data is valid. [2025-09-16T17:52:36.717951] [DEBUG] [LOCAL ] CNQ: Saving primary data. [2025-09-16T17:52:36.719080] [DEBUG] [LOCAL ] CNQ: Trimmed 58257 records to 39941 (keeping 150 days of data) [2025-09-16T17:52:36.964450] [DEBUG] [LOCAL ] CNQ: Primary data saved (39941 records). [2025-09-16T17:52:36.964603] [INFO ] [MAIN ] CNQ: Full history saved. [2025-09-16T17:52:36.964708] [INFO ] [MAIN ] Update Progress: 501/2369 symbols completed. CNQ took 36.1s. Estimated remaining: 3h 15m 59s. Rate: 9.5 symbols/min [2025-09-16T17:52:37.145027] [INFO ] [MAIN ] COLD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:37.175148] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-09-16T17:52:37.175279] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-02. [2025-09-16T17:52:37.214627] [DEBUG] [LOCAL ] COLD: Loaded 38617 existing records from primary storage. [2025-09-16T17:52:37.214785] [INFO ] [MAIN ] COLD: Loaded 38617 existing records. [2025-09-16T17:52:37.388268] [WARN ] [MAIN ] COLD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:37.388514] [INFO ] [MAIN ] COLD: Wiping existing primary data before full refresh. [2025-09-16T17:52:37.388560] [INFO ] [LOCAL ] COLD: Starting deletion of primary data. [2025-09-16T17:52:37.389345] [DEBUG] [LOCAL ] COLD: Deleted primary data file. [2025-09-16T17:52:37.389395] [INFO ] [MAIN ] COLD: Starting full primary refresh. [2025-09-16T17:52:37.739478] [INFO ] [REMOTE ] COLD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:37.935677] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T17:52:37.974848] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-04. [2025-09-16T17:52:38.774906] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-06. [2025-09-16T17:52:39.241152] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-09-16T17:52:39.574932] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-03. [2025-09-16T17:52:39.778584] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-09-16T17:52:40.186061] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T17:52:40.374916] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-07. [2025-09-16T17:52:41.180044] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-05. [2025-09-16T17:52:41.196690] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T17:52:41.974958] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-02. [2025-09-16T17:52:41.983123] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-09-16T17:52:42.774893] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-08. [2025-09-16T17:52:42.907411] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T17:52:43.522569] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T17:52:43.574939] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-03. [2025-09-16T17:52:43.857788] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T17:52:44.374913] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-09. [2025-09-16T17:52:45.174932] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-04. [2025-09-16T17:52:45.368023] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-09-16T17:52:45.458765] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T17:52:45.463118] [INFO ] [REMOTE ] CODI: Full fetch complete (41530 records). [2025-09-16T17:52:45.463220] [DEBUG] [VALIDATOR ] CODI: Validating primary data. [2025-09-16T17:52:45.845483] [DEBUG] [VALIDATOR ] CODI: All expected trading days are present. [2025-09-16T17:52:45.845598] [DEBUG] [VALIDATOR ] CODI: Primary data validation passed. [2025-09-16T17:52:45.846178] [INFO ] [MAIN ] CODI: Fetched primary data is valid. [2025-09-16T17:52:45.846222] [DEBUG] [LOCAL ] CODI: Saving primary data. [2025-09-16T17:52:45.847262] [DEBUG] [LOCAL ] CODI: Trimmed 41530 records to 31403 (keeping 150 days of data) [2025-09-16T17:52:45.990447] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-06. [2025-09-16T17:52:46.030092] [DEBUG] [LOCAL ] CODI: Primary data saved (31403 records). [2025-09-16T17:52:46.030195] [INFO ] [MAIN ] CODI: Full history saved. [2025-09-16T17:52:46.030273] [INFO ] [MAIN ] Update Progress: 502/2369 symbols completed. CODI took 18.0s. Estimated remaining: 3h 16m 3s. Rate: 9.5 symbols/min [2025-09-16T17:52:46.200064] [INFO ] [MAIN ] COMP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:46.235551] [DEBUG] [LOCAL ] COMP: Loaded 39382 existing records from primary storage. [2025-09-16T17:52:46.235720] [INFO ] [MAIN ] COMP: Loaded 39382 existing records. [2025-09-16T17:52:46.418833] [WARN ] [MAIN ] COMP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:46.419265] [INFO ] [MAIN ] COMP: Wiping existing primary data before full refresh. [2025-09-16T17:52:46.419313] [INFO ] [LOCAL ] COMP: Starting deletion of primary data. [2025-09-16T17:52:46.420063] [DEBUG] [LOCAL ] COMP: Deleted primary data file. [2025-09-16T17:52:46.420111] [INFO ] [MAIN ] COMP: Starting full primary refresh. [2025-09-16T17:52:46.769631] [INFO ] [REMOTE ] COMP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:46.774847] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-02. [2025-09-16T17:52:47.009113] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T17:52:47.475306] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T17:52:47.574910] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-07. [2025-09-16T17:52:48.062938] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T17:52:48.374899] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-03. [2025-09-16T17:52:49.103788] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-09-16T17:52:49.174841] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-08. [2025-09-16T17:52:49.974914] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-05. [2025-09-16T17:52:50.201256] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T17:52:50.774851] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-04. [2025-09-16T17:52:51.040318] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-09-16T17:52:51.574917] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-09. [2025-09-16T17:52:51.913516] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-09-16T17:52:52.180480] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-09-16T17:52:52.374930] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-05. [2025-09-16T17:52:53.136696] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T17:52:53.143224] [INFO ] [REMOTE ] COF: Full fetch complete (58263 records). [2025-09-16T17:52:53.143371] [DEBUG] [VALIDATOR ] COF: Validating primary data. [2025-09-16T17:52:53.180056] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-04. [2025-09-16T17:52:53.538010] [DEBUG] [VALIDATOR ] COF: All expected trading days are present. [2025-09-16T17:52:53.538099] [DEBUG] [VALIDATOR ] COF: Primary data validation passed. [2025-09-16T17:52:53.538792] [INFO ] [MAIN ] COF: Fetched primary data is valid. [2025-09-16T17:52:53.538838] [DEBUG] [LOCAL ] COF: Saving primary data. [2025-09-16T17:52:53.540081] [DEBUG] [LOCAL ] COF: Trimmed 58263 records to 39987 (keeping 150 days of data) [2025-09-16T17:52:53.792385] [DEBUG] [LOCAL ] COF: Primary data saved (39987 records). [2025-09-16T17:52:53.792533] [INFO ] [MAIN ] COF: Full history saved. [2025-09-16T17:52:53.793871] [INFO ] [MAIN ] Update Progress: 503/2369 symbols completed. COF took 24.5s. Estimated remaining: 3h 16m 3s. Rate: 9.5 symbols/min [2025-09-16T17:52:53.969793] [INFO ] [MAIN ] CON: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:52:53.974932] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-06. [2025-09-16T17:52:54.004673] [DEBUG] [LOCAL ] CON: Loaded 28443 existing records from primary storage. [2025-09-16T17:52:54.004943] [INFO ] [MAIN ] CON: Loaded 28443 existing records. [2025-09-16T17:52:54.206310] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-09-16T17:52:54.238729] [WARN ] [MAIN ] CON: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:52:54.239672] [INFO ] [MAIN ] CON: Wiping existing primary data before full refresh. [2025-09-16T17:52:54.239729] [INFO ] [LOCAL ] CON: Starting deletion of primary data. [2025-09-16T17:52:54.240519] [DEBUG] [LOCAL ] CON: Deleted primary data file. [2025-09-16T17:52:54.240574] [INFO ] [MAIN ] CON: Starting full primary refresh. [2025-09-16T17:52:54.621979] [INFO ] [REMOTE ] CON: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:52:54.774875] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-06. [2025-09-16T17:52:54.841246] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T17:52:55.574901] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-05. [2025-09-16T17:52:55.898387] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-09-16T17:52:56.261202] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T17:52:56.374888] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-07. [2025-09-16T17:52:56.850931] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-09-16T17:52:57.174923] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-02. [2025-09-16T17:52:57.974916] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-06. [2025-09-16T17:52:58.055932] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-09-16T17:52:58.469894] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T17:52:58.774922] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-03. [2025-09-16T17:52:59.574898] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-08. [2025-09-16T17:52:59.663277] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T17:52:59.972217] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T17:53:00.374906] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-04. [2025-09-16T17:53:01.174987] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-07. [2025-09-16T17:53:01.757700] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-09-16T17:53:01.848851] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T17:53:01.974912] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-05. [2025-09-16T17:53:02.774912] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-07. [2025-09-16T17:53:02.878459] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T17:53:03.303031] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T17:53:03.574913] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-09. [2025-09-16T17:53:04.374913] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-08. [2025-09-16T17:53:04.425241] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T17:53:05.174906] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-06. [2025-09-16T17:53:05.352265] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T17:53:05.358688] [INFO ] [REMOTE ] COMP: Full fetch complete (58017 records). [2025-09-16T17:53:05.358825] [DEBUG] [VALIDATOR ] COMP: Validating primary data. [2025-09-16T17:53:05.743717] [DEBUG] [VALIDATOR ] COMP: All expected trading days are present. [2025-09-16T17:53:05.743823] [DEBUG] [VALIDATOR ] COMP: Primary data validation passed. [2025-09-16T17:53:05.744544] [INFO ] [MAIN ] COMP: Fetched primary data is valid. [2025-09-16T17:53:05.744604] [DEBUG] [LOCAL ] COMP: Saving primary data. [2025-09-16T17:53:05.745912] [DEBUG] [LOCAL ] COMP: Trimmed 58017 records to 39767 (keeping 150 days of data) [2025-09-16T17:53:05.973129] [DEBUG] [LOCAL ] COMP: Primary data saved (39767 records). [2025-09-16T17:53:05.973284] [INFO ] [MAIN ] COMP: Full history saved. [2025-09-16T17:53:05.974395] [INFO ] [MAIN ] Update Progress: 504/2369 symbols completed. COMP took 19.9s. Estimated remaining: 3h 16m 18s. Rate: 9.5 symbols/min [2025-09-16T17:53:06.006558] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-08. [2025-09-16T17:53:06.151173] [INFO ] [MAIN ] COOK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:53:06.171992] [DEBUG] [LOCAL ] COOK: Loaded 18462 existing records from primary storage. [2025-09-16T17:53:06.172172] [INFO ] [MAIN ] COOK: Loaded 18462 existing records. [2025-09-16T17:53:06.348612] [WARN ] [MAIN ] COOK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:53:06.349243] [INFO ] [MAIN ] COOK: Wiping existing primary data before full refresh. [2025-09-16T17:53:06.349398] [INFO ] [LOCAL ] COOK: Starting deletion of primary data. [2025-09-16T17:53:06.349949] [DEBUG] [LOCAL ] COOK: Deleted primary data file. [2025-09-16T17:53:06.350092] [INFO ] [MAIN ] COOK: Starting full primary refresh. [2025-09-16T17:53:06.491610] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T17:53:06.740790] [INFO ] [REMOTE ] COOK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:53:06.771109] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-08 (2.40 seconds). [2025-09-16T17:53:06.781280] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-02. [2025-09-16T17:53:07.574901] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-07. [2025-09-16T17:53:07.670594] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T17:53:07.698469] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-09-16T17:53:08.374906] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-09. [2025-09-16T17:53:08.833447] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T17:53:09.174916] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-03. [2025-09-16T17:53:09.980033] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-09. [2025-09-16T17:53:09.996117] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T17:53:10.006356] [INFO ] [REMOTE ] COLD: Full fetch complete (56352 records). [2025-09-16T17:53:10.006625] [DEBUG] [VALIDATOR ] COLD: Validating primary data. [2025-09-16T17:53:10.285102] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-09-16T17:53:10.444965] [DEBUG] [VALIDATOR ] COLD: All expected trading days are present. [2025-09-16T17:53:10.445082] [DEBUG] [VALIDATOR ] COLD: Primary data validation passed. [2025-09-16T17:53:10.446022] [INFO ] [MAIN ] COLD: Fetched primary data is valid. [2025-09-16T17:53:10.446085] [DEBUG] [LOCAL ] COLD: Saving primary data. [2025-09-16T17:53:10.447573] [DEBUG] [LOCAL ] COLD: Trimmed 56352 records to 39001 (keeping 150 days of data) [2025-09-16T17:53:10.689663] [DEBUG] [LOCAL ] COLD: Primary data saved (39001 records). [2025-09-16T17:53:10.689834] [INFO ] [MAIN ] COLD: Full history saved. [2025-09-16T17:53:10.689924] [INFO ] [MAIN ] Update Progress: 505/2369 symbols completed. COLD took 33.7s. Estimated remaining: 3h 16m 6s. Rate: 9.5 symbols/min [2025-09-16T17:53:10.780101] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-08. [2025-09-16T17:53:10.871007] [INFO ] [MAIN ] COP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:53:10.906923] [DEBUG] [LOCAL ] COP: Loaded 39687 existing records from primary storage. [2025-09-16T17:53:10.907069] [INFO ] [MAIN ] COP: Loaded 39687 existing records. [2025-09-16T17:53:11.090567] [WARN ] [MAIN ] COP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:53:11.090853] [INFO ] [MAIN ] COP: Wiping existing primary data before full refresh. [2025-09-16T17:53:11.090910] [INFO ] [LOCAL ] COP: Starting deletion of primary data. [2025-09-16T17:53:11.091798] [DEBUG] [LOCAL ] COP: Deleted primary data file. [2025-09-16T17:53:11.091862] [INFO ] [MAIN ] COP: Starting full primary refresh. [2025-09-16T17:53:11.466107] [INFO ] [REMOTE ] COP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:53:11.574929] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-02. [2025-09-16T17:53:12.156968] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-09-16T17:53:12.167730] [INFO ] [REMOTE ] COHR: Full fetch complete (58035 records). [2025-09-16T17:53:12.167960] [DEBUG] [VALIDATOR ] COHR: Validating primary data. [2025-09-16T17:53:12.363298] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T17:53:12.378943] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-04. [2025-09-16T17:53:12.705611] [DEBUG] [VALIDATOR ] COHR: All expected trading days are present. [2025-09-16T17:53:12.705745] [DEBUG] [VALIDATOR ] COHR: Primary data validation passed. [2025-09-16T17:53:12.706695] [INFO ] [MAIN ] COHR: Fetched primary data is valid. [2025-09-16T17:53:12.706746] [DEBUG] [LOCAL ] COHR: Saving primary data. [2025-09-16T17:53:12.708234] [DEBUG] [LOCAL ] COHR: Trimmed 58035 records to 39783 (keeping 150 days of data) [2025-09-16T17:53:12.948094] [DEBUG] [LOCAL ] COHR: Primary data saved (39783 records). [2025-09-16T17:53:12.948245] [INFO ] [MAIN ] COHR: Full history saved. [2025-09-16T17:53:12.948341] [INFO ] [MAIN ] Update Progress: 506/2369 symbols completed. COHR took 39.3s. Estimated remaining: 3h 15m 44s. Rate: 9.5 symbols/min [2025-09-16T17:53:13.121226] [INFO ] [MAIN ] COR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:53:13.121384] [DEBUG] [LOCAL ] COR: No existing data file found in primary storage. [2025-09-16T17:53:13.121422] [WARN ] [MAIN ] COR: No existing data found, performing full refresh instead. [2025-09-16T17:53:13.121595] [INFO ] [MAIN ] COR: Wiping existing primary data before full refresh. [2025-09-16T17:53:13.121670] [INFO ] [LOCAL ] COR: Starting deletion of primary data. [2025-09-16T17:53:13.121729] [DEBUG] [LOCAL ] COR: No primary data file found to delete. [2025-09-16T17:53:13.121765] [INFO ] [MAIN ] COR: Starting full primary refresh. [2025-09-16T17:53:13.185693] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-09. [2025-09-16T17:53:13.448156] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-09-16T17:53:13.515824] [INFO ] [REMOTE ] COR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:53:13.546090] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-09-16T17:53:13.974914] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-05. [2025-09-16T17:53:14.774900] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-03. [2025-09-16T17:53:14.842012] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-09-16T17:53:15.574905] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-06. [2025-09-16T17:53:15.716592] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-09 (2.53 seconds). [2025-09-16T17:53:15.721871] [INFO ] [REMOTE ] CON: Full fetch complete (41829 records). [2025-09-16T17:53:15.722006] [DEBUG] [VALIDATOR ] CON: Validating primary data. [2025-09-16T17:53:15.722726] [FATAL] [VALIDATOR ] CON: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:53:15.722807] [FATAL] [MAIN ] CON: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:53:15.722846] [WARN ] [MAIN ] CON: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:53:15.722886] [INFO ] [LOCAL ] CON: Starting deletion of primary data. [2025-09-16T17:53:15.722946] [DEBUG] [LOCAL ] CON: No primary data file found to delete. [2025-09-16T17:53:15.722994] [INFO ] [LOCAL ] CON: Starting deletion of derivative data. [2025-09-16T17:53:15.723425] [DEBUG] [LOCAL ] CON: Deleted derivative data file. [2025-09-16T17:53:15.724125] [INFO ] [MAIN ] Update Progress: 507/2369 symbols completed. CON took 21.9s. Estimated remaining: 3h 15m 25s. Rate: 9.5 symbols/min [2025-09-16T17:53:15.922582] [INFO ] [MAIN ] COTY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:53:15.971569] [DEBUG] [LOCAL ] COTY: Loaded 39335 existing records from primary storage. [2025-09-16T17:53:15.971809] [INFO ] [MAIN ] COTY: Loaded 39335 existing records. [2025-09-16T17:53:16.174058] [WARN ] [MAIN ] COTY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:53:16.174321] [INFO ] [MAIN ] COTY: Wiping existing primary data before full refresh. [2025-09-16T17:53:16.174368] [INFO ] [LOCAL ] COTY: Starting deletion of primary data. [2025-09-16T17:53:16.175207] [DEBUG] [LOCAL ] COTY: Deleted primary data file. [2025-09-16T17:53:16.175260] [INFO ] [MAIN ] COTY: Starting full primary refresh. [2025-09-16T17:53:16.386024] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-02. [2025-09-16T17:53:16.537422] [INFO ] [REMOTE ] COTY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:53:16.654771] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-09-16T17:53:16.722083] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-09-16T17:53:17.174907] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-07. [2025-09-16T17:53:17.795200] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-09-16T17:53:17.974896] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-03. [2025-09-16T17:53:18.384301] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-09-16T17:53:18.774901] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-02. [2025-09-16T17:53:19.583520] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-04. [2025-09-16T17:53:19.603285] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T17:53:20.273694] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-09-16T17:53:20.374901] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-03. [2025-09-16T17:53:21.174897] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-08. [2025-09-16T17:53:21.527785] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T17:53:21.592580] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-09-16T17:53:21.982447] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-05. [2025-09-16T17:53:21.984682] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T17:53:22.774905] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-04. [2025-09-16T17:53:23.543724] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T17:53:23.574840] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-06. [2025-09-16T17:53:24.120940] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T17:53:24.374945] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-09. [2025-09-16T17:53:25.052232] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T17:53:25.178894] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-05. [2025-09-16T17:53:25.180507] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-09-16T17:53:25.184237] [INFO ] [REMOTE ] COOK: Full fetch complete (25207 records). [2025-09-16T17:53:25.184319] [DEBUG] [VALIDATOR ] COOK: Validating primary data. [2025-09-16T17:53:25.542464] [DEBUG] [VALIDATOR ] COOK: All expected trading days are present. [2025-09-16T17:53:25.542540] [DEBUG] [VALIDATOR ] COOK: Primary data validation passed. [2025-09-16T17:53:25.542958] [INFO ] [MAIN ] COOK: Fetched primary data is valid. [2025-09-16T17:53:25.543001] [DEBUG] [LOCAL ] COOK: Saving primary data. [2025-09-16T17:53:25.543624] [DEBUG] [LOCAL ] COOK: Trimmed 25207 records to 18606 (keeping 150 days of data) [2025-09-16T17:53:25.651613] [DEBUG] [LOCAL ] COOK: Primary data saved (18606 records). [2025-09-16T17:53:25.651769] [INFO ] [MAIN ] COOK: Full history saved. [2025-09-16T17:53:25.651862] [INFO ] [MAIN ] Update Progress: 508/2369 symbols completed. COOK took 19.7s. Estimated remaining: 3h 15m 32s. Rate: 9.5 symbols/min [2025-09-16T17:53:25.820529] [INFO ] [MAIN ] COUR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:53:25.858258] [DEBUG] [LOCAL ] COUR: Loaded 36333 existing records from primary storage. [2025-09-16T17:53:25.858400] [INFO ] [MAIN ] COUR: Loaded 36333 existing records. [2025-09-16T17:53:25.979974] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-07. [2025-09-16T17:53:26.037324] [WARN ] [MAIN ] COUR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:53:26.037560] [INFO ] [MAIN ] COUR: Wiping existing primary data before full refresh. [2025-09-16T17:53:26.037624] [INFO ] [LOCAL ] COUR: Starting deletion of primary data. [2025-09-16T17:53:26.038459] [DEBUG] [LOCAL ] COUR: Deleted primary data file. [2025-09-16T17:53:26.038527] [INFO ] [MAIN ] COUR: Starting full primary refresh. [2025-09-16T17:53:26.394925] [INFO ] [REMOTE ] COUR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:53:26.586501] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-09-16T17:53:26.774902] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-02. [2025-09-16T17:53:27.574918] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-06. [2025-09-16T17:53:27.627891] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T17:53:28.061401] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T17:53:28.374905] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-04. [2025-09-16T17:53:28.819291] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T17:53:29.174901] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-08. [2025-09-16T17:53:29.982770] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-03. [2025-09-16T17:53:30.004056] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-09-16T17:53:30.774908] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-05. [2025-09-16T17:53:31.246517] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T17:53:31.574943] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-07. [2025-09-16T17:53:31.876935] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-08 (2.70 seconds). [2025-09-16T17:53:32.179835] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T17:53:32.374888] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-06. [2025-09-16T17:53:33.066858] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T17:53:33.174933] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-04. [2025-09-16T17:53:33.555699] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-09-16T17:53:33.974898] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-09. [2025-09-16T17:53:34.489063] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T17:53:34.774907] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-08. [2025-09-16T17:53:35.533033] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T17:53:35.539560] [INFO ] [REMOTE ] COP: Full fetch complete (58407 records). [2025-09-16T17:53:35.539708] [DEBUG] [VALIDATOR ] COP: Validating primary data. [2025-09-16T17:53:35.580058] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-07. [2025-09-16T17:53:35.932468] [DEBUG] [VALIDATOR ] COP: All expected trading days are present. [2025-09-16T17:53:35.932551] [DEBUG] [VALIDATOR ] COP: Primary data validation passed. [2025-09-16T17:53:35.933237] [INFO ] [MAIN ] COP: Fetched primary data is valid. [2025-09-16T17:53:35.933285] [DEBUG] [LOCAL ] COP: Saving primary data. [2025-09-16T17:53:35.934511] [DEBUG] [LOCAL ] COP: Trimmed 58407 records to 40077 (keeping 150 days of data) [2025-09-16T17:53:36.220398] [DEBUG] [LOCAL ] COP: Primary data saved (40077 records). [2025-09-16T17:53:36.223283] [INFO ] [MAIN ] COP: Full history saved. [2025-09-16T17:53:36.223396] [INFO ] [MAIN ] Update Progress: 509/2369 symbols completed. COP took 25.5s. Estimated remaining: 3h 15m 42s. Rate: 9.5 symbols/min [2025-09-16T17:53:36.224963] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T17:53:36.381607] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-09. [2025-09-16T17:53:36.414523] [INFO ] [MAIN ] CP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:53:36.453732] [DEBUG] [LOCAL ] CP: Loaded 39276 existing records from primary storage. [2025-09-16T17:53:36.453909] [INFO ] [MAIN ] CP: Loaded 39276 existing records. [2025-09-16T17:53:36.630771] [WARN ] [MAIN ] CP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:53:36.631069] [INFO ] [MAIN ] CP: Wiping existing primary data before full refresh. [2025-09-16T17:53:36.631130] [INFO ] [LOCAL ] CP: Starting deletion of primary data. [2025-09-16T17:53:36.631898] [DEBUG] [LOCAL ] CP: Deleted primary data file. [2025-09-16T17:53:36.631949] [INFO ] [MAIN ] CP: Starting full primary refresh. [2025-09-16T17:53:36.986314] [INFO ] [REMOTE ] CP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:53:37.035974] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T17:53:37.174873] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-02. [2025-09-16T17:53:37.750691] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T17:53:37.757003] [INFO ] [REMOTE ] COTY: Full fetch complete (57936 records). [2025-09-16T17:53:37.757113] [DEBUG] [VALIDATOR ] COTY: Validating primary data. [2025-09-16T17:53:37.980081] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-08. [2025-09-16T17:53:38.154098] [DEBUG] [VALIDATOR ] COTY: All expected trading days are present. [2025-09-16T17:53:38.154179] [DEBUG] [VALIDATOR ] COTY: Primary data validation passed. [2025-09-16T17:53:38.154893] [INFO ] [MAIN ] COTY: Fetched primary data is valid. [2025-09-16T17:53:38.154940] [DEBUG] [LOCAL ] COTY: Saving primary data. [2025-09-16T17:53:38.156160] [DEBUG] [LOCAL ] COTY: Trimmed 57936 records to 39719 (keeping 150 days of data) [2025-09-16T17:53:38.396793] [DEBUG] [LOCAL ] COTY: Primary data saved (39719 records). [2025-09-16T17:53:38.397213] [INFO ] [MAIN ] COTY: Full history saved. [2025-09-16T17:53:38.397920] [INFO ] [MAIN ] Update Progress: 510/2369 symbols completed. COTY took 22.7s. Estimated remaining: 3h 15m 20s. Rate: 9.5 symbols/min [2025-09-16T17:53:38.581204] [INFO ] [MAIN ] CPA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:53:38.615238] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T17:53:38.639807] [DEBUG] [LOCAL ] CPA: Loaded 21144 existing records from primary storage. [2025-09-16T17:53:38.639934] [INFO ] [MAIN ] CPA: Loaded 21144 existing records. [2025-09-16T17:53:38.788141] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-03. [2025-09-16T17:53:38.826455] [WARN ] [MAIN ] CPA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:53:38.826698] [INFO ] [MAIN ] CPA: Wiping existing primary data before full refresh. [2025-09-16T17:53:38.826747] [INFO ] [LOCAL ] CPA: Starting deletion of primary data. [2025-09-16T17:53:38.827270] [DEBUG] [LOCAL ] CPA: Deleted primary data file. [2025-09-16T17:53:38.827324] [INFO ] [MAIN ] CPA: Starting full primary refresh. [2025-09-16T17:53:39.198068] [INFO ] [REMOTE ] CPA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:53:39.313694] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-09-16T17:53:39.574933] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-05. [2025-09-16T17:53:40.307660] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T17:53:40.374864] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-04. [2025-09-16T17:53:41.174897] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-02. [2025-09-16T17:53:41.513969] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-09-16T17:53:41.974890] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-09. [2025-09-16T17:53:42.099237] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T17:53:42.121282] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-09-16T17:53:42.774938] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-05. [2025-09-16T17:53:43.318926] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T17:53:43.324841] [INFO ] [REMOTE ] COR: Full fetch complete (51611 records). [2025-09-16T17:53:43.324969] [DEBUG] [VALIDATOR ] COR: Validating primary data. [2025-09-16T17:53:43.580804] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-03. [2025-09-16T17:53:43.717828] [DEBUG] [VALIDATOR ] COR: All expected trading days are present. [2025-09-16T17:53:43.717920] [DEBUG] [VALIDATOR ] COR: Primary data validation passed. [2025-09-16T17:53:43.718674] [INFO ] [MAIN ] COR: Fetched primary data is valid. [2025-09-16T17:53:43.718741] [DEBUG] [LOCAL ] COR: Saving primary data. [2025-09-16T17:53:43.719930] [DEBUG] [LOCAL ] COR: Trimmed 51611 records to 34601 (keeping 150 days of data) [2025-09-16T17:53:43.953399] [DEBUG] [LOCAL ] COR: Primary data saved (34601 records). [2025-09-16T17:53:43.953557] [INFO ] [MAIN ] COR: Full history saved. [2025-09-16T17:53:43.953660] [INFO ] [MAIN ] Update Progress: 511/2369 symbols completed. COR took 31.0s. Estimated remaining: 3h 15m 11s. Rate: 9.5 symbols/min [2025-09-16T17:53:44.133152] [INFO ] [MAIN ] CPAC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:53:44.133372] [DEBUG] [LOCAL ] CPAC: No existing data file found in primary storage. [2025-09-16T17:53:44.133417] [WARN ] [MAIN ] CPAC: No existing data found, performing full refresh instead. [2025-09-16T17:53:44.133565] [INFO ] [MAIN ] CPAC: Wiping existing primary data before full refresh. [2025-09-16T17:53:44.133613] [INFO ] [LOCAL ] CPAC: Starting deletion of primary data. [2025-09-16T17:53:44.133675] [DEBUG] [LOCAL ] CPAC: No primary data file found to delete. [2025-09-16T17:53:44.133719] [INFO ] [MAIN ] CPAC: Starting full primary refresh. [2025-09-16T17:53:44.381575] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-06. [2025-09-16T17:53:44.495418] [INFO ] [REMOTE ] CPAC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:53:44.623904] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-09-16T17:53:45.126571] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T17:53:45.174885] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-04. [2025-09-16T17:53:45.965900] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T17:53:45.975403] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-06. [2025-09-16T17:53:46.477726] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-09-16T17:53:46.774920] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-02. [2025-09-16T17:53:47.210160] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-09-16T17:53:47.574930] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-03. [2025-09-16T17:53:47.864472] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-09-16T17:53:47.889027] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-09-16T17:53:48.374975] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-05. [2025-09-16T17:53:49.174923] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-07. [2025-09-16T17:53:49.834497] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T17:53:49.974870] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-06. [2025-09-16T17:53:50.774945] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-07. [2025-09-16T17:53:51.022309] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T17:53:51.085134] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-09-16T17:53:51.574950] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-04. [2025-09-16T17:53:51.924732] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-09-16T17:53:52.374906] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-08. [2025-09-16T17:53:52.638194] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-09-16T17:53:53.174900] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-05. [2025-09-16T17:53:53.496935] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-09-16T17:53:53.748904] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-09-16T17:53:53.974898] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-06. [2025-09-16T17:53:54.257829] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-06 (0.28 seconds). [2025-09-16T17:53:54.774930] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-09. [2025-09-16T17:53:55.574904] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-07. [2025-09-16T17:53:55.973350] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-09-16T17:53:56.374909] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-07. [2025-09-16T17:53:56.485544] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T17:53:56.492169] [INFO ] [REMOTE ] CP: Full fetch complete (57914 records). [2025-09-16T17:53:56.492300] [DEBUG] [VALIDATOR ] CP: Validating primary data. [2025-09-16T17:53:56.890328] [DEBUG] [VALIDATOR ] CP: All expected trading days are present. [2025-09-16T17:53:56.890467] [DEBUG] [VALIDATOR ] CP: Primary data validation passed. [2025-09-16T17:53:56.891189] [INFO ] [MAIN ] CP: Fetched primary data is valid. [2025-09-16T17:53:56.891239] [DEBUG] [LOCAL ] CP: Saving primary data. [2025-09-16T17:53:56.892583] [DEBUG] [LOCAL ] CP: Trimmed 57914 records to 39666 (keeping 150 days of data) [2025-09-16T17:53:57.134966] [DEBUG] [LOCAL ] CP: Primary data saved (39666 records). [2025-09-16T17:53:57.135128] [INFO ] [MAIN ] CP: Full history saved. [2025-09-16T17:53:57.135227] [INFO ] [MAIN ] Update Progress: 512/2369 symbols completed. CP took 20.9s. Estimated remaining: 3h 15m 30s. Rate: 9.5 symbols/min [2025-09-16T17:53:57.180005] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-08. [2025-09-16T17:53:57.305752] [INFO ] [MAIN ] CPAY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:53:57.332991] [DEBUG] [LOCAL ] CPAY: Loaded 22320 existing records from primary storage. [2025-09-16T17:53:57.333156] [INFO ] [MAIN ] CPAY: Loaded 22320 existing records. [2025-09-16T17:53:57.559018] [WARN ] [MAIN ] CPAY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:53:57.559268] [INFO ] [MAIN ] CPAY: Wiping existing primary data before full refresh. [2025-09-16T17:53:57.559330] [INFO ] [LOCAL ] CPAY: Starting deletion of primary data. [2025-09-16T17:53:57.560029] [DEBUG] [LOCAL ] CPAY: Deleted primary data file. [2025-09-16T17:53:57.560083] [INFO ] [MAIN ] CPAY: Starting full primary refresh. [2025-09-16T17:53:57.927852] [INFO ] [REMOTE ] CPAY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:53:57.962688] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T17:53:57.974863] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-08. [2025-09-16T17:53:58.702015] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-09-16T17:53:58.774908] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-08. [2025-09-16T17:53:59.140287] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T17:53:59.237336] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-09-16T17:53:59.574892] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-09. [2025-09-16T17:54:00.374909] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-09. [2025-09-16T17:54:00.890964] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-09-16T17:54:00.891758] [INFO ] [REMOTE ] CPAC: Full fetch complete (1259 records). [2025-09-16T17:54:00.891826] [DEBUG] [VALIDATOR ] CPAC: Validating primary data. [2025-09-16T17:54:01.068316] [WARN ] [VALIDATOR ] CPAC: Validation failed - insufficient historical data. Have 149 unique days, need 150 days [2025-09-16T17:54:01.068490] [FATAL] [MAIN ] CPAC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:54:01.068539] [WARN ] [MAIN ] CPAC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:54:01.068583] [INFO ] [LOCAL ] CPAC: Starting deletion of primary data. [2025-09-16T17:54:01.068639] [DEBUG] [LOCAL ] CPAC: No primary data file found to delete. [2025-09-16T17:54:01.068695] [INFO ] [LOCAL ] CPAC: Starting deletion of derivative data. [2025-09-16T17:54:01.068737] [DEBUG] [LOCAL ] CPAC: No derivative data file found to delete. [2025-09-16T17:54:01.068794] [INFO ] [MAIN ] Update Progress: 513/2369 symbols completed. CPAC took 17.1s. Estimated remaining: 3h 15m 15s. Rate: 9.5 symbols/min [2025-09-16T17:54:01.180035] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-09. [2025-09-16T17:54:01.242345] [INFO ] [MAIN ] CPF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:01.259707] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T17:54:01.268149] [INFO ] [REMOTE ] COUR: Full fetch complete (53068 records). [2025-09-16T17:54:01.268298] [DEBUG] [VALIDATOR ] COUR: Validating primary data. [2025-09-16T17:54:01.310429] [DEBUG] [LOCAL ] CPF: Loaded 10798 existing records from primary storage. [2025-09-16T17:54:01.344254] [INFO ] [MAIN ] CPF: Loaded 10798 existing records. [2025-09-16T17:54:01.710762] [WARN ] [MAIN ] CPF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:01.747703] [INFO ] [MAIN ] CPF: Wiping existing primary data before full refresh. [2025-09-16T17:54:01.750389] [INFO ] [LOCAL ] CPF: Starting deletion of primary data. [2025-09-16T17:54:01.756289] [DEBUG] [LOCAL ] CPF: Deleted primary data file. [2025-09-16T17:54:01.766625] [INFO ] [MAIN ] CPF: Starting full primary refresh. [2025-09-16T17:54:01.943082] [DEBUG] [VALIDATOR ] COUR: All expected trading days are present. [2025-09-16T17:54:01.943221] [DEBUG] [VALIDATOR ] COUR: Primary data validation passed. [2025-09-16T17:54:01.943900] [INFO ] [MAIN ] COUR: Fetched primary data is valid. [2025-09-16T17:54:01.944000] [DEBUG] [LOCAL ] COUR: Saving primary data. [2025-09-16T17:54:01.984284] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-02. [2025-09-16T17:54:02.033535] [DEBUG] [LOCAL ] COUR: Trimmed 53068 records to 36689 (keeping 150 days of data) [2025-09-16T17:54:02.290840] [INFO ] [REMOTE ] CPF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:54:02.422826] [DEBUG] [LOCAL ] COUR: Primary data saved (36689 records). [2025-09-16T17:54:02.422979] [INFO ] [MAIN ] COUR: Full history saved. [2025-09-16T17:54:02.425178] [INFO ] [MAIN ] Update Progress: 514/2369 symbols completed. COUR took 36.8s. Estimated remaining: 3h 14m 51s. Rate: 9.5 symbols/min [2025-09-16T17:54:02.597085] [INFO ] [MAIN ] CPK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:02.606317] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-09-16T17:54:02.610221] [INFO ] [REMOTE ] CPA: Full fetch complete (31858 records). [2025-09-16T17:54:02.610363] [DEBUG] [VALIDATOR ] CPA: Validating primary data. [2025-09-16T17:54:02.717154] [DEBUG] [LOCAL ] CPK: Loaded 9139 existing records from primary storage. [2025-09-16T17:54:02.722759] [INFO ] [MAIN ] CPK: Loaded 9139 existing records. [2025-09-16T17:54:02.794314] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-02. [2025-09-16T17:54:03.108324] [WARN ] [MAIN ] CPK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:03.194659] [DEBUG] [VALIDATOR ] CPA: All expected trading days are present. [2025-09-16T17:54:03.194832] [INFO ] [MAIN ] CPK: Wiping existing primary data before full refresh. [2025-09-16T17:54:03.194966] [DEBUG] [VALIDATOR ] CPA: Primary data validation passed. [2025-09-16T17:54:03.195728] [INFO ] [LOCAL ] CPK: Starting deletion of primary data. [2025-09-16T17:54:03.195550] [INFO ] [MAIN ] CPA: Fetched primary data is valid. [2025-09-16T17:54:03.195978] [DEBUG] [LOCAL ] CPA: Saving primary data. [2025-09-16T17:54:03.196367] [DEBUG] [LOCAL ] CPK: Deleted primary data file. [2025-09-16T17:54:03.196897] [INFO ] [MAIN ] CPK: Starting full primary refresh. [2025-09-16T17:54:03.248037] [DEBUG] [LOCAL ] CPA: Trimmed 31858 records to 21298 (keeping 150 days of data) [2025-09-16T17:54:03.512404] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T17:54:03.580257] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-03. [2025-09-16T17:54:03.744033] [INFO ] [REMOTE ] CPK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:54:03.762581] [DEBUG] [LOCAL ] CPA: Primary data saved (21298 records). [2025-09-16T17:54:03.762731] [INFO ] [MAIN ] CPA: Full history saved. [2025-09-16T17:54:03.762822] [INFO ] [MAIN ] Update Progress: 515/2369 symbols completed. CPA took 25.4s. Estimated remaining: 3h 14m 26s. Rate: 9.5 symbols/min [2025-09-16T17:54:03.930925] [INFO ] [MAIN ] CPNG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:03.970893] [DEBUG] [LOCAL ] CPNG: Loaded 39648 existing records from primary storage. [2025-09-16T17:54:03.971047] [INFO ] [MAIN ] CPNG: Loaded 39648 existing records. [2025-09-16T17:54:04.151246] [WARN ] [MAIN ] CPNG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:04.151595] [INFO ] [MAIN ] CPNG: Wiping existing primary data before full refresh. [2025-09-16T17:54:04.151674] [INFO ] [LOCAL ] CPNG: Starting deletion of primary data. [2025-09-16T17:54:04.157879] [DEBUG] [LOCAL ] CPNG: Deleted primary data file. [2025-09-16T17:54:04.163387] [INFO ] [MAIN ] CPNG: Starting full primary refresh. [2025-09-16T17:54:04.185678] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T17:54:04.380127] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-03. [2025-09-16T17:54:04.735858] [INFO ] [REMOTE ] CPNG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:54:04.983341] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T17:54:05.097799] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-09-16T17:54:05.174959] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-02. [2025-09-16T17:54:05.974922] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-02. [2025-09-16T17:54:06.774903] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-04. [2025-09-16T17:54:06.839440] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T17:54:07.074092] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-09-16T17:54:07.574909] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-04. [2025-09-16T17:54:08.102695] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T17:54:08.374904] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-05. [2025-09-16T17:54:08.558203] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-09-16T17:54:09.174930] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-05. [2025-09-16T17:54:09.277919] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-09-16T17:54:09.974904] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-03. [2025-09-16T17:54:10.091564] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-09-16T17:54:10.774898] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-06. [2025-09-16T17:54:11.574941] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-06. [2025-09-16T17:54:11.651533] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-09-16T17:54:11.910843] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-09-16T17:54:12.374900] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-03. [2025-09-16T17:54:12.902286] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T17:54:13.174885] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-07. [2025-09-16T17:54:13.319373] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T17:54:13.974903] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-04. [2025-09-16T17:54:14.497851] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-09-16T17:54:14.774897] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-07. [2025-09-16T17:54:15.054384] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-09-16T17:54:15.574911] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-05. [2025-09-16T17:54:15.827708] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-09-16T17:54:16.374899] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-08. [2025-09-16T17:54:16.536337] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-09-16T17:54:17.174913] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-04. [2025-09-16T17:54:17.381864] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T17:54:17.974910] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-08. [2025-09-16T17:54:18.774914] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-06. [2025-09-16T17:54:19.207945] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-09-16T17:54:19.377728] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-09-16T17:54:19.574905] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-05. [2025-09-16T17:54:19.823129] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T17:54:20.374889] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-09. [2025-09-16T17:54:21.174951] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-09. [2025-09-16T17:54:21.297977] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-09-16T17:54:21.298335] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T17:54:21.312704] [INFO ] [REMOTE ] CPF: Full fetch complete (16947 records). [2025-09-16T17:54:21.313253] [DEBUG] [VALIDATOR ] CPF: Validating primary data. [2025-09-16T17:54:21.675632] [DEBUG] [VALIDATOR ] CPF: All expected trading days are present. [2025-09-16T17:54:21.675734] [DEBUG] [VALIDATOR ] CPF: Primary data validation passed. [2025-09-16T17:54:21.676133] [INFO ] [MAIN ] CPF: Fetched primary data is valid. [2025-09-16T17:54:21.676173] [DEBUG] [LOCAL ] CPF: Saving primary data. [2025-09-16T17:54:21.676772] [DEBUG] [LOCAL ] CPF: Trimmed 16947 records to 10949 (keeping 150 days of data) [2025-09-16T17:54:21.743259] [DEBUG] [LOCAL ] CPF: Primary data saved (10949 records). [2025-09-16T17:54:21.743401] [INFO ] [MAIN ] CPF: Full history saved. [2025-09-16T17:54:21.743486] [INFO ] [MAIN ] Update Progress: 516/2369 symbols completed. CPF took 20.7s. Estimated remaining: 3h 15m 2s. Rate: 9.5 symbols/min [2025-09-16T17:54:21.913610] [INFO ] [MAIN ] CPRI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:21.953259] [DEBUG] [LOCAL ] CPRI: Loaded 38730 existing records from primary storage. [2025-09-16T17:54:21.953399] [INFO ] [MAIN ] CPRI: Loaded 38730 existing records. [2025-09-16T17:54:21.984096] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-06. [2025-09-16T17:54:22.137628] [WARN ] [MAIN ] CPRI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:22.137896] [INFO ] [MAIN ] CPRI: Wiping existing primary data before full refresh. [2025-09-16T17:54:22.137943] [INFO ] [LOCAL ] CPRI: Starting deletion of primary data. [2025-09-16T17:54:22.138727] [DEBUG] [LOCAL ] CPRI: Deleted primary data file. [2025-09-16T17:54:22.138778] [INFO ] [MAIN ] CPRI: Starting full primary refresh. [2025-09-16T17:54:22.496531] [INFO ] [REMOTE ] CPRI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:54:22.505721] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-09-16T17:54:22.509238] [INFO ] [REMOTE ] CPAY: Full fetch complete (33905 records). [2025-09-16T17:54:22.509345] [DEBUG] [VALIDATOR ] CPAY: Validating primary data. [2025-09-16T17:54:22.779995] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-02. [2025-09-16T17:54:22.882289] [DEBUG] [VALIDATOR ] CPAY: All expected trading days are present. [2025-09-16T17:54:22.882376] [DEBUG] [VALIDATOR ] CPAY: Primary data validation passed. [2025-09-16T17:54:22.882896] [INFO ] [MAIN ] CPAY: Fetched primary data is valid. [2025-09-16T17:54:22.882941] [DEBUG] [LOCAL ] CPAY: Saving primary data. [2025-09-16T17:54:22.883921] [DEBUG] [LOCAL ] CPAY: Trimmed 33905 records to 22534 (keeping 150 days of data) [2025-09-16T17:54:23.017696] [DEBUG] [LOCAL ] CPAY: Primary data saved (22534 records). [2025-09-16T17:54:23.017833] [INFO ] [MAIN ] CPAY: Full history saved. [2025-09-16T17:54:23.017918] [INFO ] [MAIN ] Update Progress: 517/2369 symbols completed. CPAY took 25.9s. Estimated remaining: 3h 14m 38s. Rate: 9.5 symbols/min [2025-09-16T17:54:23.574916] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-07. [2025-09-16T17:54:24.010447] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-09-16T17:54:24.374918] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-07. [2025-09-16T17:54:24.482990] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-09-16T17:54:24.546326] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-09-16T17:54:25.174909] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-03. [2025-09-16T17:54:25.974936] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-08. [2025-09-16T17:54:26.257410] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T17:54:26.408607] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-09-16T17:54:26.524677] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-09-16T17:54:26.774925] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-08. [2025-09-16T17:54:27.574942] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-04. [2025-09-16T17:54:28.374930] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-09. [2025-09-16T17:54:28.449883] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T17:54:28.948871] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-09-16T17:54:28.951026] [INFO ] [REMOTE ] CPK: Full fetch complete (14061 records). [2025-09-16T17:54:28.951102] [DEBUG] [VALIDATOR ] CPK: Validating primary data. [2025-09-16T17:54:29.181267] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-09. [2025-09-16T17:54:29.342490] [DEBUG] [VALIDATOR ] CPK: All expected trading days are present. [2025-09-16T17:54:29.342610] [DEBUG] [VALIDATOR ] CPK: Primary data validation passed. [2025-09-16T17:54:29.342996] [INFO ] [MAIN ] CPK: Fetched primary data is valid. [2025-09-16T17:54:29.343101] [DEBUG] [LOCAL ] CPK: Saving primary data. [2025-09-16T17:54:29.359303] [DEBUG] [LOCAL ] CPK: Trimmed 14061 records to 9239 (keeping 150 days of data) [2025-09-16T17:54:29.407864] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T17:54:29.454282] [DEBUG] [LOCAL ] CPK: Primary data saved (9239 records). [2025-09-16T17:54:29.454453] [INFO ] [MAIN ] CPK: Full history saved. [2025-09-16T17:54:29.454545] [INFO ] [MAIN ] Update Progress: 518/2369 symbols completed. CPK took 27.0s. Estimated remaining: 3h 14m 32s. Rate: 9.5 symbols/min [2025-09-16T17:54:29.974962] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-05. [2025-09-16T17:54:31.072758] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-09-16T17:54:31.079127] [INFO ] [REMOTE ] CPNG: Full fetch complete (58349 records). [2025-09-16T17:54:31.079244] [DEBUG] [VALIDATOR ] CPNG: Validating primary data. [2025-09-16T17:54:31.458010] [DEBUG] [VALIDATOR ] CPNG: All expected trading days are present. [2025-09-16T17:54:31.458154] [DEBUG] [VALIDATOR ] CPNG: Primary data validation passed. [2025-09-16T17:54:31.458864] [INFO ] [MAIN ] CPNG: Fetched primary data is valid. [2025-09-16T17:54:31.458940] [DEBUG] [LOCAL ] CPNG: Saving primary data. [2025-09-16T17:54:31.471131] [DEBUG] [LOCAL ] CPNG: Trimmed 58349 records to 40038 (keeping 150 days of data) [2025-09-16T17:54:31.492820] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T17:54:31.590900] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-06. [2025-09-16T17:54:31.742104] [DEBUG] [LOCAL ] CPNG: Primary data saved (40038 records). [2025-09-16T17:54:31.742235] [INFO ] [MAIN ] CPNG: Full history saved. [2025-09-16T17:54:31.743321] [INFO ] [MAIN ] Update Progress: 519/2369 symbols completed. CPNG took 28.0s. Estimated remaining: 3h 14m 11s. Rate: 9.5 symbols/min [2025-09-16T17:54:32.934863] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-09-16T17:54:32.945404] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-07. [2025-09-16T17:54:34.093736] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T17:54:34.105412] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-08. [2025-09-16T17:54:35.387190] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-09-16T17:54:35.407944] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-09. [2025-09-16T17:54:36.692171] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-09-16T17:54:36.701624] [INFO ] [REMOTE ] CPRI: Full fetch complete (57175 records). [2025-09-16T17:54:36.701842] [DEBUG] [VALIDATOR ] CPRI: Validating primary data. [2025-09-16T17:54:37.227303] [DEBUG] [VALIDATOR ] CPRI: All expected trading days are present. [2025-09-16T17:54:37.227449] [DEBUG] [VALIDATOR ] CPRI: Primary data validation passed. [2025-09-16T17:54:37.228453] [INFO ] [MAIN ] CPRI: Fetched primary data is valid. [2025-09-16T17:54:37.228517] [DEBUG] [LOCAL ] CPRI: Saving primary data. [2025-09-16T17:54:37.230477] [DEBUG] [LOCAL ] CPRI: Trimmed 57175 records to 39119 (keeping 150 days of data) [2025-09-16T17:54:37.480580] [DEBUG] [LOCAL ] CPRI: Primary data saved (39119 records). [2025-09-16T17:54:37.480843] [INFO ] [MAIN ] CPRI: Full history saved. [2025-09-16T17:54:37.480957] [INFO ] [MAIN ] Update Progress: 520/2369 symbols completed. CPRI took 15.7s. Estimated remaining: 3h 14m 3s. Rate: 9.5 symbols/min [2025-09-16T17:54:38.100480] [INFO ] [MAIN ] CPT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:38.211347] [INFO ] [MAIN ] CR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:38.234472] [INFO ] [MAIN ] CPS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:38.230024] [INFO ] [MAIN ] CQP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:38.279057] [DEBUG] [LOCAL ] CPS: Loaded 18514 existing records from primary storage. [2025-09-16T17:54:38.279497] [DEBUG] [LOCAL ] CR: Loaded 17772 existing records from primary storage. [2025-09-16T17:54:38.279741] [INFO ] [MAIN ] CR: Loaded 17772 existing records. [2025-09-16T17:54:38.279626] [INFO ] [MAIN ] CPS: Loaded 18514 existing records. [2025-09-16T17:54:38.280136] [DEBUG] [LOCAL ] CQP: Loaded 14914 existing records from primary storage. [2025-09-16T17:54:38.355713] [INFO ] [MAIN ] CQP: Loaded 14914 existing records. [2025-09-16T17:54:38.597947] [DEBUG] [LOCAL ] CPT: Loaded 32473 existing records from primary storage. [2025-09-16T17:54:38.634630] [INFO ] [MAIN ] CPT: Loaded 32473 existing records. [2025-09-16T17:54:38.891917] [WARN ] [MAIN ] CPS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:38.921011] [WARN ] [MAIN ] CR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:38.942492] [INFO ] [MAIN ] CPS: Wiping existing primary data before full refresh. [2025-09-16T17:54:38.947685] [INFO ] [MAIN ] CR: Wiping existing primary data before full refresh. [2025-09-16T17:54:38.979782] [INFO ] [LOCAL ] CR: Starting deletion of primary data. [2025-09-16T17:54:38.986665] [INFO ] [LOCAL ] CPS: Starting deletion of primary data. [2025-09-16T17:54:39.020622] [DEBUG] [LOCAL ] CPS: Deleted primary data file. [2025-09-16T17:54:39.020828] [WARN ] [MAIN ] CQP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:39.031276] [DEBUG] [LOCAL ] CR: Deleted primary data file. [2025-09-16T17:54:39.048302] [INFO ] [MAIN ] CR: Starting full primary refresh. [2025-09-16T17:54:39.031453] [INFO ] [MAIN ] CPS: Starting full primary refresh. [2025-09-16T17:54:39.082213] [INFO ] [MAIN ] CQP: Wiping existing primary data before full refresh. [2025-09-16T17:54:39.089201] [INFO ] [LOCAL ] CQP: Starting deletion of primary data. [2025-09-16T17:54:39.115043] [DEBUG] [LOCAL ] CQP: Deleted primary data file. [2025-09-16T17:54:39.150546] [INFO ] [MAIN ] CQP: Starting full primary refresh. [2025-09-16T17:54:39.169082] [WARN ] [MAIN ] CPT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:39.211785] [INFO ] [MAIN ] CPT: Wiping existing primary data before full refresh. [2025-09-16T17:54:39.238800] [INFO ] [LOCAL ] CPT: Starting deletion of primary data. [2025-09-16T17:54:39.349295] [DEBUG] [LOCAL ] CPT: Deleted primary data file. [2025-09-16T17:54:39.354820] [INFO ] [MAIN ] CPT: Starting full primary refresh. [2025-09-16T17:54:40.269921] [INFO ] [REMOTE ] CR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:54:40.363668] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-02. [2025-09-16T17:54:40.363406] [INFO ] [REMOTE ] CQP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:54:40.410725] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-02. [2025-09-16T17:54:40.484602] [INFO ] [REMOTE ] CPS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:54:40.488579] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-02. [2025-09-16T17:54:40.567446] [INFO ] [REMOTE ] CPT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:54:40.567780] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-02. [2025-09-16T17:54:41.514581] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T17:54:41.519953] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-03. [2025-09-16T17:54:41.679808] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-09-16T17:54:41.684291] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-03. [2025-09-16T17:54:41.846685] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T17:54:41.851859] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-03. [2025-09-16T17:54:42.300409] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-09-16T17:54:42.305262] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-04. [2025-09-16T17:54:42.468531] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-09-16T17:54:42.477440] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-03. [2025-09-16T17:54:42.560867] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-09-16T17:54:42.566562] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-04. [2025-09-16T17:54:42.981315] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T17:54:42.999624] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-05. [2025-09-16T17:54:43.001399] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T17:54:43.007746] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-04. [2025-09-16T17:54:43.602275] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T17:54:43.614720] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-05. [2025-09-16T17:54:43.889368] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-09-16T17:54:43.939028] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-09-16T17:54:44.072251] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-09-16T17:54:44.374879] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-06. [2025-09-16T17:54:44.433615] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T17:54:45.174921] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-05. [2025-09-16T17:54:45.679702] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:54:45.974945] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-07. [2025-09-16T17:54:46.089273] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T17:54:46.774857] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-04. [2025-09-16T17:54:46.818924] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-09-16T17:54:47.574908] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-06. [2025-09-16T17:54:48.130219] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-09-16T17:54:48.373309] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-09-16T17:54:48.378631] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-06. [2025-09-16T17:54:49.178196] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-07. [2025-09-16T17:54:49.180246] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-09-16T17:54:49.946962] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T17:54:49.974881] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-08. [2025-09-16T17:54:50.571857] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T17:54:50.774926] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-05. [2025-09-16T17:54:51.574939] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-09. [2025-09-16T17:54:52.068202] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-09-16T17:54:52.374883] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-07. [2025-09-16T17:54:52.414962] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T17:54:52.416943] [INFO ] [REMOTE ] CQP: Full fetch complete (22076 records). [2025-09-16T17:54:52.417014] [DEBUG] [VALIDATOR ] CQP: Validating primary data. [2025-09-16T17:54:52.785209] [DEBUG] [VALIDATOR ] CQP: All expected trading days are present. [2025-09-16T17:54:52.785300] [DEBUG] [VALIDATOR ] CQP: Primary data validation passed. [2025-09-16T17:54:52.785719] [INFO ] [MAIN ] CQP: Fetched primary data is valid. [2025-09-16T17:54:52.785771] [DEBUG] [LOCAL ] CQP: Saving primary data. [2025-09-16T17:54:52.786492] [DEBUG] [LOCAL ] CQP: Trimmed 22076 records to 15075 (keeping 150 days of data) [2025-09-16T17:54:52.877920] [DEBUG] [LOCAL ] CQP: Primary data saved (15075 records). [2025-09-16T17:54:52.878051] [INFO ] [MAIN ] CQP: Full history saved. [2025-09-16T17:54:52.878152] [INFO ] [MAIN ] Update Progress: 521/2369 symbols completed. CQP took 15.4s. Estimated remaining: 3h 14m 29s. Rate: 9.5 symbols/min [2025-09-16T17:54:53.046669] [INFO ] [MAIN ] CRBD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:53.054781] [DEBUG] [LOCAL ] CRBD: Loaded 6872 existing records from primary storage. [2025-09-16T17:54:53.054878] [INFO ] [MAIN ] CRBD: Loaded 6872 existing records. [2025-09-16T17:54:53.179989] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-06. [2025-09-16T17:54:53.230081] [WARN ] [MAIN ] CRBD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:53.230369] [INFO ] [MAIN ] CRBD: Wiping existing primary data before full refresh. [2025-09-16T17:54:53.230414] [INFO ] [LOCAL ] CRBD: Starting deletion of primary data. [2025-09-16T17:54:53.230735] [DEBUG] [LOCAL ] CRBD: Deleted primary data file. [2025-09-16T17:54:53.230787] [INFO ] [MAIN ] CRBD: Starting full primary refresh. [2025-09-16T17:54:53.592606] [INFO ] [REMOTE ] CRBD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:54:53.607393] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-09-16T17:54:53.974933] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-08. [2025-09-16T17:54:54.726547] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-09-16T17:54:54.774856] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-07. [2025-09-16T17:54:55.221991] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-09-16T17:54:55.574897] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-08. [2025-09-16T17:54:55.892130] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T17:54:56.374871] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-02. [2025-09-16T17:54:56.405002] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-09-16T17:54:57.174919] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-09. [2025-09-16T17:54:57.256364] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-09-16T17:54:57.974978] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-09. [2025-09-16T17:54:58.074803] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-09-16T17:54:58.080399] [INFO ] [REMOTE ] CR: Full fetch complete (26595 records). [2025-09-16T17:54:58.080608] [DEBUG] [VALIDATOR ] CR: Validating primary data. [2025-09-16T17:54:58.671069] [DEBUG] [VALIDATOR ] CR: All expected trading days are present. [2025-09-16T17:54:58.671165] [DEBUG] [VALIDATOR ] CR: Primary data validation passed. [2025-09-16T17:54:58.671679] [INFO ] [MAIN ] CR: Fetched primary data is valid. [2025-09-16T17:54:58.671728] [DEBUG] [LOCAL ] CR: Saving primary data. [2025-09-16T17:54:58.672573] [DEBUG] [LOCAL ] CR: Trimmed 26595 records to 17929 (keeping 150 days of data) [2025-09-16T17:54:58.779701] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-08. [2025-09-16T17:54:58.810258] [DEBUG] [LOCAL ] CR: Primary data saved (17929 records). [2025-09-16T17:54:58.810435] [INFO ] [MAIN ] CR: Full history saved. [2025-09-16T17:54:58.810536] [INFO ] [MAIN ] Update Progress: 522/2369 symbols completed. CR took 21.3s. Estimated remaining: 3h 14m 21s. Rate: 9.5 symbols/min [2025-09-16T17:54:58.983435] [INFO ] [MAIN ] CRBG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:54:58.992209] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T17:54:58.995438] [INFO ] [REMOTE ] CPS: Full fetch complete (28935 records). [2025-09-16T17:54:58.995522] [DEBUG] [VALIDATOR ] CPS: Validating primary data. [2025-09-16T17:54:59.306940] [DEBUG] [LOCAL ] CRBG: Loaded 38714 existing records from primary storage. [2025-09-16T17:54:59.312744] [INFO ] [MAIN ] CRBG: Loaded 38714 existing records. [2025-09-16T17:54:59.503951] [DEBUG] [VALIDATOR ] CPS: All expected trading days are present. [2025-09-16T17:54:59.504123] [DEBUG] [VALIDATOR ] CPS: Primary data validation passed. [2025-09-16T17:54:59.504833] [INFO ] [MAIN ] CPS: Fetched primary data is valid. [2025-09-16T17:54:59.504919] [DEBUG] [LOCAL ] CPS: Saving primary data. [2025-09-16T17:54:59.537048] [DEBUG] [LOCAL ] CPS: Trimmed 28935 records to 18635 (keeping 150 days of data) [2025-09-16T17:54:59.628376] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-03. [2025-09-16T17:54:59.642991] [WARN ] [MAIN ] CRBG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:54:59.644306] [INFO ] [MAIN ] CRBG: Wiping existing primary data before full refresh. [2025-09-16T17:54:59.672677] [INFO ] [LOCAL ] CRBG: Starting deletion of primary data. [2025-09-16T17:54:59.674430] [DEBUG] [LOCAL ] CRBG: Deleted primary data file. [2025-09-16T17:54:59.678217] [INFO ] [MAIN ] CRBG: Starting full primary refresh. [2025-09-16T17:55:00.075401] [INFO ] [REMOTE ] CRBG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:00.078331] [DEBUG] [LOCAL ] CPS: Primary data saved (18635 records). [2025-09-16T17:55:00.079019] [INFO ] [MAIN ] CPS: Full history saved. [2025-09-16T17:55:00.079135] [INFO ] [MAIN ] Update Progress: 523/2369 symbols completed. CPS took 22.6s. Estimated remaining: 3h 13m 57s. Rate: 9.5 symbols/min [2025-09-16T17:55:00.250309] [INFO ] [MAIN ] CRC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:00.283002] [DEBUG] [LOCAL ] CRC: Loaded 32513 existing records from primary storage. [2025-09-16T17:55:00.283141] [INFO ] [MAIN ] CRC: Loaded 32513 existing records. [2025-09-16T17:55:00.380507] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-02. [2025-09-16T17:55:00.463879] [WARN ] [MAIN ] CRC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:55:00.470506] [INFO ] [MAIN ] CRC: Wiping existing primary data before full refresh. [2025-09-16T17:55:00.470381] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-09-16T17:55:00.470777] [INFO ] [LOCAL ] CRC: Starting deletion of primary data. [2025-09-16T17:55:00.474169] [DEBUG] [LOCAL ] CRC: Deleted primary data file. [2025-09-16T17:55:00.474228] [INFO ] [MAIN ] CRC: Starting full primary refresh. [2025-09-16T17:55:00.631962] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-09-16T17:55:00.879571] [INFO ] [REMOTE ] CRC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:01.174948] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-04. [2025-09-16T17:55:01.797490] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-09-16T17:55:01.975014] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-05. [2025-09-16T17:55:02.455062] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-09-16T17:55:02.624639] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-09-16T17:55:02.774941] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-02. [2025-09-16T17:55:03.574898] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-03. [2025-09-16T17:55:04.379973] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-06. [2025-09-16T17:55:04.385627] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T17:55:05.174901] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-09. [2025-09-16T17:55:05.263828] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-09-16T17:55:05.376576] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T17:55:05.974909] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-03. [2025-09-16T17:55:06.732825] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T17:55:06.737993] [INFO ] [REMOTE ] CPT: Full fetch complete (48329 records). [2025-09-16T17:55:06.738081] [DEBUG] [VALIDATOR ] CPT: Validating primary data. [2025-09-16T17:55:06.780965] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-04. [2025-09-16T17:55:07.119590] [DEBUG] [VALIDATOR ] CPT: All expected trading days are present. [2025-09-16T17:55:07.119685] [DEBUG] [VALIDATOR ] CPT: Primary data validation passed. [2025-09-16T17:55:07.120352] [INFO ] [MAIN ] CPT: Fetched primary data is valid. [2025-09-16T17:55:07.120403] [DEBUG] [LOCAL ] CPT: Saving primary data. [2025-09-16T17:55:07.121413] [DEBUG] [LOCAL ] CPT: Trimmed 48329 records to 32824 (keeping 150 days of data) [2025-09-16T17:55:07.344486] [DEBUG] [LOCAL ] CPT: Primary data saved (32824 records). [2025-09-16T17:55:07.344680] [INFO ] [MAIN ] CPT: Full history saved. [2025-09-16T17:55:07.344811] [INFO ] [MAIN ] Update Progress: 524/2369 symbols completed. CPT took 29.9s. Estimated remaining: 3h 13m 54s. Rate: 9.5 symbols/min [2025-09-16T17:55:07.521324] [INFO ] [MAIN ] CRCL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:07.521638] [DEBUG] [LOCAL ] CRCL: No existing data file found in primary storage. [2025-09-16T17:55:07.521719] [WARN ] [MAIN ] CRCL: No existing data found, performing full refresh instead. [2025-09-16T17:55:07.521929] [INFO ] [MAIN ] CRCL: Wiping existing primary data before full refresh. [2025-09-16T17:55:07.521973] [INFO ] [LOCAL ] CRCL: Starting deletion of primary data. [2025-09-16T17:55:07.522050] [DEBUG] [LOCAL ] CRCL: No primary data file found to delete. [2025-09-16T17:55:07.522096] [INFO ] [MAIN ] CRCL: Starting full primary refresh. [2025-09-16T17:55:07.585944] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-07. [2025-09-16T17:55:07.936675] [INFO ] [REMOTE ] CRCL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:07.939387] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-09-16T17:55:08.360861] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T17:55:08.374867] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-08. [2025-09-16T17:55:08.468535] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T17:55:09.174906] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-05. [2025-09-16T17:55:09.184076] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T17:55:09.974939] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-09. [2025-09-16T17:55:10.709217] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T17:55:10.711091] [INFO ] [REMOTE ] CRBD: Full fetch complete (9932 records). [2025-09-16T17:55:10.711165] [DEBUG] [VALIDATOR ] CRBD: Validating primary data. [2025-09-16T17:55:10.780044] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-02. [2025-09-16T17:55:11.078273] [DEBUG] [VALIDATOR ] CRBD: All expected trading days are present. [2025-09-16T17:55:11.078498] [DEBUG] [VALIDATOR ] CRBD: Primary data validation passed. [2025-09-16T17:55:11.078992] [INFO ] [MAIN ] CRBD: Fetched primary data is valid. [2025-09-16T17:55:11.079090] [DEBUG] [LOCAL ] CRBD: Saving primary data. [2025-09-16T17:55:11.079805] [DEBUG] [LOCAL ] CRBD: Trimmed 9932 records to 6926 (keeping 150 days of data) [2025-09-16T17:55:11.112484] [DEBUG] [REMOTE ] CRCL: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:55:11.122780] [WARN ] [REMOTE ] CRCL: No data returned for month 2025-02 [2025-09-16T17:55:11.128037] [DEBUG] [LOCAL ] CRBD: Primary data saved (6926 records). [2025-09-16T17:55:11.128189] [INFO ] [MAIN ] CRBD: Full history saved. [2025-09-16T17:55:11.128297] [INFO ] [MAIN ] Update Progress: 525/2369 symbols completed. CRBD took 18.3s. Estimated remaining: 3h 13m 39s. Rate: 9.5 symbols/min [2025-09-16T17:55:11.338369] [INFO ] [MAIN ] CRGY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:11.340693] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-09-16T17:55:11.396571] [DEBUG] [LOCAL ] CRGY: Loaded 38598 existing records from primary storage. [2025-09-16T17:55:11.396795] [INFO ] [MAIN ] CRGY: Loaded 38598 existing records. [2025-09-16T17:55:11.577200] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-03. [2025-09-16T17:55:11.577551] [WARN ] [MAIN ] CRGY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:55:11.578744] [INFO ] [MAIN ] CRGY: Wiping existing primary data before full refresh. [2025-09-16T17:55:11.578794] [INFO ] [LOCAL ] CRGY: Starting deletion of primary data. [2025-09-16T17:55:11.579553] [DEBUG] [LOCAL ] CRGY: Deleted primary data file. [2025-09-16T17:55:11.579684] [INFO ] [MAIN ] CRGY: Starting full primary refresh. [2025-09-16T17:55:11.884413] [DEBUG] [REMOTE ] CRCL: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:55:11.890471] [WARN ] [REMOTE ] CRCL: No data returned for month 2025-03 [2025-09-16T17:55:11.962085] [INFO ] [REMOTE ] CRGY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:12.374917] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-04. [2025-09-16T17:55:13.174906] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-02. [2025-09-16T17:55:13.876609] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T17:55:13.974852] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-05. [2025-09-16T17:55:14.774888] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-04. [2025-09-16T17:55:14.986332] [DEBUG] [REMOTE ] CRCL: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:55:14.986704] [WARN ] [REMOTE ] CRCL: No data returned for month 2025-04 [2025-09-16T17:55:15.142245] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-09-16T17:55:15.338598] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-09-16T17:55:15.574890] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-05. [2025-09-16T17:55:15.805084] [DEBUG] [REMOTE ] CRCL: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:55:15.805378] [WARN ] [REMOTE ] CRCL: No data returned for month 2025-05 [2025-09-16T17:55:16.374897] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-06. [2025-09-16T17:55:17.174891] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-06. [2025-09-16T17:55:17.961917] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T17:55:17.974851] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-07. [2025-09-16T17:55:18.774937] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-03. [2025-09-16T17:55:19.276191] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-09-16T17:55:19.574947] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-06. [2025-09-16T17:55:19.658506] [INFO ] [REMOTE ] CRCL: Incremental fetch complete for 2025-06 (2.48 seconds). [2025-09-16T17:55:20.374872] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-08. [2025-09-16T17:55:20.430212] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-09-16T17:55:20.803975] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T17:55:21.174892] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-07. [2025-09-16T17:55:21.974889] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-04. [2025-09-16T17:55:22.132315] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-09-16T17:55:22.774907] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-09. [2025-09-16T17:55:23.574881] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-07. [2025-09-16T17:55:23.696674] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T17:55:24.011206] [INFO ] [REMOTE ] CRCL: Incremental fetch complete for 2025-07 (2.84 seconds). [2025-09-16T17:55:24.374914] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-08. [2025-09-16T17:55:24.429125] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-09-16T17:55:24.435883] [INFO ] [REMOTE ] CRBG: Full fetch complete (56922 records). [2025-09-16T17:55:24.436040] [DEBUG] [VALIDATOR ] CRBG: Validating primary data. [2025-09-16T17:55:24.822538] [DEBUG] [VALIDATOR ] CRBG: All expected trading days are present. [2025-09-16T17:55:24.822618] [DEBUG] [VALIDATOR ] CRBG: Primary data validation passed. [2025-09-16T17:55:24.823368] [INFO ] [MAIN ] CRBG: Fetched primary data is valid. [2025-09-16T17:55:24.823423] [DEBUG] [LOCAL ] CRBG: Saving primary data. [2025-09-16T17:55:24.824594] [DEBUG] [LOCAL ] CRBG: Trimmed 56922 records to 39103 (keeping 150 days of data) [2025-09-16T17:55:25.060177] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T17:55:25.100418] [DEBUG] [LOCAL ] CRBG: Primary data saved (39103 records). [2025-09-16T17:55:25.100565] [INFO ] [MAIN ] CRBG: Full history saved. [2025-09-16T17:55:25.100686] [INFO ] [MAIN ] Update Progress: 526/2369 symbols completed. CRBG took 26.3s. Estimated remaining: 3h 13m 60s. Rate: 9.5 symbols/min [2025-09-16T17:55:25.179962] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-08. [2025-09-16T17:55:25.276120] [INFO ] [MAIN ] CRH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:25.313048] [DEBUG] [LOCAL ] CRH: Loaded 39445 existing records from primary storage. [2025-09-16T17:55:25.313213] [INFO ] [MAIN ] CRH: Loaded 39445 existing records. [2025-09-16T17:55:25.489627] [WARN ] [MAIN ] CRH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:55:25.489915] [INFO ] [MAIN ] CRH: Wiping existing primary data before full refresh. [2025-09-16T17:55:25.489963] [INFO ] [LOCAL ] CRH: Starting deletion of primary data. [2025-09-16T17:55:25.490846] [DEBUG] [LOCAL ] CRH: Deleted primary data file. [2025-09-16T17:55:25.490897] [INFO ] [MAIN ] CRH: Starting full primary refresh. [2025-09-16T17:55:25.843513] [INFO ] [REMOTE ] CRH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:25.974926] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-05. [2025-09-16T17:55:26.774932] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-02. [2025-09-16T17:55:26.825436] [INFO ] [REMOTE ] CRCL: Incremental fetch complete for 2025-08 (2.45 seconds). [2025-09-16T17:55:27.040110] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-09-16T17:55:27.451761] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T17:55:27.574902] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-09. [2025-09-16T17:55:28.374881] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-09. [2025-09-16T17:55:28.895101] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-02 (2.12 seconds). [2025-09-16T17:55:28.919519] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T17:55:28.925075] [INFO ] [REMOTE ] CRC: Full fetch complete (47864 records). [2025-09-16T17:55:28.925178] [DEBUG] [VALIDATOR ] CRC: Validating primary data. [2025-09-16T17:55:29.180742] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-06. [2025-09-16T17:55:29.305346] [DEBUG] [VALIDATOR ] CRC: All expected trading days are present. [2025-09-16T17:55:29.305426] [DEBUG] [VALIDATOR ] CRC: Primary data validation passed. [2025-09-16T17:55:29.306025] [INFO ] [MAIN ] CRC: Fetched primary data is valid. [2025-09-16T17:55:29.306069] [DEBUG] [LOCAL ] CRC: Saving primary data. [2025-09-16T17:55:29.307024] [DEBUG] [LOCAL ] CRC: Trimmed 47864 records to 32855 (keeping 150 days of data) [2025-09-16T17:55:29.507522] [DEBUG] [LOCAL ] CRC: Primary data saved (32855 records). [2025-09-16T17:55:29.507676] [INFO ] [MAIN ] CRC: Full history saved. [2025-09-16T17:55:29.507762] [INFO ] [MAIN ] Update Progress: 527/2369 symbols completed. CRC took 29.4s. Estimated remaining: 3h 13m 47s. Rate: 9.5 symbols/min [2025-09-16T17:55:29.680450] [INFO ] [MAIN ] CRI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:29.714308] [DEBUG] [LOCAL ] CRI: Loaded 36947 existing records from primary storage. [2025-09-16T17:55:29.714447] [INFO ] [MAIN ] CRI: Loaded 36947 existing records. [2025-09-16T17:55:29.889403] [WARN ] [MAIN ] CRI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:55:29.889662] [INFO ] [MAIN ] CRI: Wiping existing primary data before full refresh. [2025-09-16T17:55:29.889712] [INFO ] [LOCAL ] CRI: Starting deletion of primary data. [2025-09-16T17:55:29.890522] [DEBUG] [LOCAL ] CRI: Deleted primary data file. [2025-09-16T17:55:29.890570] [INFO ] [MAIN ] CRI: Starting full primary refresh. [2025-09-16T17:55:29.981201] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-03. [2025-09-16T17:55:30.247381] [INFO ] [REMOTE ] CRI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:30.774924] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-02. [2025-09-16T17:55:31.018920] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-09-16T17:55:31.140856] [INFO ] [REMOTE ] CRCL: Incremental fetch complete for 2025-09 (2.77 seconds). [2025-09-16T17:55:31.147276] [INFO ] [REMOTE ] CRCL: Full fetch complete (27489 records). [2025-09-16T17:55:31.147383] [DEBUG] [VALIDATOR ] CRCL: Validating primary data. [2025-09-16T17:55:31.335943] [WARN ] [VALIDATOR ] CRCL: Validation failed - insufficient historical data. Have 71 unique days, need 150 days [2025-09-16T17:55:31.336374] [FATAL] [MAIN ] CRCL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:55:31.336432] [WARN ] [MAIN ] CRCL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:55:31.336483] [INFO ] [LOCAL ] CRCL: Starting deletion of primary data. [2025-09-16T17:55:31.336546] [DEBUG] [LOCAL ] CRCL: No primary data file found to delete. [2025-09-16T17:55:31.336596] [INFO ] [LOCAL ] CRCL: Starting deletion of derivative data. [2025-09-16T17:55:31.336667] [DEBUG] [LOCAL ] CRCL: No derivative data file found to delete. [2025-09-16T17:55:31.336740] [INFO ] [MAIN ] Update Progress: 528/2369 symbols completed. CRCL took 24.0s. Estimated remaining: 3h 13m 25s. Rate: 9.5 symbols/min [2025-09-16T17:55:31.514185] [INFO ] [MAIN ] CRK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:31.553846] [DEBUG] [LOCAL ] CRK: Loaded 38364 existing records from primary storage. [2025-09-16T17:55:31.554020] [INFO ] [MAIN ] CRK: Loaded 38364 existing records. [2025-09-16T17:55:31.587040] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-07. [2025-09-16T17:55:31.781201] [WARN ] [MAIN ] CRK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:55:31.781613] [INFO ] [MAIN ] CRK: Wiping existing primary data before full refresh. [2025-09-16T17:55:31.781746] [INFO ] [LOCAL ] CRK: Starting deletion of primary data. [2025-09-16T17:55:31.782559] [DEBUG] [LOCAL ] CRK: Deleted primary data file. [2025-09-16T17:55:31.782609] [INFO ] [MAIN ] CRK: Starting full primary refresh. [2025-09-16T17:55:32.174222] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-03 (2.19 seconds). [2025-09-16T17:55:32.178170] [INFO ] [REMOTE ] CRK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:32.374186] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T17:55:32.382656] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-04. [2025-09-16T17:55:33.174910] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-02. [2025-09-16T17:55:33.397316] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T17:55:33.854340] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T17:55:33.974901] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-08. [2025-09-16T17:55:34.710631] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-09-16T17:55:34.774865] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-03. [2025-09-16T17:55:35.452476] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T17:55:35.574908] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-03. [2025-09-16T17:55:36.111388] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-09-16T17:55:36.374899] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-04. [2025-09-16T17:55:37.184068] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-05. [2025-09-16T17:55:37.207260] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T17:55:37.695953] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T17:55:37.974959] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-09. [2025-09-16T17:55:38.774939] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-04. [2025-09-16T17:55:39.464592] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T17:55:39.470985] [INFO ] [REMOTE ] CRGY: Full fetch complete (57050 records). [2025-09-16T17:55:39.471196] [DEBUG] [VALIDATOR ] CRGY: Validating primary data. [2025-09-16T17:55:39.576111] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-05. [2025-09-16T17:55:39.605981] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-05 (2.42 seconds). [2025-09-16T17:55:39.898988] [DEBUG] [VALIDATOR ] CRGY: All expected trading days are present. [2025-09-16T17:55:39.899071] [DEBUG] [VALIDATOR ] CRGY: Primary data validation passed. [2025-09-16T17:55:39.899836] [INFO ] [MAIN ] CRGY: Fetched primary data is valid. [2025-09-16T17:55:39.899882] [DEBUG] [LOCAL ] CRGY: Saving primary data. [2025-09-16T17:55:39.901047] [DEBUG] [LOCAL ] CRGY: Trimmed 57050 records to 38979 (keeping 150 days of data) [2025-09-16T17:55:40.144477] [DEBUG] [LOCAL ] CRGY: Primary data saved (38979 records). [2025-09-16T17:55:40.144665] [INFO ] [MAIN ] CRGY: Full history saved. [2025-09-16T17:55:40.146108] [INFO ] [MAIN ] Update Progress: 529/2369 symbols completed. CRGY took 29.0s. Estimated remaining: 3h 13m 27s. Rate: 9.5 symbols/min [2025-09-16T17:55:40.321604] [INFO ] [MAIN ] CRL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:40.321906] [DEBUG] [LOCAL ] CRL: No existing data file found in primary storage. [2025-09-16T17:55:40.321963] [WARN ] [MAIN ] CRL: No existing data found, performing full refresh instead. [2025-09-16T17:55:40.322244] [INFO ] [MAIN ] CRL: Wiping existing primary data before full refresh. [2025-09-16T17:55:40.322354] [INFO ] [LOCAL ] CRL: Starting deletion of primary data. [2025-09-16T17:55:40.322532] [DEBUG] [LOCAL ] CRL: No primary data file found to delete. [2025-09-16T17:55:40.322587] [INFO ] [MAIN ] CRL: Starting full primary refresh. [2025-09-16T17:55:40.385135] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-06. [2025-09-16T17:55:40.464591] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T17:55:40.727531] [INFO ] [REMOTE ] CRL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:40.868535] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-09-16T17:55:41.174905] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-02. [2025-09-16T17:55:41.974915] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-06. [2025-09-16T17:55:42.124225] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T17:55:42.442661] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T17:55:42.774932] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-05. [2025-09-16T17:55:43.343171] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T17:55:43.574944] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-07. [2025-09-16T17:55:44.224027] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T17:55:44.374886] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-06. [2025-09-16T17:55:45.174958] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-07. [2025-09-16T17:55:45.437848] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-09-16T17:55:45.570041] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-09-16T17:55:45.974948] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-08. [2025-09-16T17:55:46.478545] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-09-16T17:55:46.774929] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-07. [2025-09-16T17:55:47.324725] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-09-16T17:55:47.574898] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-09. [2025-09-16T17:55:48.346440] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T17:55:48.374845] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-08. [2025-09-16T17:55:49.124630] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-09-16T17:55:49.131348] [INFO ] [REMOTE ] CRH: Full fetch complete (58153 records). [2025-09-16T17:55:49.131492] [DEBUG] [VALIDATOR ] CRH: Validating primary data. [2025-09-16T17:55:49.186851] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-08. [2025-09-16T17:55:49.535783] [DEBUG] [VALIDATOR ] CRH: All expected trading days are present. [2025-09-16T17:55:49.535882] [DEBUG] [VALIDATOR ] CRH: Primary data validation passed. [2025-09-16T17:55:49.537065] [INFO ] [MAIN ] CRH: Fetched primary data is valid. [2025-09-16T17:55:49.537130] [DEBUG] [LOCAL ] CRH: Saving primary data. [2025-09-16T17:55:49.538449] [DEBUG] [LOCAL ] CRH: Trimmed 58153 records to 39829 (keeping 150 days of data) [2025-09-16T17:55:49.792490] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T17:55:49.835187] [DEBUG] [LOCAL ] CRH: Primary data saved (39829 records). [2025-09-16T17:55:49.835332] [INFO ] [MAIN ] CRH: Full history saved. [2025-09-16T17:55:49.835418] [INFO ] [MAIN ] Update Progress: 530/2369 symbols completed. CRH took 24.7s. Estimated remaining: 3h 13m 33s. Rate: 9.5 symbols/min [2025-09-16T17:55:49.982626] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-09. [2025-09-16T17:55:50.012935] [INFO ] [MAIN ] CRM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:50.050072] [DEBUG] [LOCAL ] CRM: Loaded 39730 existing records from primary storage. [2025-09-16T17:55:50.050206] [INFO ] [MAIN ] CRM: Loaded 39730 existing records. [2025-09-16T17:55:50.227834] [WARN ] [MAIN ] CRM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:55:50.228086] [INFO ] [MAIN ] CRM: Wiping existing primary data before full refresh. [2025-09-16T17:55:50.228139] [INFO ] [LOCAL ] CRM: Starting deletion of primary data. [2025-09-16T17:55:50.229044] [DEBUG] [LOCAL ] CRM: Deleted primary data file. [2025-09-16T17:55:50.229092] [INFO ] [MAIN ] CRM: Starting full primary refresh. [2025-09-16T17:55:50.599129] [INFO ] [REMOTE ] CRM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:50.631156] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T17:55:50.774951] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-02. [2025-09-16T17:55:51.152669] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-09-16T17:55:51.158081] [INFO ] [REMOTE ] CRI: Full fetch complete (53813 records). [2025-09-16T17:55:51.158183] [DEBUG] [VALIDATOR ] CRI: Validating primary data. [2025-09-16T17:55:51.158780] [FATAL] [VALIDATOR ] CRI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:55:51.158857] [FATAL] [MAIN ] CRI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:55:51.158913] [WARN ] [MAIN ] CRI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:55:51.158987] [INFO ] [LOCAL ] CRI: Starting deletion of primary data. [2025-09-16T17:55:51.159065] [DEBUG] [LOCAL ] CRI: No primary data file found to delete. [2025-09-16T17:55:51.159119] [INFO ] [LOCAL ] CRI: Starting deletion of derivative data. [2025-09-16T17:55:51.159531] [DEBUG] [LOCAL ] CRI: Deleted derivative data file. [2025-09-16T17:55:51.159657] [INFO ] [MAIN ] Update Progress: 531/2369 symbols completed. CRI took 21.7s. Estimated remaining: 3h 13m 9s. Rate: 9.5 symbols/min [2025-09-16T17:55:51.336138] [INFO ] [MAIN ] CRS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:51.370863] [DEBUG] [LOCAL ] CRS: Loaded 32032 existing records from primary storage. [2025-09-16T17:55:51.370999] [INFO ] [MAIN ] CRS: Loaded 32032 existing records. [2025-09-16T17:55:51.551389] [WARN ] [MAIN ] CRS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:55:51.551635] [INFO ] [MAIN ] CRS: Wiping existing primary data before full refresh. [2025-09-16T17:55:51.551700] [INFO ] [LOCAL ] CRS: Starting deletion of primary data. [2025-09-16T17:55:51.552341] [DEBUG] [LOCAL ] CRS: Deleted primary data file. [2025-09-16T17:55:51.552387] [INFO ] [MAIN ] CRS: Starting full primary refresh. [2025-09-16T17:55:51.580018] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-09. [2025-09-16T17:55:51.919094] [INFO ] [REMOTE ] CRS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:52.374893] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-02. [2025-09-16T17:55:52.874674] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-02 (2.10 seconds). [2025-09-16T17:55:53.017440] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T17:55:53.024775] [INFO ] [REMOTE ] CRK: Full fetch complete (56638 records). [2025-09-16T17:55:53.024899] [DEBUG] [VALIDATOR ] CRK: Validating primary data. [2025-09-16T17:55:53.179982] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-03. [2025-09-16T17:55:53.412319] [DEBUG] [VALIDATOR ] CRK: All expected trading days are present. [2025-09-16T17:55:53.412412] [DEBUG] [VALIDATOR ] CRK: Primary data validation passed. [2025-09-16T17:55:53.413114] [INFO ] [MAIN ] CRK: Fetched primary data is valid. [2025-09-16T17:55:53.413185] [DEBUG] [LOCAL ] CRK: Saving primary data. [2025-09-16T17:55:53.414728] [DEBUG] [LOCAL ] CRK: Trimmed 56638 records to 38747 (keeping 150 days of data) [2025-09-16T17:55:53.644750] [DEBUG] [LOCAL ] CRK: Primary data saved (38747 records). [2025-09-16T17:55:53.644912] [INFO ] [MAIN ] CRK: Full history saved. [2025-09-16T17:55:53.645893] [INFO ] [MAIN ] Update Progress: 532/2369 symbols completed. CRK took 22.3s. Estimated remaining: 3h 12m 50s. Rate: 9.5 symbols/min [2025-09-16T17:55:53.818983] [INFO ] [MAIN ] CRT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:55:53.826086] [DEBUG] [LOCAL ] CRT: Loaded 3442 existing records from primary storage. [2025-09-16T17:55:53.826212] [INFO ] [MAIN ] CRT: Loaded 3442 existing records. [2025-09-16T17:55:53.951019] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T17:55:53.984477] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-03. [2025-09-16T17:55:54.037630] [WARN ] [MAIN ] CRT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:55:54.037880] [INFO ] [MAIN ] CRT: Wiping existing primary data before full refresh. [2025-09-16T17:55:54.037928] [INFO ] [LOCAL ] CRT: Starting deletion of primary data. [2025-09-16T17:55:54.038140] [DEBUG] [LOCAL ] CRT: Deleted primary data file. [2025-09-16T17:55:54.038197] [INFO ] [MAIN ] CRT: Starting full primary refresh. [2025-09-16T17:55:54.390841] [INFO ] [REMOTE ] CRT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:55:54.774894] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-03. [2025-09-16T17:55:54.821885] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T17:55:55.574917] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-04. [2025-09-16T17:55:55.889032] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-09-16T17:55:56.102311] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T17:55:56.374913] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-04. [2025-09-16T17:55:57.022035] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T17:55:57.174909] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-05. [2025-09-16T17:55:57.974917] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-04. [2025-09-16T17:55:58.420992] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T17:55:58.670428] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-04 (2.30 seconds). [2025-09-16T17:55:58.774848] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-05. [2025-09-16T17:55:59.485292] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T17:55:59.574953] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-02. [2025-09-16T17:56:00.302725] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-09-16T17:56:00.374951] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-03. [2025-09-16T17:56:01.178691] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-05. [2025-09-16T17:56:01.179478] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T17:56:01.597439] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-05 (2.82 seconds). [2025-09-16T17:56:01.974924] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-06. [2025-09-16T17:56:02.775226] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-09-16T17:56:02.775292] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-06. [2025-09-16T17:56:03.574907] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-04. [2025-09-16T17:56:04.385102] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-06. [2025-09-16T17:56:04.390237] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-06 (2.42 seconds). [2025-09-16T17:56:04.523767] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-09-16T17:56:04.564111] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-09-16T17:56:05.174913] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-07. [2025-09-16T17:56:05.974900] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-07. [2025-09-16T17:56:06.285848] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-09-16T17:56:06.774899] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-07. [2025-09-16T17:56:07.459612] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-07 (2.28 seconds). [2025-09-16T17:56:07.574947] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-05. [2025-09-16T17:56:07.860607] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T17:56:08.157042] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-09-16T17:56:08.374882] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-06. [2025-09-16T17:56:08.462628] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T17:56:08.979214] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-09-16T17:56:09.174956] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-08. [2025-09-16T17:56:09.974922] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-07. [2025-09-16T17:56:10.748280] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T17:56:10.774947] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-08. [2025-09-16T17:56:11.542620] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-08 (2.37 seconds). [2025-09-16T17:56:11.574905] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-08. [2025-09-16T17:56:11.996290] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-09-16T17:56:12.221315] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T17:56:12.374907] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-09. [2025-09-16T17:56:13.174886] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-08. [2025-09-16T17:56:13.326714] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T17:56:13.328037] [INFO ] [REMOTE ] CRT: Full fetch complete (5425 records). [2025-09-16T17:56:13.328106] [DEBUG] [VALIDATOR ] CRT: Validating primary data. [2025-09-16T17:56:13.710408] [DEBUG] [VALIDATOR ] CRT: All expected trading days are present. [2025-09-16T17:56:13.710521] [DEBUG] [VALIDATOR ] CRT: Primary data validation passed. [2025-09-16T17:56:13.710714] [INFO ] [MAIN ] CRT: Fetched primary data is valid. [2025-09-16T17:56:13.710760] [DEBUG] [LOCAL ] CRT: Saving primary data. [2025-09-16T17:56:13.711157] [DEBUG] [LOCAL ] CRT: Trimmed 5425 records to 3500 (keeping 150 days of data) [2025-09-16T17:56:13.733186] [DEBUG] [LOCAL ] CRT: Primary data saved (3500 records). [2025-09-16T17:56:13.733293] [INFO ] [MAIN ] CRT: Full history saved. [2025-09-16T17:56:13.733385] [INFO ] [MAIN ] Update Progress: 533/2369 symbols completed. CRT took 20.1s. Estimated remaining: 3h 13m 31s. Rate: 9.5 symbols/min [2025-09-16T17:56:13.914727] [INFO ] [MAIN ] CSAN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:56:13.949003] [DEBUG] [LOCAL ] CSAN: Loaded 31174 existing records from primary storage. [2025-09-16T17:56:13.949145] [INFO ] [MAIN ] CSAN: Loaded 31174 existing records. [2025-09-16T17:56:13.982724] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-09. [2025-09-16T17:56:14.139359] [WARN ] [MAIN ] CSAN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:56:14.139603] [INFO ] [MAIN ] CSAN: Wiping existing primary data before full refresh. [2025-09-16T17:56:14.139675] [INFO ] [LOCAL ] CSAN: Starting deletion of primary data. [2025-09-16T17:56:14.140394] [DEBUG] [LOCAL ] CSAN: Deleted primary data file. [2025-09-16T17:56:14.140445] [INFO ] [MAIN ] CSAN: Starting full primary refresh. [2025-09-16T17:56:14.526049] [INFO ] [REMOTE ] CSAN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:56:14.629163] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T17:56:14.774858] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-09. [2025-09-16T17:56:15.574934] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-09. [2025-09-16T17:56:16.068502] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-09-16T17:56:16.078735] [INFO ] [REMOTE ] CRM: Full fetch complete (58058 records). [2025-09-16T17:56:16.078955] [DEBUG] [VALIDATOR ] CRM: Validating primary data. [2025-09-16T17:56:16.082009] [FATAL] [VALIDATOR ] CRM: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:56:16.082150] [FATAL] [MAIN ] CRM: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:56:16.082203] [WARN ] [MAIN ] CRM: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:56:16.082276] [INFO ] [LOCAL ] CRM: Starting deletion of primary data. [2025-09-16T17:56:16.082379] [DEBUG] [LOCAL ] CRM: No primary data file found to delete. [2025-09-16T17:56:16.082434] [INFO ] [LOCAL ] CRM: Starting deletion of derivative data. [2025-09-16T17:56:16.082792] [DEBUG] [LOCAL ] CRM: Deleted derivative data file. [2025-09-16T17:56:16.082952] [INFO ] [MAIN ] Update Progress: 534/2369 symbols completed. CRM took 26.2s. Estimated remaining: 3h 13m 11s. Rate: 9.5 symbols/min [2025-09-16T17:56:16.274432] [INFO ] [MAIN ] CSL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:56:16.298117] [DEBUG] [LOCAL ] CSL: Loaded 22192 existing records from primary storage. [2025-09-16T17:56:16.298283] [INFO ] [MAIN ] CSL: Loaded 22192 existing records. [2025-09-16T17:56:16.379960] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-02. [2025-09-16T17:56:16.486820] [WARN ] [MAIN ] CSL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:56:16.487192] [INFO ] [MAIN ] CSL: Wiping existing primary data before full refresh. [2025-09-16T17:56:16.487305] [INFO ] [LOCAL ] CSL: Starting deletion of primary data. [2025-09-16T17:56:16.493013] [DEBUG] [LOCAL ] CSL: Deleted primary data file. [2025-09-16T17:56:16.499301] [INFO ] [MAIN ] CSL: Starting full primary refresh. [2025-09-16T17:56:16.560344] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-09-16T17:56:16.614129] [INFO ] [REMOTE ] CRS: Full fetch complete (47441 records). [2025-09-16T17:56:16.634992] [DEBUG] [VALIDATOR ] CRS: Validating primary data. [2025-09-16T17:56:17.123448] [INFO ] [REMOTE ] CSL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:56:17.184444] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-02. [2025-09-16T17:56:17.287504] [DEBUG] [VALIDATOR ] CRS: All expected trading days are present. [2025-09-16T17:56:17.287667] [DEBUG] [VALIDATOR ] CRS: Primary data validation passed. [2025-09-16T17:56:17.288320] [INFO ] [MAIN ] CRS: Fetched primary data is valid. [2025-09-16T17:56:17.288421] [DEBUG] [LOCAL ] CRS: Saving primary data. [2025-09-16T17:56:17.303083] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-09-16T17:56:17.308288] [DEBUG] [LOCAL ] CRS: Trimmed 47441 records to 32329 (keeping 150 days of data) [2025-09-16T17:56:17.309620] [INFO ] [REMOTE ] CRL: Full fetch complete (48230 records). [2025-09-16T17:56:17.309735] [DEBUG] [VALIDATOR ] CRL: Validating primary data. [2025-09-16T17:56:17.807362] [DEBUG] [VALIDATOR ] CRL: All expected trading days are present. [2025-09-16T17:56:17.807900] [DEBUG] [VALIDATOR ] CRL: Primary data validation passed. [2025-09-16T17:56:17.809269] [INFO ] [MAIN ] CRL: Fetched primary data is valid. [2025-09-16T17:56:17.809626] [DEBUG] [LOCAL ] CRL: Saving primary data. [2025-09-16T17:56:17.813047] [DEBUG] [LOCAL ] CRL: Trimmed 48230 records to 31389 (keeping 150 days of data) [2025-09-16T17:56:18.122417] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T17:56:18.135091] [DEBUG] [LOCAL ] CRS: Primary data saved (32329 records). [2025-09-16T17:56:18.134758] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-03. [2025-09-16T17:56:18.137539] [INFO ] [MAIN ] CRS: Full history saved. [2025-09-16T17:56:18.138371] [INFO ] [MAIN ] Update Progress: 535/2369 symbols completed. CRS took 27.0s. Estimated remaining: 3h 12m 50s. Rate: 9.5 symbols/min [2025-09-16T17:56:18.187450] [DEBUG] [LOCAL ] CRL: Primary data saved (31389 records). [2025-09-16T17:56:18.212370] [INFO ] [MAIN ] CRL: Full history saved. [2025-09-16T17:56:18.241286] [INFO ] [MAIN ] Update Progress: 536/2369 symbols completed. CRL took 38.1s. Estimated remaining: 3h 12m 22s. Rate: 9.5 symbols/min [2025-09-16T17:56:18.408232] [INFO ] [MAIN ] CSR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:56:18.513443] [INFO ] [MAIN ] CSTM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:56:18.517789] [DEBUG] [LOCAL ] CSR: Loaded 9476 existing records from primary storage. [2025-09-16T17:56:18.517960] [INFO ] [MAIN ] CSR: Loaded 9476 existing records. [2025-09-16T17:56:18.661191] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T17:56:18.742682] [WARN ] [MAIN ] CSR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:56:18.752991] [INFO ] [MAIN ] CSR: Wiping existing primary data before full refresh. [2025-09-16T17:56:18.753062] [INFO ] [LOCAL ] CSR: Starting deletion of primary data. [2025-09-16T17:56:18.753542] [DEBUG] [LOCAL ] CSTM: Loaded 34900 existing records from primary storage. [2025-09-16T17:56:18.753664] [INFO ] [MAIN ] CSTM: Loaded 34900 existing records. [2025-09-16T17:56:18.783977] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-03. [2025-09-16T17:56:18.789545] [DEBUG] [LOCAL ] CSR: Deleted primary data file. [2025-09-16T17:56:18.816919] [INFO ] [MAIN ] CSR: Starting full primary refresh. [2025-09-16T17:56:19.106393] [WARN ] [MAIN ] CSTM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:56:19.204988] [INFO ] [MAIN ] CSTM: Wiping existing primary data before full refresh. [2025-09-16T17:56:19.216922] [INFO ] [LOCAL ] CSTM: Starting deletion of primary data. [2025-09-16T17:56:19.243110] [DEBUG] [LOCAL ] CSTM: Deleted primary data file. [2025-09-16T17:56:19.243206] [INFO ] [MAIN ] CSTM: Starting full primary refresh. [2025-09-16T17:56:19.441149] [INFO ] [REMOTE ] CSR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:56:19.579098] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-02. [2025-09-16T17:56:19.682827] [INFO ] [REMOTE ] CSTM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:56:19.707111] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-09-16T17:56:20.184498] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T17:56:20.257028] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-09-16T17:56:20.374905] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-03. [2025-09-16T17:56:21.174905] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-02. [2025-09-16T17:56:21.285979] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-09-16T17:56:21.974899] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-04. [2025-09-16T17:56:22.774903] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-04. [2025-09-16T17:56:22.968945] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-09-16T17:56:23.352317] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-09-16T17:56:23.574920] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-03. [2025-09-16T17:56:23.718133] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-09-16T17:56:24.374947] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-05. [2025-09-16T17:56:25.113681] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-09-16T17:56:25.174890] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-04. [2025-09-16T17:56:25.693414] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T17:56:25.974901] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-05. [2025-09-16T17:56:26.774883] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-04. [2025-09-16T17:56:26.834572] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T17:56:26.853051] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-09-16T17:56:27.574925] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-06. [2025-09-16T17:56:28.208490] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-09-16T17:56:28.241301] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T17:56:28.374905] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-06. [2025-09-16T17:56:29.174907] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-05. [2025-09-16T17:56:29.785022] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T17:56:29.974921] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-07. [2025-09-16T17:56:30.699602] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T17:56:30.774878] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-06. [2025-09-16T17:56:31.441182] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T17:56:31.574896] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-08. [2025-09-16T17:56:32.374929] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-07. [2025-09-16T17:56:32.414166] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T17:56:32.970327] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-09-16T17:56:33.167868] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T17:56:33.174834] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-08. [2025-09-16T17:56:33.974887] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-09. [2025-09-16T17:56:34.051499] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-09-16T17:56:34.774914] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-09. [2025-09-16T17:56:35.318306] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T17:56:35.322583] [INFO ] [REMOTE ] CSL: Full fetch complete (34290 records). [2025-09-16T17:56:35.322687] [DEBUG] [VALIDATOR ] CSL: Validating primary data. [2025-09-16T17:56:35.587766] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-05. [2025-09-16T17:56:35.721812] [DEBUG] [VALIDATOR ] CSL: All expected trading days are present. [2025-09-16T17:56:35.722096] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T17:56:35.722283] [DEBUG] [VALIDATOR ] CSL: Primary data validation passed. [2025-09-16T17:56:35.724617] [INFO ] [MAIN ] CSL: Fetched primary data is valid. [2025-09-16T17:56:35.725102] [DEBUG] [LOCAL ] CSL: Saving primary data. [2025-09-16T17:56:35.725194] [INFO ] [REMOTE ] CSR: Full fetch complete (13233 records). [2025-09-16T17:56:35.725420] [DEBUG] [VALIDATOR ] CSR: Validating primary data. [2025-09-16T17:56:35.824754] [DEBUG] [LOCAL ] CSL: Trimmed 34290 records to 22516 (keeping 150 days of data) [2025-09-16T17:56:36.194885] [DEBUG] [VALIDATOR ] CSR: All expected trading days are present. [2025-09-16T17:56:36.196575] [DEBUG] [VALIDATOR ] CSR: Primary data validation passed. [2025-09-16T17:56:36.198114] [INFO ] [MAIN ] CSR: Fetched primary data is valid. [2025-09-16T17:56:36.198173] [DEBUG] [LOCAL ] CSR: Saving primary data. [2025-09-16T17:56:36.203987] [DEBUG] [LOCAL ] CSR: Trimmed 13233 records to 9535 (keeping 150 days of data) [2025-09-16T17:56:36.279137] [DEBUG] [LOCAL ] CSR: Primary data saved (9535 records). [2025-09-16T17:56:36.279317] [INFO ] [MAIN ] CSR: Full history saved. [2025-09-16T17:56:36.286082] [INFO ] [MAIN ] Update Progress: 537/2369 symbols completed. CSR took 18.1s. Estimated remaining: 3h 12m 56s. Rate: 9.5 symbols/min [2025-09-16T17:56:36.381214] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-07. [2025-09-16T17:56:36.375967] [DEBUG] [LOCAL ] CSL: Primary data saved (22516 records). [2025-09-16T17:56:36.386805] [INFO ] [MAIN ] CSL: Full history saved. [2025-09-16T17:56:36.386946] [INFO ] [MAIN ] Update Progress: 538/2369 symbols completed. CSL took 20.3s. Estimated remaining: 3h 12m 29s. Rate: 9.5 symbols/min [2025-09-16T17:56:36.590043] [INFO ] [MAIN ] CSV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:56:36.650012] [INFO ] [MAIN ] CSW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:56:36.652522] [DEBUG] [LOCAL ] CSW: No existing data file found in primary storage. [2025-09-16T17:56:36.652611] [WARN ] [MAIN ] CSW: No existing data found, performing full refresh instead. [2025-09-16T17:56:36.652764] [INFO ] [MAIN ] CSW: Wiping existing primary data before full refresh. [2025-09-16T17:56:36.652846] [INFO ] [LOCAL ] CSW: Starting deletion of primary data. [2025-09-16T17:56:36.652896] [DEBUG] [LOCAL ] CSW: No primary data file found to delete. [2025-09-16T17:56:36.652932] [INFO ] [MAIN ] CSW: Starting full primary refresh. [2025-09-16T17:56:36.758831] [DEBUG] [LOCAL ] CSV: Loaded 8876 existing records from primary storage. [2025-09-16T17:56:36.758989] [INFO ] [MAIN ] CSV: Loaded 8876 existing records. [2025-09-16T17:56:37.120436] [WARN ] [MAIN ] CSV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:56:37.124089] [INFO ] [MAIN ] CSV: Wiping existing primary data before full refresh. [2025-09-16T17:56:37.124169] [INFO ] [LOCAL ] CSV: Starting deletion of primary data. [2025-09-16T17:56:37.140085] [DEBUG] [LOCAL ] CSV: Deleted primary data file. [2025-09-16T17:56:37.164373] [INFO ] [MAIN ] CSV: Starting full primary refresh. [2025-09-16T17:56:37.222548] [INFO ] [REMOTE ] CSW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:56:37.258529] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-02. [2025-09-16T17:56:37.537255] [DEBUG] [REMOTE ] CSW: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:56:37.587591] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-09-16T17:56:37.590973] [WARN ] [REMOTE ] CSW: No data returned for month 2025-02 [2025-09-16T17:56:37.590851] [INFO ] [REMOTE ] CSV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:56:37.974902] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-02. [2025-09-16T17:56:38.231295] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T17:56:38.774885] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-03. [2025-09-16T17:56:38.969370] [DEBUG] [REMOTE ] CSW: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:56:38.969697] [WARN ] [REMOTE ] CSW: No data returned for month 2025-03 [2025-09-16T17:56:39.034359] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T17:56:39.574919] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-08. [2025-09-16T17:56:40.374934] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-06. [2025-09-16T17:56:41.176674] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-04. [2025-09-16T17:56:41.182963] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-09-16T17:56:41.384241] [DEBUG] [REMOTE ] CSW: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:56:41.384553] [WARN ] [REMOTE ] CSW: No data returned for month 2025-04 [2025-09-16T17:56:41.961526] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T17:56:41.974848] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-07. [2025-09-16T17:56:42.774933] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-09. [2025-09-16T17:56:43.420006] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T17:56:43.574895] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-05. [2025-09-16T17:56:43.791255] [DEBUG] [REMOTE ] CSW: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:56:43.791554] [WARN ] [REMOTE ] CSW: No data returned for month 2025-05 [2025-09-16T17:56:44.374933] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-06. [2025-09-16T17:56:44.473267] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T17:56:44.481404] [INFO ] [REMOTE ] CSTM: Full fetch complete (52206 records). [2025-09-16T17:56:44.481572] [DEBUG] [VALIDATOR ] CSTM: Validating primary data. [2025-09-16T17:56:44.878544] [DEBUG] [VALIDATOR ] CSTM: All expected trading days are present. [2025-09-16T17:56:44.878625] [DEBUG] [VALIDATOR ] CSTM: Primary data validation passed. [2025-09-16T17:56:44.879282] [INFO ] [MAIN ] CSTM: Fetched primary data is valid. [2025-09-16T17:56:44.879327] [DEBUG] [LOCAL ] CSTM: Saving primary data. [2025-09-16T17:56:44.880385] [DEBUG] [LOCAL ] CSTM: Trimmed 52206 records to 35217 (keeping 150 days of data) [2025-09-16T17:56:44.977625] [INFO ] [REMOTE ] CSW: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-09-16T17:56:45.096681] [DEBUG] [LOCAL ] CSTM: Primary data saved (35217 records). [2025-09-16T17:56:45.096830] [INFO ] [MAIN ] CSTM: Full history saved. [2025-09-16T17:56:45.097713] [INFO ] [MAIN ] Update Progress: 539/2369 symbols completed. CSTM took 26.8s. Estimated remaining: 3h 12m 30s. Rate: 9.5 symbols/min [2025-09-16T17:56:45.180008] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-08. [2025-09-16T17:56:45.287727] [INFO ] [MAIN ] CTBB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:56:45.297020] [DEBUG] [LOCAL ] CTBB: Loaded 4787 existing records from primary storage. [2025-09-16T17:56:45.297178] [INFO ] [MAIN ] CTBB: Loaded 4787 existing records. [2025-09-16T17:56:45.469999] [WARN ] [MAIN ] CTBB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:56:45.470232] [INFO ] [MAIN ] CTBB: Wiping existing primary data before full refresh. [2025-09-16T17:56:45.470278] [INFO ] [LOCAL ] CTBB: Starting deletion of primary data. [2025-09-16T17:56:45.470517] [DEBUG] [LOCAL ] CTBB: Deleted primary data file. [2025-09-16T17:56:45.470560] [INFO ] [MAIN ] CTBB: Starting full primary refresh. [2025-09-16T17:56:45.817802] [INFO ] [REMOTE ] CTBB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:56:45.974900] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-07. [2025-09-16T17:56:46.774940] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-03. [2025-09-16T17:56:46.891825] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-09-16T17:56:46.938146] [INFO ] [REMOTE ] CSW: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-09-16T17:56:47.574924] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-02. [2025-09-16T17:56:47.588300] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-09-16T17:56:48.265083] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T17:56:48.374874] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-08. [2025-09-16T17:56:49.174895] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-04. [2025-09-16T17:56:49.314836] [INFO ] [REMOTE ] CSW: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-09-16T17:56:49.974875] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-09. [2025-09-16T17:56:50.051661] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-09-16T17:56:50.707633] [INFO ] [REMOTE ] CSW: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T17:56:50.709967] [INFO ] [REMOTE ] CSW: Full fetch complete (7329 records). [2025-09-16T17:56:50.710036] [DEBUG] [VALIDATOR ] CSW: Validating primary data. [2025-09-16T17:56:50.780014] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-03. [2025-09-16T17:56:50.893005] [WARN ] [VALIDATOR ] CSW: Validation failed - insufficient historical data. Have 69 unique days, need 150 days [2025-09-16T17:56:50.893213] [FATAL] [MAIN ] CSW: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:56:50.893265] [WARN ] [MAIN ] CSW: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:56:50.893314] [INFO ] [LOCAL ] CSW: Starting deletion of primary data. [2025-09-16T17:56:50.893371] [DEBUG] [LOCAL ] CSW: No primary data file found to delete. [2025-09-16T17:56:50.893408] [INFO ] [LOCAL ] CSW: Starting deletion of derivative data. [2025-09-16T17:56:50.893450] [DEBUG] [LOCAL ] CSW: No derivative data file found to delete. [2025-09-16T17:56:50.893508] [INFO ] [MAIN ] Update Progress: 540/2369 symbols completed. CSW took 14.5s. Estimated remaining: 3h 12m 22s. Rate: 9.5 symbols/min [2025-09-16T17:56:51.063871] [INFO ] [MAIN ] CTDD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:56:51.069089] [DEBUG] [LOCAL ] CTDD: Loaded 3783 existing records from primary storage. [2025-09-16T17:56:51.069198] [INFO ] [MAIN ] CTDD: Loaded 3783 existing records. [2025-09-16T17:56:51.246369] [WARN ] [MAIN ] CTDD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:56:51.246621] [INFO ] [MAIN ] CTDD: Wiping existing primary data before full refresh. [2025-09-16T17:56:51.246696] [INFO ] [LOCAL ] CTDD: Starting deletion of primary data. [2025-09-16T17:56:51.246972] [DEBUG] [LOCAL ] CTDD: Deleted primary data file. [2025-09-16T17:56:51.247021] [INFO ] [MAIN ] CTDD: Starting full primary refresh. [2025-09-16T17:56:51.597658] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-09. [2025-09-16T17:56:51.602255] [INFO ] [REMOTE ] CTDD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:56:51.645017] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-09-16T17:56:52.374905] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-05. [2025-09-16T17:56:53.174869] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-02. [2025-09-16T17:56:53.204210] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T17:56:53.220970] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T17:56:53.226082] [INFO ] [REMOTE ] CSAN: Full fetch complete (45489 records). [2025-09-16T17:56:53.226167] [DEBUG] [VALIDATOR ] CSAN: Validating primary data. [2025-09-16T17:56:53.603483] [DEBUG] [VALIDATOR ] CSAN: All expected trading days are present. [2025-09-16T17:56:53.603601] [DEBUG] [VALIDATOR ] CSAN: Primary data validation passed. [2025-09-16T17:56:53.604203] [INFO ] [MAIN ] CSAN: Fetched primary data is valid. [2025-09-16T17:56:53.604250] [DEBUG] [LOCAL ] CSAN: Saving primary data. [2025-09-16T17:56:53.605159] [DEBUG] [LOCAL ] CSAN: Trimmed 45489 records to 31504 (keeping 150 days of data) [2025-09-16T17:56:53.788547] [DEBUG] [LOCAL ] CSAN: Primary data saved (31504 records). [2025-09-16T17:56:53.788702] [INFO ] [MAIN ] CSAN: Full history saved. [2025-09-16T17:56:53.788806] [INFO ] [MAIN ] Update Progress: 541/2369 symbols completed. CSAN took 40.1s. Estimated remaining: 3h 12m 5s. Rate: 9.5 symbols/min [2025-09-16T17:56:53.959504] [INFO ] [MAIN ] CTEST: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:56:53.959813] [DEBUG] [LOCAL ] CTEST: No existing data file found in primary storage. [2025-09-16T17:56:53.964130] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T17:56:53.963306] [WARN ] [MAIN ] CTEST: No existing data found, performing full refresh instead. [2025-09-16T17:56:53.966094] [INFO ] [MAIN ] CTEST: Wiping existing primary data before full refresh. [2025-09-16T17:56:53.966171] [INFO ] [LOCAL ] CTEST: Starting deletion of primary data. [2025-09-16T17:56:53.966218] [DEBUG] [LOCAL ] CTEST: No primary data file found to delete. [2025-09-16T17:56:53.966251] [INFO ] [MAIN ] CTEST: Starting full primary refresh. [2025-09-16T17:56:53.979936] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-04. [2025-09-16T17:56:54.331404] [INFO ] [REMOTE ] CTEST: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:56:54.774895] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-03. [2025-09-16T17:56:54.890698] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-09-16T17:56:55.326963] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-09-16T17:56:55.574895] [INFO ] [REMOTE ] CTEST: Fetching primary data for 2025-02. [2025-09-16T17:56:55.891120] [DEBUG] [REMOTE ] CTEST: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:56:55.891455] [WARN ] [REMOTE ] CTEST: No data returned for month 2025-02 [2025-09-16T17:56:56.374896] [INFO ] [REMOTE ] CTEST: Fetching primary data for 2025-03. [2025-09-16T17:56:56.575740] [DEBUG] [REMOTE ] CTEST: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:56:56.575984] [WARN ] [REMOTE ] CTEST: No data returned for month 2025-03 [2025-09-16T17:56:57.174936] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-05. [2025-09-16T17:56:57.792212] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-09-16T17:56:57.974947] [INFO ] [REMOTE ] CTEST: Fetching primary data for 2025-04. [2025-09-16T17:56:58.197356] [DEBUG] [REMOTE ] CTEST: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:56:58.197685] [WARN ] [REMOTE ] CTEST: No data returned for month 2025-04 [2025-09-16T17:56:58.774908] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-06. [2025-09-16T17:56:59.390544] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-09-16T17:56:59.574914] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-06. [2025-09-16T17:57:00.374857] [INFO ] [REMOTE ] CTEST: Fetching primary data for 2025-05. [2025-09-16T17:57:00.435312] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-09-16T17:57:01.174923] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-04. [2025-09-16T17:57:01.182755] [INFO ] [REMOTE ] CTEST: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-09-16T17:57:01.803363] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-09-16T17:57:01.974930] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-05. [2025-09-16T17:57:02.536174] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-09-16T17:57:02.774914] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-07. [2025-09-16T17:57:03.395017] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-09-16T17:57:03.575113] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-08. [2025-09-16T17:57:04.205166] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-09-16T17:57:04.374990] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-09. [2025-09-16T17:57:05.174837] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T17:57:05.174954] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-06. [2025-09-16T17:57:05.177907] [INFO ] [REMOTE ] CTBB: Full fetch complete (7893 records). [2025-09-16T17:57:05.178013] [DEBUG] [VALIDATOR ] CTBB: Validating primary data. [2025-09-16T17:57:05.556605] [DEBUG] [VALIDATOR ] CTBB: All expected trading days are present. [2025-09-16T17:57:05.556709] [DEBUG] [VALIDATOR ] CTBB: Primary data validation passed. [2025-09-16T17:57:05.556911] [INFO ] [MAIN ] CTBB: Fetched primary data is valid. [2025-09-16T17:57:05.556952] [DEBUG] [LOCAL ] CTBB: Saving primary data. [2025-09-16T17:57:05.557390] [DEBUG] [LOCAL ] CTBB: Trimmed 7893 records to 4822 (keeping 150 days of data) [2025-09-16T17:57:05.590332] [DEBUG] [LOCAL ] CTBB: Primary data saved (4822 records). [2025-09-16T17:57:05.590499] [INFO ] [MAIN ] CTBB: Full history saved. [2025-09-16T17:57:05.590592] [INFO ] [MAIN ] Update Progress: 542/2369 symbols completed. CTBB took 20.5s. Estimated remaining: 3h 12m 17s. Rate: 9.5 symbols/min [2025-09-16T17:57:05.761872] [INFO ] [MAIN ] CTEV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:57:05.762059] [DEBUG] [LOCAL ] CTEV: No existing data file found in primary storage. [2025-09-16T17:57:05.762130] [WARN ] [MAIN ] CTEV: No existing data found, performing full refresh instead. [2025-09-16T17:57:05.762276] [INFO ] [MAIN ] CTEV: Wiping existing primary data before full refresh. [2025-09-16T17:57:05.762316] [INFO ] [LOCAL ] CTEV: Starting deletion of primary data. [2025-09-16T17:57:05.762356] [DEBUG] [LOCAL ] CTEV: No primary data file found to delete. [2025-09-16T17:57:05.762389] [INFO ] [MAIN ] CTEV: Starting full primary refresh. [2025-09-16T17:57:05.980141] [INFO ] [REMOTE ] CTEST: Fetching primary data for 2025-06. [2025-09-16T17:57:06.062073] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T17:57:06.125717] [INFO ] [REMOTE ] CTEV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:57:06.453783] [INFO ] [REMOTE ] CTEST: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-09-16T17:57:06.774890] [INFO ] [REMOTE ] CTEST: Fetching primary data for 2025-07. [2025-09-16T17:57:07.003770] [INFO ] [REMOTE ] CTEST: Incremental fetch complete for 2025-07 (0.23 seconds). [2025-09-16T17:57:07.575094] [INFO ] [REMOTE ] CTEST: Fetching primary data for 2025-08. [2025-09-16T17:57:07.768310] [DEBUG] [REMOTE ] CTEST: Error fetching primary data for 2025-08: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:57:07.768605] [WARN ] [REMOTE ] CTEST: No data returned for month 2025-08 [2025-09-16T17:57:08.374903] [INFO ] [REMOTE ] CTEST: Fetching primary data for 2025-09. [2025-09-16T17:57:08.683058] [DEBUG] [REMOTE ] CTEST: Error fetching primary data for 2025-09: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T17:57:08.683367] [WARN ] [REMOTE ] CTEST: No data returned for month 2025-09 [2025-09-16T17:57:08.683451] [INFO ] [REMOTE ] CTEST: Full fetch complete (1020 records). [2025-09-16T17:57:08.683512] [DEBUG] [VALIDATOR ] CTEST: Validating primary data. [2025-09-16T17:57:08.684156] [FATAL] [VALIDATOR ] CTEST: Data is stale - most recent date (2025-07-02) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:57:08.684228] [FATAL] [MAIN ] CTEST: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:57:08.684271] [WARN ] [MAIN ] CTEST: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:57:08.684315] [INFO ] [LOCAL ] CTEST: Starting deletion of primary data. [2025-09-16T17:57:08.684371] [DEBUG] [LOCAL ] CTEST: No primary data file found to delete. [2025-09-16T17:57:08.684407] [INFO ] [LOCAL ] CTEST: Starting deletion of derivative data. [2025-09-16T17:57:08.684462] [DEBUG] [LOCAL ] CTEST: No derivative data file found to delete. [2025-09-16T17:57:08.684521] [INFO ] [MAIN ] Update Progress: 543/2369 symbols completed. CTEST took 14.9s. Estimated remaining: 3h 11m 60s. Rate: 9.5 symbols/min [2025-09-16T17:57:08.894930] [INFO ] [MAIN ] CTO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:57:08.914310] [DEBUG] [LOCAL ] CTO: Loaded 19381 existing records from primary storage. [2025-09-16T17:57:08.914468] [INFO ] [MAIN ] CTO: Loaded 19381 existing records. [2025-09-16T17:57:09.090293] [WARN ] [MAIN ] CTO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:57:09.090533] [INFO ] [MAIN ] CTO: Wiping existing primary data before full refresh. [2025-09-16T17:57:09.090579] [INFO ] [LOCAL ] CTO: Starting deletion of primary data. [2025-09-16T17:57:09.091170] [DEBUG] [LOCAL ] CTO: Deleted primary data file. [2025-09-16T17:57:09.091219] [INFO ] [MAIN ] CTO: Starting full primary refresh. [2025-09-16T17:57:09.180119] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-07. [2025-09-16T17:57:09.449380] [INFO ] [REMOTE ] CTO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:57:09.756421] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-09-16T17:57:09.974914] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-08. [2025-09-16T17:57:10.582492] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-09-16T17:57:10.774913] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-09. [2025-09-16T17:57:11.529639] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-09-16T17:57:11.530878] [INFO ] [REMOTE ] CTDD: Full fetch complete (6241 records). [2025-09-16T17:57:11.530946] [DEBUG] [VALIDATOR ] CTDD: Validating primary data. [2025-09-16T17:57:11.583855] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-02. [2025-09-16T17:57:11.887156] [DEBUG] [VALIDATOR ] CTDD: All expected trading days are present. [2025-09-16T17:57:11.887236] [DEBUG] [VALIDATOR ] CTDD: Primary data validation passed. [2025-09-16T17:57:11.887423] [INFO ] [MAIN ] CTDD: Fetched primary data is valid. [2025-09-16T17:57:11.887462] [DEBUG] [LOCAL ] CTDD: Saving primary data. [2025-09-16T17:57:11.887865] [DEBUG] [LOCAL ] CTDD: Trimmed 6241 records to 3822 (keeping 150 days of data) [2025-09-16T17:57:11.911806] [DEBUG] [LOCAL ] CTDD: Primary data saved (3822 records). [2025-09-16T17:57:11.911903] [INFO ] [MAIN ] CTDD: Full history saved. [2025-09-16T17:57:11.911991] [INFO ] [MAIN ] Update Progress: 544/2369 symbols completed. CTDD took 21.0s. Estimated remaining: 3h 11m 43s. Rate: 9.5 symbols/min [2025-09-16T17:57:12.086313] [INFO ] [MAIN ] CTOS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:57:12.111588] [DEBUG] [LOCAL ] CTOS: Loaded 26642 existing records from primary storage. [2025-09-16T17:57:12.111752] [INFO ] [MAIN ] CTOS: Loaded 26642 existing records. [2025-09-16T17:57:12.294858] [WARN ] [MAIN ] CTOS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:57:12.295117] [INFO ] [MAIN ] CTOS: Wiping existing primary data before full refresh. [2025-09-16T17:57:12.295171] [INFO ] [LOCAL ] CTOS: Starting deletion of primary data. [2025-09-16T17:57:12.295810] [DEBUG] [LOCAL ] CTOS: Deleted primary data file. [2025-09-16T17:57:12.295872] [INFO ] [MAIN ] CTOS: Starting full primary refresh. [2025-09-16T17:57:12.380062] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-02. [2025-09-16T17:57:12.675091] [INFO ] [REMOTE ] CTOS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:57:13.090104] [INFO ] [REMOTE ] CTEV: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-09-16T17:57:13.090981] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-09-16T17:57:13.174865] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-03. [2025-09-16T17:57:13.899462] [INFO ] [REMOTE ] CTEV: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-09-16T17:57:13.974871] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-04. [2025-09-16T17:57:14.749033] [INFO ] [REMOTE ] CTEV: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-09-16T17:57:14.774850] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-05. [2025-09-16T17:57:15.537962] [INFO ] [REMOTE ] CTEV: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-09-16T17:57:15.574850] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-06. [2025-09-16T17:57:16.374902] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-07. [2025-09-16T17:57:16.465823] [INFO ] [REMOTE ] CTEV: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T17:57:17.012790] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-09-16T17:57:17.174898] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-02. [2025-09-16T17:57:17.974895] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-03. [2025-09-16T17:57:18.532938] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T17:57:18.774906] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-03. [2025-09-16T17:57:19.517962] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-09-16T17:57:19.574855] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-04. [2025-09-16T17:57:20.280397] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-09-16T17:57:20.374883] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-04. [2025-09-16T17:57:20.974117] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T17:57:21.174915] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-07. [2025-09-16T17:57:21.669144] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-09-16T17:57:21.770958] [INFO ] [REMOTE ] CTEV: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-09-16T17:57:21.974898] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-08. [2025-09-16T17:57:22.774898] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-05. [2025-09-16T17:57:22.848973] [INFO ] [REMOTE ] CTEV: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-09-16T17:57:23.574916] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-05. [2025-09-16T17:57:24.151915] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-09-16T17:57:24.374910] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-09. [2025-09-16T17:57:25.156725] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T17:57:25.174843] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-06. [2025-09-16T17:57:25.375526] [INFO ] [REMOTE ] CTEV: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T17:57:25.377516] [INFO ] [REMOTE ] CTEV: Full fetch complete (13025 records). [2025-09-16T17:57:25.377598] [DEBUG] [VALIDATOR ] CTEV: Validating primary data. [2025-09-16T17:57:25.596397] [WARN ] [VALIDATOR ] CTEV: Validation failed - insufficient historical data. Have 138 unique days, need 150 days [2025-09-16T17:57:25.596680] [FATAL] [MAIN ] CTEV: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:57:25.596729] [WARN ] [MAIN ] CTEV: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:57:25.596773] [INFO ] [LOCAL ] CTEV: Starting deletion of primary data. [2025-09-16T17:57:25.596830] [DEBUG] [LOCAL ] CTEV: No primary data file found to delete. [2025-09-16T17:57:25.596865] [INFO ] [LOCAL ] CTEV: Starting deletion of derivative data. [2025-09-16T17:57:25.596904] [DEBUG] [LOCAL ] CTEV: No derivative data file found to delete. [2025-09-16T17:57:25.596966] [INFO ] [MAIN ] Update Progress: 545/2369 symbols completed. CTEV took 20.0s. Estimated remaining: 3h 12m 1s. Rate: 9.5 symbols/min [2025-09-16T17:57:25.778833] [INFO ] [MAIN ] CTRA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:57:25.819674] [DEBUG] [LOCAL ] CTRA: Loaded 39612 existing records from primary storage. [2025-09-16T17:57:25.819838] [INFO ] [MAIN ] CTRA: Loaded 39612 existing records. [2025-09-16T17:57:25.978138] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-08. [2025-09-16T17:57:25.999858] [WARN ] [MAIN ] CTRA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:57:26.000117] [INFO ] [MAIN ] CTRA: Wiping existing primary data before full refresh. [2025-09-16T17:57:26.000167] [INFO ] [LOCAL ] CTRA: Starting deletion of primary data. [2025-09-16T17:57:26.001096] [DEBUG] [LOCAL ] CTRA: Deleted primary data file. [2025-09-16T17:57:26.001156] [INFO ] [MAIN ] CTRA: Starting full primary refresh. [2025-09-16T17:57:26.356670] [INFO ] [REMOTE ] CTRA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:57:26.426741] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T17:57:26.653890] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-09-16T17:57:26.774952] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-06. [2025-09-16T17:57:27.574884] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-02. [2025-09-16T17:57:27.890614] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T17:57:28.374937] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-09. [2025-09-16T17:57:29.174934] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-07. [2025-09-16T17:57:29.395943] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-09-16T17:57:29.504600] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T17:57:29.506730] [INFO ] [REMOTE ] CSV: Full fetch complete (14401 records). [2025-09-16T17:57:29.506808] [DEBUG] [VALIDATOR ] CSV: Validating primary data. [2025-09-16T17:57:29.886723] [DEBUG] [VALIDATOR ] CSV: All expected trading days are present. [2025-09-16T17:57:29.886807] [DEBUG] [VALIDATOR ] CSV: Primary data validation passed. [2025-09-16T17:57:29.887174] [INFO ] [MAIN ] CSV: Fetched primary data is valid. [2025-09-16T17:57:29.887238] [DEBUG] [LOCAL ] CSV: Saving primary data. [2025-09-16T17:57:29.887769] [DEBUG] [LOCAL ] CSV: Trimmed 14401 records to 8953 (keeping 150 days of data) [2025-09-16T17:57:29.942591] [DEBUG] [LOCAL ] CSV: Primary data saved (8953 records). [2025-09-16T17:57:29.942746] [INFO ] [MAIN ] CSV: Full history saved. [2025-09-16T17:57:29.942834] [INFO ] [MAIN ] Update Progress: 546/2369 symbols completed. CSV took 53.7s. Estimated remaining: 3h 11m 48s. Rate: 9.5 symbols/min [2025-09-16T17:57:29.979968] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-03. [2025-09-16T17:57:30.130429] [INFO ] [MAIN ] CTRE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:57:30.169132] [DEBUG] [LOCAL ] CTRE: Loaded 36257 existing records from primary storage. [2025-09-16T17:57:30.169276] [INFO ] [MAIN ] CTRE: Loaded 36257 existing records. [2025-09-16T17:57:30.357307] [WARN ] [MAIN ] CTRE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:57:30.357569] [INFO ] [MAIN ] CTRE: Wiping existing primary data before full refresh. [2025-09-16T17:57:30.357617] [INFO ] [LOCAL ] CTRE: Starting deletion of primary data. [2025-09-16T17:57:30.358454] [DEBUG] [LOCAL ] CTRE: Deleted primary data file. [2025-09-16T17:57:30.358503] [INFO ] [MAIN ] CTRE: Starting full primary refresh. [2025-09-16T17:57:30.728397] [INFO ] [REMOTE ] CTRE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:57:30.745368] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T17:57:30.774846] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-08. [2025-09-16T17:57:31.574866] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-02. [2025-09-16T17:57:31.757829] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-09-16T17:57:31.975397] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-09-16T17:57:32.374925] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-04. [2025-09-16T17:57:33.103587] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T17:57:33.174878] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-03. [2025-09-16T17:57:33.974908] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-09. [2025-09-16T17:57:34.203299] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T17:57:34.774930] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-05. [2025-09-16T17:57:34.921570] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T17:57:35.330244] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T17:57:35.333807] [INFO ] [REMOTE ] CTO: Full fetch complete (31266 records). [2025-09-16T17:57:35.333910] [DEBUG] [VALIDATOR ] CTO: Validating primary data. [2025-09-16T17:57:35.581772] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-04. [2025-09-16T17:57:35.718349] [DEBUG] [VALIDATOR ] CTO: All expected trading days are present. [2025-09-16T17:57:35.718427] [DEBUG] [VALIDATOR ] CTO: Primary data validation passed. [2025-09-16T17:57:35.718979] [INFO ] [MAIN ] CTO: Fetched primary data is valid. [2025-09-16T17:57:35.719045] [DEBUG] [LOCAL ] CTO: Saving primary data. [2025-09-16T17:57:35.719808] [DEBUG] [LOCAL ] CTO: Trimmed 31266 records to 19613 (keeping 150 days of data) [2025-09-16T17:57:35.839151] [DEBUG] [LOCAL ] CTO: Primary data saved (19613 records). [2025-09-16T17:57:35.839293] [INFO ] [MAIN ] CTO: Full history saved. [2025-09-16T17:57:35.839395] [INFO ] [MAIN ] Update Progress: 547/2369 symbols completed. CTO took 27.2s. Estimated remaining: 3h 11m 41s. Rate: 9.5 symbols/min [2025-09-16T17:57:36.020865] [INFO ] [MAIN ] CTRI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:57:36.047896] [DEBUG] [LOCAL ] CTRI: Loaded 29026 existing records from primary storage. [2025-09-16T17:57:36.048058] [INFO ] [MAIN ] CTRI: Loaded 29026 existing records. [2025-09-16T17:57:36.235275] [WARN ] [MAIN ] CTRI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:57:36.235513] [INFO ] [MAIN ] CTRI: Wiping existing primary data before full refresh. [2025-09-16T17:57:36.235563] [INFO ] [LOCAL ] CTRI: Starting deletion of primary data. [2025-09-16T17:57:36.236249] [DEBUG] [LOCAL ] CTRI: Deleted primary data file. [2025-09-16T17:57:36.236306] [INFO ] [MAIN ] CTRI: Starting full primary refresh. [2025-09-16T17:57:36.385202] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-07. [2025-09-16T17:57:36.607913] [INFO ] [REMOTE ] CTRI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:57:36.733140] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-09-16T17:57:37.088596] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T17:57:37.174887] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-05. [2025-09-16T17:57:37.974916] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-06. [2025-09-16T17:57:38.041457] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-09-16T17:57:38.555240] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-09-16T17:57:38.774906] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-06. [2025-09-16T17:57:39.560916] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T17:57:39.574846] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-07. [2025-09-16T17:57:40.374900] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-08. [2025-09-16T17:57:40.446419] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-09-16T17:57:41.174916] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-07. [2025-09-16T17:57:41.392391] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T17:57:41.852716] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T17:57:41.974897] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-09. [2025-09-16T17:57:42.780077] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-02. [2025-09-16T17:57:42.799263] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-09-16T17:57:42.835530] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T17:57:42.839669] [INFO ] [REMOTE ] CTOS: Full fetch complete (39810 records). [2025-09-16T17:57:42.839751] [DEBUG] [VALIDATOR ] CTOS: Validating primary data. [2025-09-16T17:57:43.207453] [DEBUG] [VALIDATOR ] CTOS: All expected trading days are present. [2025-09-16T17:57:43.207530] [DEBUG] [VALIDATOR ] CTOS: Primary data validation passed. [2025-09-16T17:57:43.208088] [INFO ] [MAIN ] CTOS: Fetched primary data is valid. [2025-09-16T17:57:43.208132] [DEBUG] [LOCAL ] CTOS: Saving primary data. [2025-09-16T17:57:43.209040] [DEBUG] [LOCAL ] CTOS: Trimmed 39810 records to 26856 (keeping 150 days of data) [2025-09-16T17:57:43.361074] [DEBUG] [LOCAL ] CTOS: Primary data saved (26856 records). [2025-09-16T17:57:43.361220] [INFO ] [MAIN ] CTOS: Full history saved. [2025-09-16T17:57:43.361314] [INFO ] [MAIN ] Update Progress: 548/2369 symbols completed. CTOS took 31.4s. Estimated remaining: 3h 11m 38s. Rate: 9.5 symbols/min [2025-09-16T17:57:43.527892] [INFO ] [MAIN ] CTS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:57:43.542112] [DEBUG] [LOCAL ] CTS: Loaded 13778 existing records from primary storage. [2025-09-16T17:57:43.542250] [INFO ] [MAIN ] CTS: Loaded 13778 existing records. [2025-09-16T17:57:43.584435] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-08. [2025-09-16T17:57:43.718812] [WARN ] [MAIN ] CTS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:57:43.719058] [INFO ] [MAIN ] CTS: Wiping existing primary data before full refresh. [2025-09-16T17:57:43.719112] [INFO ] [LOCAL ] CTS: Starting deletion of primary data. [2025-09-16T17:57:43.719588] [DEBUG] [LOCAL ] CTS: Deleted primary data file. [2025-09-16T17:57:43.719635] [INFO ] [MAIN ] CTS: Starting full primary refresh. [2025-09-16T17:57:44.065679] [INFO ] [REMOTE ] CTS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:57:44.106831] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-09-16T17:57:44.374896] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-03. [2025-09-16T17:57:45.174905] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-08. [2025-09-16T17:57:45.317903] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-09-16T17:57:45.658075] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-09-16T17:57:45.974905] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-09. [2025-09-16T17:57:46.774930] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-04. [2025-09-16T17:57:46.963248] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T17:57:47.170747] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-09-16T17:57:47.176796] [INFO ] [REMOTE ] CTRE: Full fetch complete (53472 records). [2025-09-16T17:57:47.176940] [DEBUG] [VALIDATOR ] CTRE: Validating primary data. [2025-09-16T17:57:47.574973] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-09. [2025-09-16T17:57:47.589832] [DEBUG] [VALIDATOR ] CTRE: All expected trading days are present. [2025-09-16T17:57:47.589896] [DEBUG] [VALIDATOR ] CTRE: Primary data validation passed. [2025-09-16T17:57:47.590606] [INFO ] [MAIN ] CTRE: Fetched primary data is valid. [2025-09-16T17:57:47.590669] [DEBUG] [LOCAL ] CTRE: Saving primary data. [2025-09-16T17:57:47.592729] [DEBUG] [LOCAL ] CTRE: Trimmed 53472 records to 36634 (keeping 150 days of data) [2025-09-16T17:57:47.806542] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T17:57:47.840254] [DEBUG] [LOCAL ] CTRE: Primary data saved (36634 records). [2025-09-16T17:57:47.840391] [INFO ] [MAIN ] CTRE: Full history saved. [2025-09-16T17:57:47.840478] [INFO ] [MAIN ] Update Progress: 549/2369 symbols completed. CTRE took 17.9s. Estimated remaining: 3h 11m 26s. Rate: 9.5 symbols/min [2025-09-16T17:57:48.021196] [INFO ] [MAIN ] CTVA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:57:48.061818] [DEBUG] [LOCAL ] CTVA: Loaded 39324 existing records from primary storage. [2025-09-16T17:57:48.061970] [INFO ] [MAIN ] CTVA: Loaded 39324 existing records. [2025-09-16T17:57:48.252018] [WARN ] [MAIN ] CTVA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:57:48.252266] [INFO ] [MAIN ] CTVA: Wiping existing primary data before full refresh. [2025-09-16T17:57:48.252321] [INFO ] [LOCAL ] CTVA: Starting deletion of primary data. [2025-09-16T17:57:48.253279] [DEBUG] [LOCAL ] CTVA: Deleted primary data file. [2025-09-16T17:57:48.253329] [INFO ] [MAIN ] CTVA: Starting full primary refresh. [2025-09-16T17:57:48.380044] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-02. [2025-09-16T17:57:48.631630] [INFO ] [REMOTE ] CTVA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:57:49.174904] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-05. [2025-09-16T17:57:49.475046] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-09 (1.90 seconds). [2025-09-16T17:57:49.481544] [INFO ] [REMOTE ] CTRA: Full fetch complete (58329 records). [2025-09-16T17:57:49.481633] [DEBUG] [VALIDATOR ] CTRA: Validating primary data. [2025-09-16T17:57:49.864065] [DEBUG] [VALIDATOR ] CTRA: All expected trading days are present. [2025-09-16T17:57:49.864163] [DEBUG] [VALIDATOR ] CTRA: Primary data validation passed. [2025-09-16T17:57:49.864905] [INFO ] [MAIN ] CTRA: Fetched primary data is valid. [2025-09-16T17:57:49.864962] [DEBUG] [LOCAL ] CTRA: Saving primary data. [2025-09-16T17:57:49.875551] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-09-16T17:57:49.880107] [DEBUG] [LOCAL ] CTRA: Trimmed 58329 records to 40002 (keeping 150 days of data) [2025-09-16T17:57:49.977868] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-03. [2025-09-16T17:57:50.135169] [DEBUG] [LOCAL ] CTRA: Primary data saved (40002 records). [2025-09-16T17:57:50.135298] [INFO ] [MAIN ] CTRA: Full history saved. [2025-09-16T17:57:50.135999] [INFO ] [MAIN ] Update Progress: 550/2369 symbols completed. CTRA took 24.5s. Estimated remaining: 3h 11m 6s. Rate: 9.5 symbols/min [2025-09-16T17:57:50.307584] [INFO ] [MAIN ] CUBB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:57:50.307848] [DEBUG] [LOCAL ] CUBB: No existing data file found in primary storage. [2025-09-16T17:57:50.307954] [WARN ] [MAIN ] CUBB: No existing data found, performing full refresh instead. [2025-09-16T17:57:50.308240] [INFO ] [MAIN ] CUBB: Wiping existing primary data before full refresh. [2025-09-16T17:57:50.308377] [INFO ] [LOCAL ] CUBB: Starting deletion of primary data. [2025-09-16T17:57:50.308453] [DEBUG] [LOCAL ] CUBB: No primary data file found to delete. [2025-09-16T17:57:50.308493] [INFO ] [MAIN ] CUBB: Starting full primary refresh. [2025-09-16T17:57:50.554219] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-09-16T17:57:50.710236] [INFO ] [REMOTE ] CUBB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:57:50.774887] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-02. [2025-09-16T17:57:51.148859] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T17:57:51.222000] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-09-16T17:57:51.574942] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-04. [2025-09-16T17:57:52.374936] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-06. [2025-09-16T17:57:52.498163] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-09-16T17:57:53.174922] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-05. [2025-09-16T17:57:53.822689] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T17:57:53.974877] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-07. [2025-09-16T17:57:54.227073] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-09-16T17:57:54.774909] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-06. [2025-09-16T17:57:55.470431] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T17:57:55.574881] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-03. [2025-09-16T17:57:55.873578] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-09-16T17:57:55.994330] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-09-16T17:57:56.374914] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-07. [2025-09-16T17:57:57.174905] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-08. [2025-09-16T17:57:57.516059] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T17:57:57.974961] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-04. [2025-09-16T17:57:58.315604] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-09-16T17:57:58.516492] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T17:57:58.774953] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-09. [2025-09-16T17:57:59.574925] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-02. [2025-09-16T17:58:00.258317] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T17:58:00.266284] [INFO ] [REMOTE ] CTRI: Full fetch complete (39890 records). [2025-09-16T17:58:00.266455] [DEBUG] [VALIDATOR ] CTRI: Validating primary data. [2025-09-16T17:58:00.380027] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-08. [2025-09-16T17:58:00.636555] [DEBUG] [VALIDATOR ] CTRI: All expected trading days are present. [2025-09-16T17:58:00.636632] [DEBUG] [VALIDATOR ] CTRI: Primary data validation passed. [2025-09-16T17:58:00.637208] [INFO ] [MAIN ] CTRI: Fetched primary data is valid. [2025-09-16T17:58:00.637257] [DEBUG] [LOCAL ] CTRI: Saving primary data. [2025-09-16T17:58:00.638100] [DEBUG] [LOCAL ] CTRI: Trimmed 39890 records to 29346 (keeping 150 days of data) [2025-09-16T17:58:00.815542] [DEBUG] [LOCAL ] CTRI: Primary data saved (29346 records). [2025-09-16T17:58:00.815761] [INFO ] [MAIN ] CTRI: Full history saved. [2025-09-16T17:58:00.815914] [INFO ] [MAIN ] Update Progress: 551/2369 symbols completed. CTRI took 25.0s. Estimated remaining: 3h 11m 15s. Rate: 9.5 symbols/min [2025-09-16T17:58:00.984619] [INFO ] [MAIN ] CUBE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:01.022564] [DEBUG] [LOCAL ] CUBE: Loaded 37556 existing records from primary storage. [2025-09-16T17:58:01.022759] [INFO ] [MAIN ] CUBE: Loaded 37556 existing records. [2025-09-16T17:58:01.177695] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-05. [2025-09-16T17:58:01.200303] [WARN ] [MAIN ] CUBE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:01.200491] [INFO ] [MAIN ] CUBE: Wiping existing primary data before full refresh. [2025-09-16T17:58:01.200547] [INFO ] [LOCAL ] CUBE: Starting deletion of primary data. [2025-09-16T17:58:01.201388] [DEBUG] [LOCAL ] CUBE: Deleted primary data file. [2025-09-16T17:58:01.201457] [INFO ] [MAIN ] CUBE: Starting full primary refresh. [2025-09-16T17:58:01.576723] [INFO ] [REMOTE ] CUBE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:01.629010] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-09-16T17:58:01.707368] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-09-16T17:58:01.723139] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T17:58:01.974942] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-02. [2025-09-16T17:58:02.774918] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-09. [2025-09-16T17:58:03.469881] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T17:58:03.487349] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-09-16T17:58:03.490105] [INFO ] [REMOTE ] CTS: Full fetch complete (20625 records). [2025-09-16T17:58:03.490215] [DEBUG] [VALIDATOR ] CTS: Validating primary data. [2025-09-16T17:58:03.580049] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-03. [2025-09-16T17:58:03.860958] [DEBUG] [VALIDATOR ] CTS: All expected trading days are present. [2025-09-16T17:58:03.861042] [DEBUG] [VALIDATOR ] CTS: Primary data validation passed. [2025-09-16T17:58:03.861433] [INFO ] [MAIN ] CTS: Fetched primary data is valid. [2025-09-16T17:58:03.861486] [DEBUG] [LOCAL ] CTS: Saving primary data. [2025-09-16T17:58:03.862116] [DEBUG] [LOCAL ] CTS: Trimmed 20625 records to 13918 (keeping 150 days of data) [2025-09-16T17:58:03.950789] [DEBUG] [LOCAL ] CTS: Primary data saved (13918 records). [2025-09-16T17:58:03.950939] [INFO ] [MAIN ] CTS: Full history saved. [2025-09-16T17:58:03.951063] [INFO ] [MAIN ] Update Progress: 552/2369 symbols completed. CTS took 20.6s. Estimated remaining: 3h 10m 58s. Rate: 9.5 symbols/min [2025-09-16T17:58:04.126700] [INFO ] [MAIN ] CUBI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:04.148278] [DEBUG] [LOCAL ] CUBI: Loaded 19575 existing records from primary storage. [2025-09-16T17:58:04.148400] [INFO ] [MAIN ] CUBI: Loaded 19575 existing records. [2025-09-16T17:58:04.336202] [WARN ] [MAIN ] CUBI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:04.336467] [INFO ] [MAIN ] CUBI: Wiping existing primary data before full refresh. [2025-09-16T17:58:04.336515] [INFO ] [LOCAL ] CUBI: Starting deletion of primary data. [2025-09-16T17:58:04.337025] [DEBUG] [LOCAL ] CUBI: Deleted primary data file. [2025-09-16T17:58:04.337085] [INFO ] [MAIN ] CUBI: Starting full primary refresh. [2025-09-16T17:58:04.380059] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-06. [2025-09-16T17:58:04.696711] [INFO ] [REMOTE ] CUBI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:04.776789] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-09-16T17:58:05.174880] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-02. [2025-09-16T17:58:05.423079] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T17:58:05.974918] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-03. [2025-09-16T17:58:06.070848] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-09-16T17:58:06.774918] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-07. [2025-09-16T17:58:07.102714] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-09-16T17:58:07.574899] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-08. [2025-09-16T17:58:07.852749] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-09-16T17:58:07.901935] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-09-16T17:58:08.374913] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-03. [2025-09-16T17:58:09.174865] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-04. [2025-09-16T17:58:09.496676] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-09-16T17:58:09.974934] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-04. [2025-09-16T17:58:10.774876] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-04. [2025-09-16T17:58:10.830359] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T17:58:11.574955] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-09. [2025-09-16T17:58:11.919878] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T17:58:12.040121] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-09-16T17:58:12.042979] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-09-16T17:58:12.044583] [INFO ] [REMOTE ] CUBB: Full fetch complete (749 records). [2025-09-16T17:58:12.044893] [DEBUG] [VALIDATOR ] CUBB: Validating primary data. [2025-09-16T17:58:12.236835] [WARN ] [VALIDATOR ] CUBB: Validation failed - insufficient historical data. Have 138 unique days, need 150 days [2025-09-16T17:58:12.237049] [FATAL] [MAIN ] CUBB: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:58:12.237102] [WARN ] [MAIN ] CUBB: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:58:12.237151] [INFO ] [LOCAL ] CUBB: Starting deletion of primary data. [2025-09-16T17:58:12.237209] [DEBUG] [LOCAL ] CUBB: No primary data file found to delete. [2025-09-16T17:58:12.237255] [INFO ] [LOCAL ] CUBB: Starting deletion of derivative data. [2025-09-16T17:58:12.237312] [DEBUG] [LOCAL ] CUBB: No derivative data file found to delete. [2025-09-16T17:58:12.237375] [INFO ] [MAIN ] Update Progress: 553/2369 symbols completed. CUBB took 22.1s. Estimated remaining: 3h 10m 58s. Rate: 9.5 symbols/min [2025-09-16T17:58:12.384445] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-05. [2025-09-16T17:58:12.409152] [INFO ] [MAIN ] CUK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:12.465920] [DEBUG] [LOCAL ] CUK: Loaded 37358 existing records from primary storage. [2025-09-16T17:58:12.466102] [INFO ] [MAIN ] CUK: Loaded 37358 existing records. [2025-09-16T17:58:12.680559] [WARN ] [MAIN ] CUK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:12.680840] [INFO ] [MAIN ] CUK: Wiping existing primary data before full refresh. [2025-09-16T17:58:12.680887] [INFO ] [LOCAL ] CUK: Starting deletion of primary data. [2025-09-16T17:58:12.681622] [DEBUG] [LOCAL ] CUK: Deleted primary data file. [2025-09-16T17:58:12.681702] [INFO ] [MAIN ] CUK: Starting full primary refresh. [2025-09-16T17:58:13.031496] [INFO ] [REMOTE ] CUK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:13.174919] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-05. [2025-09-16T17:58:13.979984] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-05. [2025-09-16T17:58:13.995027] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T17:58:14.774918] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-06. [2025-09-16T17:58:14.902006] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T17:58:14.939038] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-09-16T17:58:15.574931] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-06. [2025-09-16T17:58:16.097525] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T17:58:16.374863] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-06. [2025-09-16T17:58:16.572793] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T17:58:17.174900] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-07. [2025-09-16T17:58:17.747590] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T17:58:17.974885] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-07. [2025-09-16T17:58:18.464923] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-09-16T17:58:18.774903] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-08. [2025-09-16T17:58:19.510939] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-09-16T17:58:19.574855] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-07. [2025-09-16T17:58:19.949116] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T17:58:20.374899] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-02. [2025-09-16T17:58:21.174888] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-08. [2025-09-16T17:58:21.255270] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-09-16T17:58:21.801231] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-09-16T17:58:21.974885] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-08. [2025-09-16T17:58:22.774870] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-09. [2025-09-16T17:58:22.853438] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T17:58:23.574870] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-03. [2025-09-16T17:58:23.629993] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T17:58:23.653696] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-09-16T17:58:23.657954] [INFO ] [REMOTE ] CUBI: Full fetch complete (29219 records). [2025-09-16T17:58:23.658049] [DEBUG] [VALIDATOR ] CUBI: Validating primary data. [2025-09-16T17:58:24.025185] [DEBUG] [VALIDATOR ] CUBI: All expected trading days are present. [2025-09-16T17:58:24.025260] [DEBUG] [VALIDATOR ] CUBI: Primary data validation passed. [2025-09-16T17:58:24.025725] [INFO ] [MAIN ] CUBI: Fetched primary data is valid. [2025-09-16T17:58:24.025769] [DEBUG] [LOCAL ] CUBI: Saving primary data. [2025-09-16T17:58:24.026480] [DEBUG] [LOCAL ] CUBI: Trimmed 29219 records to 19849 (keeping 150 days of data) [2025-09-16T17:58:24.144737] [DEBUG] [LOCAL ] CUBI: Primary data saved (19849 records). [2025-09-16T17:58:24.144880] [INFO ] [MAIN ] CUBI: Full history saved. [2025-09-16T17:58:24.144968] [INFO ] [MAIN ] Update Progress: 554/2369 symbols completed. CUBI took 20.2s. Estimated remaining: 3h 11m 10s. Rate: 9.5 symbols/min [2025-09-16T17:58:24.317259] [INFO ] [MAIN ] CULP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:24.320852] [DEBUG] [LOCAL ] CULP: Loaded 1982 existing records from primary storage. [2025-09-16T17:58:24.320926] [INFO ] [MAIN ] CULP: Loaded 1982 existing records. [2025-09-16T17:58:24.384168] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-09. [2025-09-16T17:58:24.499245] [WARN ] [MAIN ] CULP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:24.499469] [INFO ] [MAIN ] CULP: Wiping existing primary data before full refresh. [2025-09-16T17:58:24.499514] [INFO ] [LOCAL ] CULP: Starting deletion of primary data. [2025-09-16T17:58:24.499832] [DEBUG] [LOCAL ] CULP: Deleted primary data file. [2025-09-16T17:58:24.499879] [INFO ] [MAIN ] CULP: Starting full primary refresh. [2025-09-16T17:58:24.853969] [INFO ] [REMOTE ] CULP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:25.174901] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-02. [2025-09-16T17:58:25.299466] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T17:58:25.866513] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T17:58:25.974890] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-04. [2025-09-16T17:58:26.015591] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-09-16T17:58:26.021593] [INFO ] [REMOTE ] CUBE: Full fetch complete (55020 records). [2025-09-16T17:58:26.021711] [DEBUG] [VALIDATOR ] CUBE: Validating primary data. [2025-09-16T17:58:26.422994] [DEBUG] [VALIDATOR ] CUBE: All expected trading days are present. [2025-09-16T17:58:26.423072] [DEBUG] [VALIDATOR ] CUBE: Primary data validation passed. [2025-09-16T17:58:26.423739] [INFO ] [MAIN ] CUBE: Fetched primary data is valid. [2025-09-16T17:58:26.423783] [DEBUG] [LOCAL ] CUBE: Saving primary data. [2025-09-16T17:58:26.424907] [DEBUG] [LOCAL ] CUBE: Trimmed 55020 records to 37936 (keeping 150 days of data) [2025-09-16T17:58:26.657908] [DEBUG] [LOCAL ] CUBE: Primary data saved (37936 records). [2025-09-16T17:58:26.658064] [INFO ] [MAIN ] CUBE: Full history saved. [2025-09-16T17:58:26.659598] [INFO ] [MAIN ] Update Progress: 555/2369 symbols completed. CUBE took 25.8s. Estimated remaining: 3h 10m 51s. Rate: 9.5 symbols/min [2025-09-16T17:58:26.780048] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-09. [2025-09-16T17:58:26.829426] [INFO ] [MAIN ] CURB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:26.852960] [DEBUG] [LOCAL ] CURB: Loaded 23402 existing records from primary storage. [2025-09-16T17:58:26.853093] [INFO ] [MAIN ] CURB: Loaded 23402 existing records. [2025-09-16T17:58:27.027734] [WARN ] [MAIN ] CURB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:27.027982] [INFO ] [MAIN ] CURB: Wiping existing primary data before full refresh. [2025-09-16T17:58:27.028027] [INFO ] [LOCAL ] CURB: Starting deletion of primary data. [2025-09-16T17:58:27.028615] [DEBUG] [LOCAL ] CURB: Deleted primary data file. [2025-09-16T17:58:27.028689] [INFO ] [MAIN ] CURB: Starting full primary refresh. [2025-09-16T17:58:27.382999] [INFO ] [REMOTE ] CURB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:27.574899] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-03. [2025-09-16T17:58:27.629055] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T17:58:28.175537] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-09-16T17:58:28.374940] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-04. [2025-09-16T17:58:28.620774] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-09-16T17:58:28.627589] [INFO ] [REMOTE ] CTVA: Full fetch complete (57840 records). [2025-09-16T17:58:28.627742] [DEBUG] [VALIDATOR ] CTVA: Validating primary data. [2025-09-16T17:58:29.027907] [DEBUG] [VALIDATOR ] CTVA: All expected trading days are present. [2025-09-16T17:58:29.031624] [DEBUG] [VALIDATOR ] CTVA: Primary data validation passed. [2025-09-16T17:58:29.031791] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-09-16T17:58:29.032498] [INFO ] [MAIN ] CTVA: Fetched primary data is valid. [2025-09-16T17:58:29.033579] [DEBUG] [LOCAL ] CTVA: Saving primary data. [2025-09-16T17:58:29.035565] [DEBUG] [LOCAL ] CTVA: Trimmed 57840 records to 39714 (keeping 150 days of data) [2025-09-16T17:58:29.223511] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-02. [2025-09-16T17:58:29.295607] [DEBUG] [LOCAL ] CTVA: Primary data saved (39714 records). [2025-09-16T17:58:29.295781] [INFO ] [MAIN ] CTVA: Full history saved. [2025-09-16T17:58:29.295869] [INFO ] [MAIN ] Update Progress: 556/2369 symbols completed. CTVA took 41.5s. Estimated remaining: 3h 10m 33s. Rate: 9.5 symbols/min [2025-09-16T17:58:29.468491] [INFO ] [MAIN ] CURV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:29.489387] [DEBUG] [LOCAL ] CURV: Loaded 19728 existing records from primary storage. [2025-09-16T17:58:29.489528] [INFO ] [MAIN ] CURV: Loaded 19728 existing records. [2025-09-16T17:58:29.668061] [WARN ] [MAIN ] CURV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:29.668298] [INFO ] [MAIN ] CURV: Wiping existing primary data before full refresh. [2025-09-16T17:58:29.668344] [INFO ] [LOCAL ] CURV: Starting deletion of primary data. [2025-09-16T17:58:29.668790] [DEBUG] [LOCAL ] CURV: Deleted primary data file. [2025-09-16T17:58:29.668838] [INFO ] [MAIN ] CURV: Starting full primary refresh. [2025-09-16T17:58:29.980049] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-05. [2025-09-16T17:58:30.046584] [INFO ] [REMOTE ] CURV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:30.604947] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-09-16T17:58:30.606352] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T17:58:30.774939] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-06. [2025-09-16T17:58:31.308777] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-09-16T17:58:31.574890] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-03. [2025-09-16T17:58:32.374914] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-02. [2025-09-16T17:58:33.002346] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-09-16T17:58:33.174901] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-05. [2025-09-16T17:58:33.208974] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-09-16T17:58:33.974897] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-03. [2025-09-16T17:58:34.742615] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T17:58:34.774845] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-06. [2025-09-16T17:58:35.070685] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T17:58:35.574934] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-07. [2025-09-16T17:58:36.035536] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-09-16T17:58:36.374896] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-04. [2025-09-16T17:58:36.505110] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-09-16T17:58:37.174891] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-04. [2025-09-16T17:58:37.211310] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-09-16T17:58:37.974900] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-07. [2025-09-16T17:58:38.455260] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-09-16T17:58:38.774905] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-05. [2025-09-16T17:58:39.574906] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-08. [2025-09-16T17:58:39.771827] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T17:58:39.843397] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T17:58:40.000446] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-09-16T17:58:40.374917] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-08. [2025-09-16T17:58:41.174917] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-05. [2025-09-16T17:58:41.758572] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T17:58:41.974915] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-09. [2025-09-16T17:58:42.456700] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-09-16T17:58:42.458327] [INFO ] [REMOTE ] CULP: Full fetch complete (3313 records). [2025-09-16T17:58:42.459478] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-09-16T17:58:42.459693] [DEBUG] [VALIDATOR ] CULP: Validating primary data. [2025-09-16T17:58:42.780909] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-06. [2025-09-16T17:58:42.817081] [DEBUG] [VALIDATOR ] CULP: All expected trading days are present. [2025-09-16T17:58:42.817136] [DEBUG] [VALIDATOR ] CULP: Primary data validation passed. [2025-09-16T17:58:42.817245] [INFO ] [MAIN ] CULP: Fetched primary data is valid. [2025-09-16T17:58:42.817283] [DEBUG] [LOCAL ] CULP: Saving primary data. [2025-09-16T17:58:42.817630] [DEBUG] [LOCAL ] CULP: Trimmed 3313 records to 2000 (keeping 150 days of data) [2025-09-16T17:58:42.829329] [DEBUG] [LOCAL ] CULP: Primary data saved (2000 records). [2025-09-16T17:58:42.829626] [INFO ] [MAIN ] CULP: Full history saved. [2025-09-16T17:58:42.829791] [INFO ] [MAIN ] Update Progress: 557/2369 symbols completed. CULP took 18.7s. Estimated remaining: 3h 10m 50s. Rate: 9.5 symbols/min [2025-09-16T17:58:43.574927] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-09. [2025-09-16T17:58:44.119153] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-09-16T17:58:44.374898] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-07. [2025-09-16T17:58:45.174924] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-06. [2025-09-16T17:58:45.493795] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T17:58:45.582392] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-09-16T17:58:45.591175] [INFO ] [REMOTE ] CUK: Full fetch complete (55544 records). [2025-09-16T17:58:45.591350] [DEBUG] [VALIDATOR ] CUK: Validating primary data. [2025-09-16T17:58:45.980143] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-08. [2025-09-16T17:58:45.984202] [DEBUG] [VALIDATOR ] CUK: All expected trading days are present. [2025-09-16T17:58:45.984324] [DEBUG] [VALIDATOR ] CUK: Primary data validation passed. [2025-09-16T17:58:45.985228] [INFO ] [MAIN ] CUK: Fetched primary data is valid. [2025-09-16T17:58:45.985368] [DEBUG] [LOCAL ] CUK: Saving primary data. [2025-09-16T17:58:45.986631] [DEBUG] [LOCAL ] CUK: Trimmed 55544 records to 37737 (keeping 150 days of data) [2025-09-16T17:58:46.218456] [DEBUG] [LOCAL ] CUK: Primary data saved (37737 records). [2025-09-16T17:58:46.218618] [INFO ] [MAIN ] CUK: Full history saved. [2025-09-16T17:58:46.219881] [INFO ] [MAIN ] Update Progress: 558/2369 symbols completed. CUK took 34.0s. Estimated remaining: 3h 10m 34s. Rate: 9.5 symbols/min [2025-09-16T17:58:46.472838] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T17:58:46.774950] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-07. [2025-09-16T17:58:47.329840] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-09-16T17:58:47.574958] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-09. [2025-09-16T17:58:47.970586] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T17:58:48.374907] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-08. [2025-09-16T17:58:48.951755] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T17:58:48.956700] [INFO ] [REMOTE ] CURV: Full fetch complete (28683 records). [2025-09-16T17:58:48.956784] [DEBUG] [VALIDATOR ] CURV: Validating primary data. [2025-09-16T17:58:49.339429] [DEBUG] [VALIDATOR ] CURV: All expected trading days are present. [2025-09-16T17:58:49.339511] [DEBUG] [VALIDATOR ] CURV: Primary data validation passed. [2025-09-16T17:58:49.339977] [INFO ] [MAIN ] CURV: Fetched primary data is valid. [2025-09-16T17:58:49.340020] [DEBUG] [LOCAL ] CURV: Saving primary data. [2025-09-16T17:58:49.340756] [DEBUG] [LOCAL ] CURV: Trimmed 28683 records to 20011 (keeping 150 days of data) [2025-09-16T17:58:49.450492] [DEBUG] [LOCAL ] CURV: Primary data saved (20011 records). [2025-09-16T17:58:49.450698] [INFO ] [MAIN ] CURV: Full history saved. [2025-09-16T17:58:49.450810] [INFO ] [MAIN ] Update Progress: 559/2369 symbols completed. CURV took 20.2s. Estimated remaining: 3h 10m 18s. Rate: 9.5 symbols/min [2025-09-16T17:58:49.525566] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T17:58:49.533040] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-09. [2025-09-16T17:58:50.856379] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T17:58:50.864003] [INFO ] [REMOTE ] CURB: Full fetch complete (35496 records). [2025-09-16T17:58:50.864274] [DEBUG] [VALIDATOR ] CURB: Validating primary data. [2025-09-16T17:58:51.438845] [DEBUG] [VALIDATOR ] CURB: All expected trading days are present. [2025-09-16T17:58:51.438948] [DEBUG] [VALIDATOR ] CURB: Primary data validation passed. [2025-09-16T17:58:51.439600] [INFO ] [MAIN ] CURB: Fetched primary data is valid. [2025-09-16T17:58:51.439667] [DEBUG] [LOCAL ] CURB: Saving primary data. [2025-09-16T17:58:51.440784] [DEBUG] [LOCAL ] CURB: Trimmed 35496 records to 23660 (keeping 150 days of data) [2025-09-16T17:58:51.601592] [DEBUG] [LOCAL ] CURB: Primary data saved (23660 records). [2025-09-16T17:58:51.601772] [INFO ] [MAIN ] CURB: Full history saved. [2025-09-16T17:58:51.601865] [INFO ] [MAIN ] Update Progress: 560/2369 symbols completed. CURB took 24.9s. Estimated remaining: 3h 9m 58s. Rate: 9.5 symbols/min [2025-09-16T17:58:52.252050] [INFO ] [MAIN ] CVI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:52.317051] [INFO ] [MAIN ] CVEO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:52.316032] [INFO ] [MAIN ] CVE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:52.338085] [INFO ] [MAIN ] CUZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:58:52.353567] [DEBUG] [LOCAL ] CVEO: Loaded 11679 existing records from primary storage. [2025-09-16T17:58:52.353895] [INFO ] [MAIN ] CVEO: Loaded 11679 existing records. [2025-09-16T17:58:52.588162] [WARN ] [MAIN ] CVEO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:52.587976] [DEBUG] [LOCAL ] CVI: Loaded 33984 existing records from primary storage. [2025-09-16T17:58:52.599733] [DEBUG] [LOCAL ] CUZ: Loaded 36592 existing records from primary storage. [2025-09-16T17:58:52.599112] [INFO ] [MAIN ] CVEO: Wiping existing primary data before full refresh. [2025-09-16T17:58:52.599903] [INFO ] [MAIN ] CVI: Loaded 33984 existing records. [2025-09-16T17:58:52.600022] [INFO ] [LOCAL ] CVEO: Starting deletion of primary data. [2025-09-16T17:58:52.631730] [INFO ] [MAIN ] CUZ: Loaded 36592 existing records. [2025-09-16T17:58:52.646240] [DEBUG] [LOCAL ] CVEO: Deleted primary data file. [2025-09-16T17:58:52.651512] [INFO ] [MAIN ] CVEO: Starting full primary refresh. [2025-09-16T17:58:52.806308] [DEBUG] [LOCAL ] CVE: Loaded 39534 existing records from primary storage. [2025-09-16T17:58:52.885688] [INFO ] [MAIN ] CVE: Loaded 39534 existing records. [2025-09-16T17:58:53.093196] [WARN ] [MAIN ] CVI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:53.249373] [WARN ] [MAIN ] CUZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:53.302848] [INFO ] [MAIN ] CUZ: Wiping existing primary data before full refresh. [2025-09-16T17:58:53.306085] [INFO ] [MAIN ] CVI: Wiping existing primary data before full refresh. [2025-09-16T17:58:53.311638] [INFO ] [LOCAL ] CVI: Starting deletion of primary data. [2025-09-16T17:58:53.306232] [INFO ] [LOCAL ] CUZ: Starting deletion of primary data. [2025-09-16T17:58:53.352274] [DEBUG] [LOCAL ] CVI: Deleted primary data file. [2025-09-16T17:58:53.387472] [DEBUG] [LOCAL ] CUZ: Deleted primary data file. [2025-09-16T17:58:53.419410] [INFO ] [MAIN ] CUZ: Starting full primary refresh. [2025-09-16T17:58:53.387680] [INFO ] [MAIN ] CVI: Starting full primary refresh. [2025-09-16T17:58:53.477526] [WARN ] [MAIN ] CVE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:58:53.507714] [INFO ] [MAIN ] CVE: Wiping existing primary data before full refresh. [2025-09-16T17:58:53.650115] [INFO ] [LOCAL ] CVE: Starting deletion of primary data. [2025-09-16T17:58:53.729507] [INFO ] [REMOTE ] CVEO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:53.739745] [DEBUG] [LOCAL ] CVE: Deleted primary data file. [2025-09-16T17:58:53.743329] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-02. [2025-09-16T17:58:53.743453] [INFO ] [MAIN ] CVE: Starting full primary refresh. [2025-09-16T17:58:54.503462] [INFO ] [REMOTE ] CVI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:54.572730] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-02. [2025-09-16T17:58:54.577411] [INFO ] [REMOTE ] CUZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:54.577712] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-02. [2025-09-16T17:58:54.656890] [INFO ] [REMOTE ] CVE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:58:54.657346] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-02. [2025-09-16T17:58:54.766333] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T17:58:54.769406] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-03. [2025-09-16T17:58:55.321484] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-09-16T17:58:55.327385] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-04. [2025-09-16T17:58:55.906563] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-09-16T17:58:55.910401] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-05. [2025-09-16T17:58:56.194109] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T17:58:56.202984] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-03. [2025-09-16T17:58:56.231673] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T17:58:56.374953] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-03. [2025-09-16T17:58:56.503489] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-09-16T17:58:56.620555] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-09-16T17:58:57.174923] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-06. [2025-09-16T17:58:57.453128] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-09-16T17:58:57.825166] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T17:58:57.974924] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-04. [2025-09-16T17:58:58.198399] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-09-16T17:58:58.774906] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-03. [2025-09-16T17:58:59.574921] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-04. [2025-09-16T17:58:59.620861] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T17:59:00.374940] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-07. [2025-09-16T17:59:00.732374] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-09-16T17:59:01.174918] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-05. [2025-09-16T17:59:01.313927] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-09-16T17:59:01.417892] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T17:59:01.974889] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-05. [2025-09-16T17:59:02.583763] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-09-16T17:59:02.774861] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-06. [2025-09-16T17:59:03.574944] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-04. [2025-09-16T17:59:03.755315] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-09-16T17:59:04.374937] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-06. [2025-09-16T17:59:04.515189] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T17:59:05.182687] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-07. [2025-09-16T17:59:05.189145] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-09-16T17:59:05.723265] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T17:59:05.974898] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-05. [2025-09-16T17:59:06.712863] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-09-16T17:59:06.774891] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-07. [2025-09-16T17:59:07.574891] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-08. [2025-09-16T17:59:07.961624] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-09-16T17:59:08.044505] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-09-16T17:59:08.322665] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T17:59:08.374929] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-08. [2025-09-16T17:59:09.174900] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-09. [2025-09-16T17:59:09.897417] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-09-16T17:59:09.899707] [INFO ] [REMOTE ] CVEO: Full fetch complete (16901 records). [2025-09-16T17:59:09.899806] [DEBUG] [VALIDATOR ] CVEO: Validating primary data. [2025-09-16T17:59:09.976591] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-08. [2025-09-16T17:59:10.074058] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-09-16T17:59:10.337711] [DEBUG] [VALIDATOR ] CVEO: All expected trading days are present. [2025-09-16T17:59:10.337819] [DEBUG] [VALIDATOR ] CVEO: Primary data validation passed. [2025-09-16T17:59:10.338175] [INFO ] [MAIN ] CVEO: Fetched primary data is valid. [2025-09-16T17:59:10.338216] [DEBUG] [LOCAL ] CVEO: Saving primary data. [2025-09-16T17:59:10.338757] [DEBUG] [LOCAL ] CVEO: Trimmed 16901 records to 11755 (keeping 150 days of data) [2025-09-16T17:59:10.406942] [DEBUG] [LOCAL ] CVEO: Primary data saved (11755 records). [2025-09-16T17:59:10.407090] [INFO ] [MAIN ] CVEO: Full history saved. [2025-09-16T17:59:10.407197] [INFO ] [MAIN ] Update Progress: 561/2369 symbols completed. CVEO took 18.8s. Estimated remaining: 3h 10m 32s. Rate: 9.5 symbols/min [2025-09-16T17:59:10.587075] [INFO ] [MAIN ] CVLG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:59:10.598805] [DEBUG] [LOCAL ] CVLG: Loaded 11290 existing records from primary storage. [2025-09-16T17:59:10.598938] [INFO ] [MAIN ] CVLG: Loaded 11290 existing records. [2025-09-16T17:59:10.781469] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-06. [2025-09-16T17:59:10.786007] [WARN ] [MAIN ] CVLG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:59:10.786987] [INFO ] [MAIN ] CVLG: Wiping existing primary data before full refresh. [2025-09-16T17:59:10.787058] [INFO ] [LOCAL ] CVLG: Starting deletion of primary data. [2025-09-16T17:59:10.787480] [DEBUG] [LOCAL ] CVLG: Deleted primary data file. [2025-09-16T17:59:10.787525] [INFO ] [MAIN ] CVLG: Starting full primary refresh. [2025-09-16T17:59:11.147141] [INFO ] [REMOTE ] CVLG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:59:11.574921] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-02. [2025-09-16T17:59:11.712335] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-09-16T17:59:12.355884] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-09-16T17:59:12.380007] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-09. [2025-09-16T17:59:12.403219] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T17:59:13.174906] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-03. [2025-09-16T17:59:13.784425] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T17:59:13.790729] [INFO ] [REMOTE ] CUZ: Full fetch complete (53412 records). [2025-09-16T17:59:13.790881] [DEBUG] [VALIDATOR ] CUZ: Validating primary data. [2025-09-16T17:59:13.975380] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-07. [2025-09-16T17:59:14.163927] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-09-16T17:59:14.198577] [DEBUG] [VALIDATOR ] CUZ: All expected trading days are present. [2025-09-16T17:59:14.199021] [DEBUG] [VALIDATOR ] CUZ: Primary data validation passed. [2025-09-16T17:59:14.199730] [INFO ] [MAIN ] CUZ: Fetched primary data is valid. [2025-09-16T17:59:14.199775] [DEBUG] [LOCAL ] CUZ: Saving primary data. [2025-09-16T17:59:14.200847] [DEBUG] [LOCAL ] CUZ: Trimmed 53412 records to 36954 (keeping 150 days of data) [2025-09-16T17:59:14.421386] [DEBUG] [LOCAL ] CUZ: Primary data saved (36954 records). [2025-09-16T17:59:14.421524] [INFO ] [MAIN ] CUZ: Full history saved. [2025-09-16T17:59:14.421608] [INFO ] [MAIN ] Update Progress: 562/2369 symbols completed. CUZ took 22.8s. Estimated remaining: 3h 10m 19s. Rate: 9.5 symbols/min [2025-09-16T17:59:14.595813] [INFO ] [MAIN ] CVNA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:59:14.632603] [DEBUG] [LOCAL ] CVNA: Loaded 39119 existing records from primary storage. [2025-09-16T17:59:14.632807] [INFO ] [MAIN ] CVNA: Loaded 39119 existing records. [2025-09-16T17:59:14.776983] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-09. [2025-09-16T17:59:14.809905] [WARN ] [MAIN ] CVNA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:59:14.810133] [INFO ] [MAIN ] CVNA: Wiping existing primary data before full refresh. [2025-09-16T17:59:14.810183] [INFO ] [LOCAL ] CVNA: Starting deletion of primary data. [2025-09-16T17:59:14.811028] [DEBUG] [LOCAL ] CVNA: Deleted primary data file. [2025-09-16T17:59:14.811078] [INFO ] [MAIN ] CVNA: Starting full primary refresh. [2025-09-16T17:59:15.176148] [INFO ] [REMOTE ] CVNA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:59:15.460692] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T17:59:15.574936] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-02. [2025-09-16T17:59:16.374900] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-04. [2025-09-16T17:59:16.595361] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-09-16T17:59:16.600813] [INFO ] [REMOTE ] CVI: Full fetch complete (49970 records). [2025-09-16T17:59:16.600923] [DEBUG] [VALIDATOR ] CVI: Validating primary data. [2025-09-16T17:59:16.601573] [FATAL] [VALIDATOR ] CVI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T17:59:16.601705] [FATAL] [MAIN ] CVI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T17:59:16.601745] [WARN ] [MAIN ] CVI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T17:59:16.601790] [INFO ] [LOCAL ] CVI: Starting deletion of primary data. [2025-09-16T17:59:16.601856] [DEBUG] [LOCAL ] CVI: No primary data file found to delete. [2025-09-16T17:59:16.601893] [INFO ] [LOCAL ] CVI: Starting deletion of derivative data. [2025-09-16T17:59:16.602498] [DEBUG] [LOCAL ] CVI: Deleted derivative data file. [2025-09-16T17:59:16.604537] [INFO ] [MAIN ] Update Progress: 563/2369 symbols completed. CVI took 25.0s. Estimated remaining: 3h 9m 59s. Rate: 9.5 symbols/min [2025-09-16T17:59:16.780539] [INFO ] [MAIN ] CVS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:59:16.822532] [DEBUG] [LOCAL ] CVS: Loaded 39658 existing records from primary storage. [2025-09-16T17:59:16.822716] [INFO ] [MAIN ] CVS: Loaded 39658 existing records. [2025-09-16T17:59:17.003743] [WARN ] [MAIN ] CVS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:59:17.004009] [INFO ] [MAIN ] CVS: Wiping existing primary data before full refresh. [2025-09-16T17:59:17.004068] [INFO ] [LOCAL ] CVS: Starting deletion of primary data. [2025-09-16T17:59:17.005100] [DEBUG] [LOCAL ] CVS: Deleted primary data file. [2025-09-16T17:59:17.005152] [INFO ] [MAIN ] CVS: Starting full primary refresh. [2025-09-16T17:59:17.178368] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-08. [2025-09-16T17:59:17.383680] [INFO ] [REMOTE ] CVS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:59:17.398199] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-09-16T17:59:17.462943] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-09-16T17:59:17.974898] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-03. [2025-09-16T17:59:18.774881] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-02. [2025-09-16T17:59:19.251286] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-09-16T17:59:19.574906] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-09. [2025-09-16T17:59:19.747859] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T17:59:20.328167] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T17:59:20.374863] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-03. [2025-09-16T17:59:21.174887] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-05. [2025-09-16T17:59:21.415629] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-09-16T17:59:21.421822] [INFO ] [REMOTE ] CVE: Full fetch complete (58231 records). [2025-09-16T17:59:21.421917] [DEBUG] [VALIDATOR ] CVE: Validating primary data. [2025-09-16T17:59:21.804486] [DEBUG] [VALIDATOR ] CVE: All expected trading days are present. [2025-09-16T17:59:21.804569] [DEBUG] [VALIDATOR ] CVE: Primary data validation passed. [2025-09-16T17:59:21.805283] [INFO ] [MAIN ] CVE: Fetched primary data is valid. [2025-09-16T17:59:21.805331] [DEBUG] [LOCAL ] CVE: Saving primary data. [2025-09-16T17:59:21.806770] [DEBUG] [LOCAL ] CVE: Trimmed 58231 records to 39924 (keeping 150 days of data) [2025-09-16T17:59:21.993421] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-04. [2025-09-16T17:59:22.063776] [DEBUG] [LOCAL ] CVE: Primary data saved (39924 records). [2025-09-16T17:59:22.063952] [INFO ] [MAIN ] CVE: Full history saved. [2025-09-16T17:59:22.064059] [INFO ] [MAIN ] Update Progress: 564/2369 symbols completed. CVE took 30.5s. Estimated remaining: 3h 9m 50s. Rate: 9.5 symbols/min [2025-09-16T17:59:22.237205] [INFO ] [MAIN ] CVX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:59:22.250548] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-09-16T17:59:22.293777] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-09-16T17:59:22.333143] [DEBUG] [LOCAL ] CVX: Loaded 39720 existing records from primary storage. [2025-09-16T17:59:22.333311] [INFO ] [MAIN ] CVX: Loaded 39720 existing records. [2025-09-16T17:59:22.547251] [WARN ] [MAIN ] CVX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:59:22.547505] [INFO ] [MAIN ] CVX: Wiping existing primary data before full refresh. [2025-09-16T17:59:22.547552] [INFO ] [LOCAL ] CVX: Starting deletion of primary data. [2025-09-16T17:59:22.548515] [DEBUG] [LOCAL ] CVX: Deleted primary data file. [2025-09-16T17:59:22.548567] [INFO ] [MAIN ] CVX: Starting full primary refresh. [2025-09-16T17:59:22.780138] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-04. [2025-09-16T17:59:22.918726] [INFO ] [REMOTE ] CVX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:59:23.574925] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-02. [2025-09-16T17:59:24.375939] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-04 (2.38 seconds). [2025-09-16T17:59:24.376048] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-06. [2025-09-16T17:59:24.459383] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-09-16T17:59:25.174974] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-05. [2025-09-16T17:59:25.512007] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-02 (1.94 seconds). [2025-09-16T17:59:25.657409] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T17:59:25.974878] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-03. [2025-09-16T17:59:26.774969] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-07. [2025-09-16T17:59:27.293777] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-09-16T17:59:27.541295] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T17:59:27.574853] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-08. [2025-09-16T17:59:28.186391] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-03 (2.21 seconds). [2025-09-16T17:59:28.206489] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-09-16T17:59:28.374927] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-05. [2025-09-16T17:59:29.174961] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-06. [2025-09-16T17:59:29.974936] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-04. [2025-09-16T17:59:30.208389] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-09-16T17:59:30.774952] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-06. [2025-09-16T17:59:31.514085] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-06 (2.34 seconds). [2025-09-16T17:59:31.574952] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-07. [2025-09-16T17:59:32.295216] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T17:59:32.383745] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-09. [2025-09-16T17:59:32.389311] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-04 (2.41 seconds). [2025-09-16T17:59:33.097500] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-09-16T17:59:33.099355] [INFO ] [REMOTE ] CVLG: Full fetch complete (16227 records). [2025-09-16T17:59:33.099442] [DEBUG] [VALIDATOR ] CVLG: Validating primary data. [2025-09-16T17:59:33.182657] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-07. [2025-09-16T17:59:33.473217] [DEBUG] [VALIDATOR ] CVLG: All expected trading days are present. [2025-09-16T17:59:33.473297] [DEBUG] [VALIDATOR ] CVLG: Primary data validation passed. [2025-09-16T17:59:33.473704] [INFO ] [MAIN ] CVLG: Fetched primary data is valid. [2025-09-16T17:59:33.473764] [DEBUG] [LOCAL ] CVLG: Saving primary data. [2025-09-16T17:59:33.474338] [DEBUG] [LOCAL ] CVLG: Trimmed 16227 records to 11374 (keeping 150 days of data) [2025-09-16T17:59:33.544026] [DEBUG] [LOCAL ] CVLG: Primary data saved (11374 records). [2025-09-16T17:59:33.544165] [INFO ] [MAIN ] CVLG: Full history saved. [2025-09-16T17:59:33.544250] [INFO ] [MAIN ] Update Progress: 565/2369 symbols completed. CVLG took 23.1s. Estimated remaining: 3h 10m 0s. Rate: 9.5 symbols/min [2025-09-16T17:59:33.720735] [INFO ] [MAIN ] CW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:59:33.769051] [DEBUG] [LOCAL ] CW: Loaded 16766 existing records from primary storage. [2025-09-16T17:59:33.769786] [INFO ] [MAIN ] CW: Loaded 16766 existing records. [2025-09-16T17:59:33.771226] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-07 (2.20 seconds). [2025-09-16T17:59:33.960247] [WARN ] [MAIN ] CW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:59:33.960500] [INFO ] [MAIN ] CW: Wiping existing primary data before full refresh. [2025-09-16T17:59:33.960567] [INFO ] [LOCAL ] CW: Starting deletion of primary data. [2025-09-16T17:59:33.961124] [DEBUG] [LOCAL ] CW: Deleted primary data file. [2025-09-16T17:59:33.961171] [INFO ] [MAIN ] CW: Starting full primary refresh. [2025-09-16T17:59:33.981338] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-05. [2025-09-16T17:59:34.316548] [INFO ] [REMOTE ] CW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:59:34.774950] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-02. [2025-09-16T17:59:35.301130] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-09-16T17:59:35.574927] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-08. [2025-09-16T17:59:36.198154] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T17:59:36.374907] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-03. [2025-09-16T17:59:36.445868] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-05 (2.46 seconds). [2025-09-16T17:59:37.174867] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-06. [2025-09-16T17:59:37.426510] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-09-16T17:59:37.535910] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T17:59:37.974923] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-09. [2025-09-16T17:59:38.774947] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-08. [2025-09-16T17:59:39.393049] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-06 (2.22 seconds). [2025-09-16T17:59:39.574933] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-07. [2025-09-16T17:59:40.154461] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-09-16T17:59:40.160875] [INFO ] [REMOTE ] CVS: Full fetch complete (58378 records). [2025-09-16T17:59:40.160988] [DEBUG] [VALIDATOR ] CVS: Validating primary data. [2025-09-16T17:59:40.385234] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-04. [2025-09-16T17:59:40.544756] [DEBUG] [VALIDATOR ] CVS: All expected trading days are present. [2025-09-16T17:59:40.544835] [DEBUG] [VALIDATOR ] CVS: Primary data validation passed. [2025-09-16T17:59:40.545503] [INFO ] [MAIN ] CVS: Fetched primary data is valid. [2025-09-16T17:59:40.545546] [DEBUG] [LOCAL ] CVS: Saving primary data. [2025-09-16T17:59:40.546734] [DEBUG] [LOCAL ] CVS: Trimmed 58378 records to 40048 (keeping 150 days of data) [2025-09-16T17:59:40.791573] [DEBUG] [LOCAL ] CVS: Primary data saved (40048 records). [2025-09-16T17:59:40.791764] [INFO ] [MAIN ] CVS: Full history saved. [2025-09-16T17:59:40.792667] [INFO ] [MAIN ] Update Progress: 566/2369 symbols completed. CVS took 24.2s. Estimated remaining: 3h 9m 57s. Rate: 9.5 symbols/min [2025-09-16T17:59:40.968481] [INFO ] [MAIN ] CWAN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:59:41.000270] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-08 (2.23 seconds). [2025-09-16T17:59:41.042885] [DEBUG] [LOCAL ] CWAN: Loaded 38857 existing records from primary storage. [2025-09-16T17:59:41.043022] [INFO ] [MAIN ] CWAN: Loaded 38857 existing records. [2025-09-16T17:59:41.183689] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-09. [2025-09-16T17:59:41.220947] [WARN ] [MAIN ] CWAN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:59:41.221156] [INFO ] [MAIN ] CWAN: Wiping existing primary data before full refresh. [2025-09-16T17:59:41.221207] [INFO ] [LOCAL ] CWAN: Starting deletion of primary data. [2025-09-16T17:59:41.222079] [DEBUG] [LOCAL ] CWAN: Deleted primary data file. [2025-09-16T17:59:41.222135] [INFO ] [MAIN ] CWAN: Starting full primary refresh. [2025-09-16T17:59:41.576532] [INFO ] [REMOTE ] CWAN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:59:41.591822] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-09-16T17:59:41.815059] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-07 (2.24 seconds). [2025-09-16T17:59:41.974871] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-08. [2025-09-16T17:59:42.774927] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-02. [2025-09-16T17:59:43.075473] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-09-16T17:59:43.082444] [INFO ] [REMOTE ] CVNA: Full fetch complete (57693 records). [2025-09-16T17:59:43.082564] [DEBUG] [VALIDATOR ] CVNA: Validating primary data. [2025-09-16T17:59:43.460099] [DEBUG] [VALIDATOR ] CVNA: All expected trading days are present. [2025-09-16T17:59:43.460178] [DEBUG] [VALIDATOR ] CVNA: Primary data validation passed. [2025-09-16T17:59:43.460853] [INFO ] [MAIN ] CVNA: Fetched primary data is valid. [2025-09-16T17:59:43.460899] [DEBUG] [LOCAL ] CVNA: Saving primary data. [2025-09-16T17:59:43.462054] [DEBUG] [LOCAL ] CVNA: Trimmed 57693 records to 39474 (keeping 150 days of data) [2025-09-16T17:59:43.578107] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-05. [2025-09-16T17:59:43.707163] [DEBUG] [LOCAL ] CVNA: Primary data saved (39474 records). [2025-09-16T17:59:43.707317] [INFO ] [MAIN ] CVNA: Full history saved. [2025-09-16T17:59:43.707404] [INFO ] [MAIN ] Update Progress: 567/2369 symbols completed. CVNA took 29.3s. Estimated remaining: 3h 9m 40s. Rate: 9.5 symbols/min [2025-09-16T17:59:43.880270] [INFO ] [MAIN ] CWEN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:59:43.912229] [DEBUG] [LOCAL ] CWEN: Loaded 32791 existing records from primary storage. [2025-09-16T17:59:43.912370] [INFO ] [MAIN ] CWEN: Loaded 32791 existing records. [2025-09-16T17:59:44.094605] [WARN ] [MAIN ] CWEN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:59:44.094982] [INFO ] [MAIN ] CWEN: Wiping existing primary data before full refresh. [2025-09-16T17:59:44.095246] [INFO ] [LOCAL ] CWEN: Starting deletion of primary data. [2025-09-16T17:59:44.096283] [DEBUG] [LOCAL ] CWEN: Deleted primary data file. [2025-09-16T17:59:44.096400] [INFO ] [MAIN ] CWEN: Starting full primary refresh. [2025-09-16T17:59:44.403330] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-08 (2.43 seconds). [2025-09-16T17:59:44.481942] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-09. [2025-09-16T17:59:44.510243] [INFO ] [REMOTE ] CWEN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:59:44.584590] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-09-16T17:59:44.882713] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T17:59:45.174927] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-02. [2025-09-16T17:59:45.974920] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-03. [2025-09-16T17:59:46.774910] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-06. [2025-09-16T17:59:46.924767] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T17:59:46.981172] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-09 (2.50 seconds). [2025-09-16T17:59:46.987575] [INFO ] [REMOTE ] CVX: Full fetch complete (58440 records). [2025-09-16T17:59:46.987727] [DEBUG] [VALIDATOR ] CVX: Validating primary data. [2025-09-16T17:59:47.367355] [DEBUG] [VALIDATOR ] CVX: All expected trading days are present. [2025-09-16T17:59:47.367445] [DEBUG] [VALIDATOR ] CVX: Primary data validation passed. [2025-09-16T17:59:47.368153] [INFO ] [MAIN ] CVX: Fetched primary data is valid. [2025-09-16T17:59:47.368233] [DEBUG] [LOCAL ] CVX: Saving primary data. [2025-09-16T17:59:47.369812] [DEBUG] [LOCAL ] CVX: Trimmed 58440 records to 40110 (keeping 150 days of data) [2025-09-16T17:59:47.522695] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T17:59:47.594493] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-03. [2025-09-16T17:59:47.665075] [DEBUG] [LOCAL ] CVX: Primary data saved (40110 records). [2025-09-16T17:59:47.665194] [INFO ] [MAIN ] CVX: Full history saved. [2025-09-16T17:59:47.666090] [INFO ] [MAIN ] Update Progress: 568/2369 symbols completed. CVX took 25.6s. Estimated remaining: 3h 9m 26s. Rate: 9.5 symbols/min [2025-09-16T17:59:47.833441] [INFO ] [MAIN ] CWH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T17:59:47.873707] [DEBUG] [LOCAL ] CWH: Loaded 38060 existing records from primary storage. [2025-09-16T17:59:47.873864] [INFO ] [MAIN ] CWH: Loaded 38060 existing records. [2025-09-16T17:59:48.047146] [WARN ] [MAIN ] CWH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T17:59:48.047367] [INFO ] [MAIN ] CWH: Wiping existing primary data before full refresh. [2025-09-16T17:59:48.047431] [INFO ] [LOCAL ] CWH: Starting deletion of primary data. [2025-09-16T17:59:48.048301] [DEBUG] [LOCAL ] CWH: Deleted primary data file. [2025-09-16T17:59:48.048357] [INFO ] [MAIN ] CWH: Starting full primary refresh. [2025-09-16T17:59:48.383245] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-04. [2025-09-16T17:59:48.402747] [INFO ] [REMOTE ] CWH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T17:59:48.703467] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-09-16T17:59:49.049092] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T17:59:49.174895] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-04. [2025-09-16T17:59:49.974936] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-07. [2025-09-16T17:59:50.140450] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T17:59:50.355385] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T17:59:50.774869] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-05. [2025-09-16T17:59:51.175160] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T17:59:51.574921] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-05. [2025-09-16T17:59:52.352446] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T17:59:52.374919] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-02. [2025-09-16T17:59:53.066266] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T17:59:53.174904] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-06. [2025-09-16T17:59:53.978769] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-06. [2025-09-16T17:59:53.983695] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T17:59:54.590026] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T17:59:54.774874] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-03. [2025-09-16T17:59:55.395558] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T17:59:55.574887] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-07. [2025-09-16T17:59:56.155131] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-09-16T17:59:56.374908] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-07. [2025-09-16T17:59:56.931237] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-09-16T17:59:57.174903] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-08. [2025-09-16T17:59:57.974931] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-04. [2025-09-16T17:59:58.262298] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T17:59:58.292888] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T17:59:58.774909] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-08. [2025-09-16T17:59:59.556242] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T17:59:59.574849] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-05. [2025-09-16T18:00:00.374949] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-09. [2025-09-16T18:00:00.430232] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T18:00:01.174910] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-08. [2025-09-16T18:00:01.566983] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-09-16T18:00:01.975170] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-06. [2025-09-16T18:00:02.095112] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-09-16T18:00:02.098475] [INFO ] [REMOTE ] CW: Full fetch complete (25596 records). [2025-09-16T18:00:02.098684] [DEBUG] [VALIDATOR ] CW: Validating primary data. [2025-09-16T18:00:02.526876] [DEBUG] [VALIDATOR ] CW: All expected trading days are present. [2025-09-16T18:00:02.526996] [DEBUG] [VALIDATOR ] CW: Primary data validation passed. [2025-09-16T18:00:02.527549] [INFO ] [MAIN ] CW: Fetched primary data is valid. [2025-09-16T18:00:02.527626] [DEBUG] [LOCAL ] CW: Saving primary data. [2025-09-16T18:00:02.530041] [DEBUG] [LOCAL ] CW: Trimmed 25596 records to 16921 (keeping 150 days of data) [2025-09-16T18:00:02.676595] [DEBUG] [LOCAL ] CW: Primary data saved (16921 records). [2025-09-16T18:00:02.676839] [INFO ] [MAIN ] CW: Full history saved. [2025-09-16T18:00:02.676998] [INFO ] [MAIN ] Update Progress: 569/2369 symbols completed. CW took 29.1s. Estimated remaining: 3h 9m 47s. Rate: 9.5 symbols/min [2025-09-16T18:00:02.776694] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-09. [2025-09-16T18:00:02.951195] [INFO ] [MAIN ] CWK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:02.951441] [DEBUG] [LOCAL ] CWK: No existing data file found in primary storage. [2025-09-16T18:00:02.951495] [WARN ] [MAIN ] CWK: No existing data found, performing full refresh instead. [2025-09-16T18:00:02.951719] [INFO ] [MAIN ] CWK: Wiping existing primary data before full refresh. [2025-09-16T18:00:02.951786] [INFO ] [LOCAL ] CWK: Starting deletion of primary data. [2025-09-16T18:00:02.951848] [DEBUG] [LOCAL ] CWK: No primary data file found to delete. [2025-09-16T18:00:02.951896] [INFO ] [MAIN ] CWK: Starting full primary refresh. [2025-09-16T18:00:03.505379] [INFO ] [REMOTE ] CWK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:03.563922] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-08 (2.39 seconds). [2025-09-16T18:00:03.578029] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-02. [2025-09-16T18:00:03.629950] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T18:00:04.374938] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-09. [2025-09-16T18:00:04.800133] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-09-16T18:00:04.806846] [INFO ] [REMOTE ] CWAN: Full fetch complete (56492 records). [2025-09-16T18:00:04.807057] [DEBUG] [VALIDATOR ] CWAN: Validating primary data. [2025-09-16T18:00:04.809272] [FATAL] [VALIDATOR ] CWAN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:00:04.809370] [FATAL] [MAIN ] CWAN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:00:04.809447] [WARN ] [MAIN ] CWAN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:00:04.809534] [INFO ] [LOCAL ] CWAN: Starting deletion of primary data. [2025-09-16T18:00:04.809623] [DEBUG] [LOCAL ] CWAN: No primary data file found to delete. [2025-09-16T18:00:04.809732] [INFO ] [LOCAL ] CWAN: Starting deletion of derivative data. [2025-09-16T18:00:04.810032] [DEBUG] [LOCAL ] CWAN: Deleted derivative data file. [2025-09-16T18:00:04.811136] [INFO ] [MAIN ] Update Progress: 570/2369 symbols completed. CWAN took 24.0s. Estimated remaining: 3h 9m 28s. Rate: 9.5 symbols/min [2025-09-16T18:00:04.983140] [INFO ] [MAIN ] CWT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:05.009152] [DEBUG] [LOCAL ] CWT: Loaded 20038 existing records from primary storage. [2025-09-16T18:00:05.009285] [INFO ] [MAIN ] CWT: Loaded 20038 existing records. [2025-09-16T18:00:05.174954] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-07. [2025-09-16T18:00:05.196533] [WARN ] [MAIN ] CWT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:00:05.198136] [INFO ] [MAIN ] CWT: Wiping existing primary data before full refresh. [2025-09-16T18:00:05.198255] [INFO ] [LOCAL ] CWT: Starting deletion of primary data. [2025-09-16T18:00:05.198855] [DEBUG] [LOCAL ] CWT: Deleted primary data file. [2025-09-16T18:00:05.198917] [INFO ] [MAIN ] CWT: Starting full primary refresh. [2025-09-16T18:00:05.388683] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-09-16T18:00:05.594379] [INFO ] [REMOTE ] CWT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:05.722371] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T18:00:05.728389] [INFO ] [REMOTE ] CWEN: Full fetch complete (48966 records). [2025-09-16T18:00:05.728483] [DEBUG] [VALIDATOR ] CWEN: Validating primary data. [2025-09-16T18:00:05.980447] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-03. [2025-09-16T18:00:06.112566] [DEBUG] [VALIDATOR ] CWEN: All expected trading days are present. [2025-09-16T18:00:06.112685] [DEBUG] [VALIDATOR ] CWEN: Primary data validation passed. [2025-09-16T18:00:06.113658] [INFO ] [MAIN ] CWEN: Fetched primary data is valid. [2025-09-16T18:00:06.113730] [DEBUG] [LOCAL ] CWEN: Saving primary data. [2025-09-16T18:00:06.115025] [DEBUG] [LOCAL ] CWEN: Trimmed 48966 records to 33122 (keeping 150 days of data) [2025-09-16T18:00:06.321587] [DEBUG] [LOCAL ] CWEN: Primary data saved (33122 records). [2025-09-16T18:00:06.321738] [INFO ] [MAIN ] CWEN: Full history saved. [2025-09-16T18:00:06.321862] [INFO ] [MAIN ] Update Progress: 571/2369 symbols completed. CWEN took 22.6s. Estimated remaining: 3h 9m 6s. Rate: 9.5 symbols/min [2025-09-16T18:00:06.495101] [INFO ] [MAIN ] CX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:06.533474] [DEBUG] [LOCAL ] CX: Loaded 38965 existing records from primary storage. [2025-09-16T18:00:06.533593] [INFO ] [MAIN ] CX: Loaded 38965 existing records. [2025-09-16T18:00:06.710265] [WARN ] [MAIN ] CX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:00:06.710505] [INFO ] [MAIN ] CX: Wiping existing primary data before full refresh. [2025-09-16T18:00:06.710555] [INFO ] [LOCAL ] CX: Starting deletion of primary data. [2025-09-16T18:00:06.711401] [DEBUG] [LOCAL ] CX: Deleted primary data file. [2025-09-16T18:00:06.711452] [INFO ] [MAIN ] CX: Starting full primary refresh. [2025-09-16T18:00:06.785472] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-02. [2025-09-16T18:00:07.072321] [INFO ] [REMOTE ] CX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:07.376460] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T18:00:07.574914] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-04. [2025-09-16T18:00:07.671544] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-09-16T18:00:08.374941] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-02. [2025-09-16T18:00:08.878907] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-09-16T18:00:09.174908] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-05. [2025-09-16T18:00:09.617617] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-07 (4.44 seconds). [2025-09-16T18:00:09.880996] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-09-16T18:00:09.974904] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-08. [2025-09-16T18:00:10.390287] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T18:00:10.774910] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-06. [2025-09-16T18:00:11.192517] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-09-16T18:00:11.574905] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-09. [2025-09-16T18:00:11.975530] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T18:00:12.374910] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-07. [2025-09-16T18:00:13.174920] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-03. [2025-09-16T18:00:13.625966] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-09-16T18:00:13.974954] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-08. [2025-09-16T18:00:14.539768] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T18:00:14.773521] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-09 (3.20 seconds). [2025-09-16T18:00:14.778527] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-04. [2025-09-16T18:00:14.780291] [INFO ] [REMOTE ] CWH: Full fetch complete (55080 records). [2025-09-16T18:00:14.780490] [DEBUG] [VALIDATOR ] CWH: Validating primary data. [2025-09-16T18:00:14.781326] [FATAL] [VALIDATOR ] CWH: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:00:14.781429] [FATAL] [MAIN ] CWH: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:00:14.781494] [WARN ] [MAIN ] CWH: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:00:14.781545] [INFO ] [LOCAL ] CWH: Starting deletion of primary data. [2025-09-16T18:00:14.781621] [DEBUG] [LOCAL ] CWH: No primary data file found to delete. [2025-09-16T18:00:14.781682] [INFO ] [LOCAL ] CWH: Starting deletion of derivative data. [2025-09-16T18:00:14.781921] [DEBUG] [LOCAL ] CWH: Deleted derivative data file. [2025-09-16T18:00:14.782795] [INFO ] [MAIN ] Update Progress: 572/2369 symbols completed. CWH took 27.1s. Estimated remaining: 3h 9m 7s. Rate: 9.5 symbols/min [2025-09-16T18:00:14.957516] [INFO ] [MAIN ] CXE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:14.969934] [DEBUG] [LOCAL ] CXE: Loaded 4647 existing records from primary storage. [2025-09-16T18:00:14.970068] [INFO ] [MAIN ] CXE: Loaded 4647 existing records. [2025-09-16T18:00:15.155973] [WARN ] [MAIN ] CXE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:00:15.156220] [INFO ] [MAIN ] CXE: Wiping existing primary data before full refresh. [2025-09-16T18:00:15.156276] [INFO ] [LOCAL ] CXE: Starting deletion of primary data. [2025-09-16T18:00:15.156539] [DEBUG] [LOCAL ] CXE: Deleted primary data file. [2025-09-16T18:00:15.156584] [INFO ] [MAIN ] CXE: Starting full primary refresh. [2025-09-16T18:00:15.505595] [INFO ] [REMOTE ] CXE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:15.574880] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-02. [2025-09-16T18:00:15.749749] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T18:00:16.210206] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T18:00:16.374911] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-09. [2025-09-16T18:00:16.777268] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-09-16T18:00:17.174902] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-03. [2025-09-16T18:00:17.585572] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-09-16T18:00:17.592341] [INFO ] [REMOTE ] CWK: Full fetch complete (53613 records). [2025-09-16T18:00:17.592497] [DEBUG] [VALIDATOR ] CWK: Validating primary data. [2025-09-16T18:00:17.981189] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-05. [2025-09-16T18:00:18.037994] [DEBUG] [VALIDATOR ] CWK: All expected trading days are present. [2025-09-16T18:00:18.038090] [DEBUG] [VALIDATOR ] CWK: Primary data validation passed. [2025-09-16T18:00:18.038831] [INFO ] [MAIN ] CWK: Fetched primary data is valid. [2025-09-16T18:00:18.038891] [DEBUG] [LOCAL ] CWK: Saving primary data. [2025-09-16T18:00:18.040720] [DEBUG] [LOCAL ] CWK: Trimmed 53613 records to 36655 (keeping 150 days of data) [2025-09-16T18:00:18.261902] [DEBUG] [LOCAL ] CWK: Primary data saved (36655 records). [2025-09-16T18:00:18.262053] [INFO ] [MAIN ] CWK: Full history saved. [2025-09-16T18:00:18.262131] [INFO ] [MAIN ] Update Progress: 573/2369 symbols completed. CWK took 15.6s. Estimated remaining: 3h 8m 51s. Rate: 9.5 symbols/min [2025-09-16T18:00:18.432119] [INFO ] [MAIN ] CXH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:18.437768] [DEBUG] [LOCAL ] CXH: Loaded 3905 existing records from primary storage. [2025-09-16T18:00:18.437880] [INFO ] [MAIN ] CXH: Loaded 3905 existing records. [2025-09-16T18:00:18.616222] [WARN ] [MAIN ] CXH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:00:18.616532] [INFO ] [MAIN ] CXH: Wiping existing primary data before full refresh. [2025-09-16T18:00:18.616579] [INFO ] [LOCAL ] CXH: Starting deletion of primary data. [2025-09-16T18:00:18.616880] [DEBUG] [LOCAL ] CXH: Deleted primary data file. [2025-09-16T18:00:18.616927] [INFO ] [MAIN ] CXH: Starting full primary refresh. [2025-09-16T18:00:18.776200] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-03. [2025-09-16T18:00:18.972021] [INFO ] [REMOTE ] CXH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:19.307349] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T18:00:19.574905] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-06. [2025-09-16T18:00:19.626254] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-03 (2.45 seconds). [2025-09-16T18:00:19.856172] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T18:00:20.374908] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-04. [2025-09-16T18:00:20.802914] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-09-16T18:00:21.158487] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T18:00:21.174840] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-07. [2025-09-16T18:00:21.974908] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-02. [2025-09-16T18:00:22.635294] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-09-16T18:00:22.774910] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-03. [2025-09-16T18:00:22.889528] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T18:00:23.057569] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-03 (0.28 seconds). [2025-09-16T18:00:23.574908] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-04. [2025-09-16T18:00:23.927789] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-09-16T18:00:24.374897] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-04. [2025-09-16T18:00:25.174897] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-05. [2025-09-16T18:00:25.412578] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T18:00:25.883229] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-09-16T18:00:25.974920] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-05. [2025-09-16T18:00:26.547168] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-09-16T18:00:26.774902] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-06. [2025-09-16T18:00:27.301532] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-09-16T18:00:27.574887] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-06. [2025-09-16T18:00:28.374914] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-05. [2025-09-16T18:00:29.174911] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-07. [2025-09-16T18:00:29.256636] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T18:00:29.676575] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-09-16T18:00:29.974861] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-08. [2025-09-16T18:00:30.730048] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-05 (2.36 seconds). [2025-09-16T18:00:30.774851] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-06. [2025-09-16T18:00:31.574904] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-08. [2025-09-16T18:00:31.847834] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T18:00:31.909805] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-09-16T18:00:32.308269] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-08 (2.33 seconds). [2025-09-16T18:00:32.374859] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-09. [2025-09-16T18:00:33.174922] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-07. [2025-09-16T18:00:33.797286] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T18:00:33.803519] [INFO ] [REMOTE ] CX: Full fetch complete (57441 records). [2025-09-16T18:00:33.803634] [DEBUG] [VALIDATOR ] CX: Validating primary data. [2025-09-16T18:00:33.984651] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-09. [2025-09-16T18:00:34.201817] [DEBUG] [VALIDATOR ] CX: All expected trading days are present. [2025-09-16T18:00:34.201937] [DEBUG] [VALIDATOR ] CX: Primary data validation passed. [2025-09-16T18:00:34.202679] [INFO ] [MAIN ] CX: Fetched primary data is valid. [2025-09-16T18:00:34.202734] [DEBUG] [LOCAL ] CX: Saving primary data. [2025-09-16T18:00:34.220804] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-09-16T18:00:34.219624] [DEBUG] [LOCAL ] CX: Trimmed 57441 records to 39346 (keeping 150 days of data) [2025-09-16T18:00:34.458160] [DEBUG] [LOCAL ] CX: Primary data saved (39346 records). [2025-09-16T18:00:34.458303] [INFO ] [MAIN ] CX: Full history saved. [2025-09-16T18:00:34.458389] [INFO ] [MAIN ] Update Progress: 574/2369 symbols completed. CX took 28.1s. Estimated remaining: 3h 9m 16s. Rate: 9.5 symbols/min [2025-09-16T18:00:34.628189] [INFO ] [MAIN ] CXM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:34.664499] [DEBUG] [LOCAL ] CXM: Loaded 35117 existing records from primary storage. [2025-09-16T18:00:34.664636] [INFO ] [MAIN ] CXM: Loaded 35117 existing records. [2025-09-16T18:00:34.780041] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-07. [2025-09-16T18:00:34.843021] [WARN ] [MAIN ] CXM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:00:34.843232] [INFO ] [MAIN ] CXM: Wiping existing primary data before full refresh. [2025-09-16T18:00:34.843280] [INFO ] [LOCAL ] CXM: Starting deletion of primary data. [2025-09-16T18:00:34.844112] [DEBUG] [LOCAL ] CXM: Deleted primary data file. [2025-09-16T18:00:34.844165] [INFO ] [MAIN ] CXM: Starting full primary refresh. [2025-09-16T18:00:34.947894] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T18:00:34.957358] [INFO ] [REMOTE ] CXH: Full fetch complete (4619 records). [2025-09-16T18:00:34.967625] [DEBUG] [VALIDATOR ] CXH: Validating primary data. [2025-09-16T18:00:35.465153] [INFO ] [REMOTE ] CXM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:35.569808] [DEBUG] [VALIDATOR ] CXH: All expected trading days are present. [2025-09-16T18:00:35.569887] [DEBUG] [VALIDATOR ] CXH: Primary data validation passed. [2025-09-16T18:00:35.570030] [INFO ] [MAIN ] CXH: Fetched primary data is valid. [2025-09-16T18:00:35.570068] [DEBUG] [LOCAL ] CXH: Saving primary data. [2025-09-16T18:00:35.570450] [DEBUG] [LOCAL ] CXH: Trimmed 4619 records to 3921 (keeping 150 days of data) [2025-09-16T18:00:35.579838] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-08. [2025-09-16T18:00:35.594986] [DEBUG] [LOCAL ] CXH: Primary data saved (3921 records). [2025-09-16T18:00:35.595069] [INFO ] [MAIN ] CXH: Full history saved. [2025-09-16T18:00:35.595148] [INFO ] [MAIN ] Update Progress: 575/2369 symbols completed. CXH took 17.3s. Estimated remaining: 3h 8m 53s. Rate: 9.5 symbols/min [2025-09-16T18:00:35.774779] [INFO ] [MAIN ] CXT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:35.775387] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T18:00:35.798139] [DEBUG] [LOCAL ] CXT: Loaded 22071 existing records from primary storage. [2025-09-16T18:00:35.798268] [INFO ] [MAIN ] CXT: Loaded 22071 existing records. [2025-09-16T18:00:35.982563] [WARN ] [MAIN ] CXT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:00:35.982837] [INFO ] [MAIN ] CXT: Wiping existing primary data before full refresh. [2025-09-16T18:00:35.982885] [INFO ] [LOCAL ] CXT: Starting deletion of primary data. [2025-09-16T18:00:35.983600] [DEBUG] [LOCAL ] CXT: Deleted primary data file. [2025-09-16T18:00:35.983676] [INFO ] [MAIN ] CXT: Starting full primary refresh. [2025-09-16T18:00:36.357310] [INFO ] [REMOTE ] CXT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:36.374911] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-02. [2025-09-16T18:00:36.419607] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-09-16T18:00:37.174964] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-02. [2025-09-16T18:00:37.424353] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-09-16T18:00:37.974946] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-03. [2025-09-16T18:00:38.258186] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-09-16T18:00:38.774896] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-03. [2025-09-16T18:00:38.910497] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T18:00:39.574905] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-08. [2025-09-16T18:00:40.013143] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-09-16T18:00:40.374875] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-04. [2025-09-16T18:00:40.455137] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-09-16T18:00:41.174909] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-09. [2025-09-16T18:00:41.804069] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T18:00:41.974890] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-04. [2025-09-16T18:00:42.674064] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:00:42.677459] [INFO ] [REMOTE ] CWT: Full fetch complete (30655 records). [2025-09-16T18:00:42.677544] [DEBUG] [VALIDATOR ] CWT: Validating primary data. [2025-09-16T18:00:42.779440] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-09. [2025-09-16T18:00:43.049904] [DEBUG] [VALIDATOR ] CWT: All expected trading days are present. [2025-09-16T18:00:43.049981] [DEBUG] [VALIDATOR ] CWT: Primary data validation passed. [2025-09-16T18:00:43.050446] [INFO ] [MAIN ] CWT: Fetched primary data is valid. [2025-09-16T18:00:43.050487] [DEBUG] [LOCAL ] CWT: Saving primary data. [2025-09-16T18:00:43.051214] [DEBUG] [LOCAL ] CWT: Trimmed 30655 records to 20240 (keeping 150 days of data) [2025-09-16T18:00:43.208730] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T18:00:43.224098] [DEBUG] [LOCAL ] CWT: Primary data saved (20240 records). [2025-09-16T18:00:43.224209] [INFO ] [MAIN ] CWT: Full history saved. [2025-09-16T18:00:43.224288] [INFO ] [MAIN ] Update Progress: 576/2369 symbols completed. CWT took 38.4s. Estimated remaining: 3h 8m 51s. Rate: 9.5 symbols/min [2025-09-16T18:00:43.437189] [INFO ] [MAIN ] CXW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:43.474501] [DEBUG] [LOCAL ] CXW: Loaded 32561 existing records from primary storage. [2025-09-16T18:00:43.474659] [INFO ] [MAIN ] CXW: Loaded 32561 existing records. [2025-09-16T18:00:43.576534] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-05. [2025-09-16T18:00:43.653562] [WARN ] [MAIN ] CXW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:00:43.653810] [INFO ] [MAIN ] CXW: Wiping existing primary data before full refresh. [2025-09-16T18:00:43.653885] [INFO ] [LOCAL ] CXW: Starting deletion of primary data. [2025-09-16T18:00:43.654554] [DEBUG] [LOCAL ] CXW: Deleted primary data file. [2025-09-16T18:00:43.654620] [INFO ] [MAIN ] CXW: Starting full primary refresh. [2025-09-16T18:00:43.773369] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T18:00:43.814553] [INFO ] [REMOTE ] CXE: Full fetch complete (7914 records). [2025-09-16T18:00:43.838988] [DEBUG] [VALIDATOR ] CXE: Validating primary data. [2025-09-16T18:00:44.232409] [INFO ] [REMOTE ] CXW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:44.379844] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-02. [2025-09-16T18:00:44.386384] [DEBUG] [VALIDATOR ] CXE: All expected trading days are present. [2025-09-16T18:00:44.386449] [DEBUG] [VALIDATOR ] CXE: Primary data validation passed. [2025-09-16T18:00:44.386619] [INFO ] [MAIN ] CXE: Fetched primary data is valid. [2025-09-16T18:00:44.386683] [DEBUG] [LOCAL ] CXE: Saving primary data. [2025-09-16T18:00:44.387301] [DEBUG] [LOCAL ] CXE: Trimmed 7914 records to 4683 (keeping 150 days of data) [2025-09-16T18:00:44.421118] [DEBUG] [LOCAL ] CXE: Primary data saved (4683 records). [2025-09-16T18:00:44.421314] [INFO ] [MAIN ] CXE: Full history saved. [2025-09-16T18:00:44.421409] [INFO ] [MAIN ] Update Progress: 577/2369 symbols completed. CXE took 29.6s. Estimated remaining: 3h 8m 29s. Rate: 9.5 symbols/min [2025-09-16T18:00:44.597808] [INFO ] [MAIN ] CYD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:44.618016] [DEBUG] [LOCAL ] CYD: Loaded 18953 existing records from primary storage. [2025-09-16T18:00:44.618155] [INFO ] [MAIN ] CYD: Loaded 18953 existing records. [2025-09-16T18:00:44.794977] [WARN ] [MAIN ] CYD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:00:44.795222] [INFO ] [MAIN ] CYD: Wiping existing primary data before full refresh. [2025-09-16T18:00:44.795270] [INFO ] [LOCAL ] CYD: Starting deletion of primary data. [2025-09-16T18:00:44.795774] [DEBUG] [LOCAL ] CYD: Deleted primary data file. [2025-09-16T18:00:44.795830] [INFO ] [MAIN ] CYD: Starting full primary refresh. [2025-09-16T18:00:45.147078] [INFO ] [REMOTE ] CYD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:45.174859] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-02. [2025-09-16T18:00:45.217683] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T18:00:45.974939] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-06. [2025-09-16T18:00:46.283760] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-09-16T18:00:46.603363] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-09-16T18:00:46.774905] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-03. [2025-09-16T18:00:47.574896] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-05. [2025-09-16T18:00:47.608881] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T18:00:48.196028] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T18:00:48.374918] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-03. [2025-09-16T18:00:48.950925] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-09-16T18:00:49.174938] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-04. [2025-09-16T18:00:49.413962] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-09-16T18:00:49.974963] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-06. [2025-09-16T18:00:50.774961] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-07. [2025-09-16T18:00:50.901725] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T18:00:51.356566] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T18:00:51.574840] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-05. [2025-09-16T18:00:52.374899] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-04. [2025-09-16T18:00:52.504326] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T18:00:52.959923] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T18:00:53.174925] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-06. [2025-09-16T18:00:53.298597] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-09-16T18:00:53.974891] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-08. [2025-09-16T18:00:54.782454] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-07. [2025-09-16T18:00:54.799076] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T18:00:55.436380] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-09-16T18:00:55.574936] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-09. [2025-09-16T18:00:56.152471] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T18:00:56.374970] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-08. [2025-09-16T18:00:57.174889] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-07. [2025-09-16T18:00:57.466871] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-09-16T18:00:57.497420] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-09-16T18:00:57.503199] [INFO ] [REMOTE ] CXM: Full fetch complete (52008 records). [2025-09-16T18:00:57.503295] [DEBUG] [VALIDATOR ] CXM: Validating primary data. [2025-09-16T18:00:57.882079] [DEBUG] [VALIDATOR ] CXM: All expected trading days are present. [2025-09-16T18:00:57.882156] [DEBUG] [VALIDATOR ] CXM: Primary data validation passed. [2025-09-16T18:00:57.882789] [INFO ] [MAIN ] CXM: Fetched primary data is valid. [2025-09-16T18:00:57.882834] [DEBUG] [LOCAL ] CXM: Saving primary data. [2025-09-16T18:00:57.883925] [DEBUG] [LOCAL ] CXM: Trimmed 52008 records to 35428 (keeping 150 days of data) [2025-09-16T18:00:57.978513] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-09. [2025-09-16T18:00:58.086714] [DEBUG] [LOCAL ] CXM: Primary data saved (35428 records). [2025-09-16T18:00:58.086852] [INFO ] [MAIN ] CXM: Full history saved. [2025-09-16T18:00:58.086935] [INFO ] [MAIN ] Update Progress: 578/2369 symbols completed. CXM took 23.6s. Estimated remaining: 3h 8m 45s. Rate: 9.5 symbols/min [2025-09-16T18:00:58.260540] [INFO ] [MAIN ] CYH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:00:58.293347] [DEBUG] [LOCAL ] CYH: Loaded 35531 existing records from primary storage. [2025-09-16T18:00:58.293533] [INFO ] [MAIN ] CYH: Loaded 35531 existing records. [2025-09-16T18:00:58.477605] [WARN ] [MAIN ] CYH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:00:58.477856] [INFO ] [MAIN ] CYH: Wiping existing primary data before full refresh. [2025-09-16T18:00:58.477902] [INFO ] [LOCAL ] CYH: Starting deletion of primary data. [2025-09-16T18:00:58.478633] [DEBUG] [LOCAL ] CYH: Deleted primary data file. [2025-09-16T18:00:58.478714] [INFO ] [MAIN ] CYH: Starting full primary refresh. [2025-09-16T18:00:58.777364] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-05. [2025-09-16T18:00:58.840955] [INFO ] [REMOTE ] CYH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:00:58.870025] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T18:00:59.574952] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-08. [2025-09-16T18:00:59.591318] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T18:00:59.595240] [INFO ] [REMOTE ] CXT: Full fetch complete (34045 records). [2025-09-16T18:00:59.595319] [DEBUG] [VALIDATOR ] CXT: Validating primary data. [2025-09-16T18:00:59.966119] [DEBUG] [VALIDATOR ] CXT: All expected trading days are present. [2025-09-16T18:00:59.966197] [DEBUG] [VALIDATOR ] CXT: Primary data validation passed. [2025-09-16T18:00:59.966708] [INFO ] [MAIN ] CXT: Fetched primary data is valid. [2025-09-16T18:00:59.966753] [DEBUG] [LOCAL ] CXT: Saving primary data. [2025-09-16T18:00:59.967545] [DEBUG] [LOCAL ] CXT: Trimmed 34045 records to 22307 (keeping 150 days of data) [2025-09-16T18:01:00.104635] [DEBUG] [LOCAL ] CXT: Primary data saved (22307 records). [2025-09-16T18:01:00.104794] [INFO ] [MAIN ] CXT: Full history saved. [2025-09-16T18:01:00.104881] [INFO ] [MAIN ] Update Progress: 579/2369 symbols completed. CXT took 24.5s. Estimated remaining: 3h 8m 26s. Rate: 9.5 symbols/min [2025-09-16T18:01:00.278241] [INFO ] [MAIN ] D: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:00.278430] [DEBUG] [LOCAL ] D: No existing data file found in primary storage. [2025-09-16T18:01:00.278501] [WARN ] [MAIN ] D: No existing data found, performing full refresh instead. [2025-09-16T18:01:00.278835] [INFO ] [MAIN ] D: Wiping existing primary data before full refresh. [2025-09-16T18:01:00.278895] [INFO ] [LOCAL ] D: Starting deletion of primary data. [2025-09-16T18:01:00.279024] [DEBUG] [LOCAL ] D: No primary data file found to delete. [2025-09-16T18:01:00.279128] [INFO ] [MAIN ] D: Starting full primary refresh. [2025-09-16T18:01:00.377848] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-02. [2025-09-16T18:01:00.591880] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T18:01:00.687381] [INFO ] [REMOTE ] D: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:01.100228] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T18:01:01.174926] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-09. [2025-09-16T18:01:01.974925] [INFO ] [REMOTE ] D: Fetching primary data for 2025-02. [2025-09-16T18:01:02.353843] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-09-16T18:01:02.737371] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T18:01:02.742753] [INFO ] [REMOTE ] CXW: Full fetch complete (48694 records). [2025-09-16T18:01:02.742920] [DEBUG] [VALIDATOR ] CXW: Validating primary data. [2025-09-16T18:01:02.743875] [FATAL] [VALIDATOR ] CXW: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:01:02.743988] [FATAL] [MAIN ] CXW: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:01:02.744044] [WARN ] [MAIN ] CXW: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:01:02.744092] [INFO ] [LOCAL ] CXW: Starting deletion of primary data. [2025-09-16T18:01:02.744164] [DEBUG] [LOCAL ] CXW: No primary data file found to delete. [2025-09-16T18:01:02.744205] [INFO ] [LOCAL ] CXW: Starting deletion of derivative data. [2025-09-16T18:01:02.744571] [DEBUG] [LOCAL ] CXW: Deleted derivative data file. [2025-09-16T18:01:02.745910] [INFO ] [MAIN ] Update Progress: 580/2369 symbols completed. CXW took 19.5s. Estimated remaining: 3h 8m 8s. Rate: 9.5 symbols/min [2025-09-16T18:01:02.779988] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-03. [2025-09-16T18:01:02.934726] [INFO ] [MAIN ] DAC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:02.934919] [DEBUG] [LOCAL ] DAC: No existing data file found in primary storage. [2025-09-16T18:01:02.934959] [WARN ] [MAIN ] DAC: No existing data found, performing full refresh instead. [2025-09-16T18:01:02.935096] [INFO ] [MAIN ] DAC: Wiping existing primary data before full refresh. [2025-09-16T18:01:02.935151] [INFO ] [LOCAL ] DAC: Starting deletion of primary data. [2025-09-16T18:01:02.935208] [DEBUG] [LOCAL ] DAC: No primary data file found to delete. [2025-09-16T18:01:02.935255] [INFO ] [MAIN ] DAC: Starting full primary refresh. [2025-09-16T18:01:03.302670] [INFO ] [REMOTE ] DAC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:03.574897] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-02. [2025-09-16T18:01:04.065240] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-02 (2.09 seconds). [2025-09-16T18:01:04.374874] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-06. [2025-09-16T18:01:04.748594] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-09-16T18:01:05.174917] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-03. [2025-09-16T18:01:05.309992] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-03 (2.53 seconds). [2025-09-16T18:01:05.538895] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-09-16T18:01:05.974884] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-07. [2025-09-16T18:01:06.099308] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-09-16T18:01:06.774943] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-04. [2025-09-16T18:01:07.232490] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T18:01:07.554698] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-09-16T18:01:07.574853] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-05. [2025-09-16T18:01:08.374925] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-08. [2025-09-16T18:01:08.464342] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-09-16T18:01:09.174905] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-06. [2025-09-16T18:01:09.558595] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T18:01:09.974892] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-09. [2025-09-16T18:01:10.310584] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:01:10.774912] [INFO ] [REMOTE ] D: Fetching primary data for 2025-03. [2025-09-16T18:01:11.303912] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-09-16T18:01:11.307615] [INFO ] [REMOTE ] CYD: Full fetch complete (29178 records). [2025-09-16T18:01:11.307719] [DEBUG] [VALIDATOR ] CYD: Validating primary data. [2025-09-16T18:01:11.580026] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-04. [2025-09-16T18:01:11.689730] [DEBUG] [VALIDATOR ] CYD: All expected trading days are present. [2025-09-16T18:01:11.689818] [DEBUG] [VALIDATOR ] CYD: Primary data validation passed. [2025-09-16T18:01:11.690343] [INFO ] [MAIN ] CYD: Fetched primary data is valid. [2025-09-16T18:01:11.690387] [DEBUG] [LOCAL ] CYD: Saving primary data. [2025-09-16T18:01:11.691197] [DEBUG] [LOCAL ] CYD: Trimmed 29178 records to 19154 (keeping 150 days of data) [2025-09-16T18:01:11.805697] [DEBUG] [LOCAL ] CYD: Primary data saved (19154 records). [2025-09-16T18:01:11.805847] [INFO ] [MAIN ] CYD: Full history saved. [2025-09-16T18:01:11.805931] [INFO ] [MAIN ] Update Progress: 581/2369 symbols completed. CYD took 27.4s. Estimated remaining: 3h 8m 10s. Rate: 9.5 symbols/min [2025-09-16T18:01:11.977821] [INFO ] [MAIN ] DAL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:11.978035] [DEBUG] [LOCAL ] DAL: No existing data file found in primary storage. [2025-09-16T18:01:11.978075] [WARN ] [MAIN ] DAL: No existing data found, performing full refresh instead. [2025-09-16T18:01:11.978228] [INFO ] [MAIN ] DAL: Wiping existing primary data before full refresh. [2025-09-16T18:01:11.978268] [INFO ] [LOCAL ] DAL: Starting deletion of primary data. [2025-09-16T18:01:11.978312] [DEBUG] [LOCAL ] DAL: No primary data file found to delete. [2025-09-16T18:01:11.978351] [INFO ] [MAIN ] DAL: Starting full primary refresh. [2025-09-16T18:01:12.332003] [INFO ] [REMOTE ] DAL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:12.374915] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-02. [2025-09-16T18:01:12.962777] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-03 (2.19 seconds). [2025-09-16T18:01:13.095308] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T18:01:13.174902] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-05. [2025-09-16T18:01:13.974898] [INFO ] [REMOTE ] D: Fetching primary data for 2025-04. [2025-09-16T18:01:14.629031] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T18:01:14.730327] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-02 (2.36 seconds). [2025-09-16T18:01:14.774846] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-03. [2025-09-16T18:01:15.574897] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-07. [2025-09-16T18:01:16.111461] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-09-16T18:01:16.374899] [INFO ] [REMOTE ] D: Fetching primary data for 2025-05. [2025-09-16T18:01:16.445092] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T18:01:17.179994] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-08. [2025-09-16T18:01:17.195401] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-03 (2.42 seconds). [2025-09-16T18:01:17.974894] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-06. [2025-09-16T18:01:18.218918] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T18:01:18.238789] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-09-16T18:01:18.774897] [INFO ] [REMOTE ] D: Fetching primary data for 2025-06. [2025-09-16T18:01:19.579968] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-04. [2025-09-16T18:01:19.593590] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T18:01:20.374900] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-07. [2025-09-16T18:01:20.562038] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-09-16T18:01:21.174894] [INFO ] [REMOTE ] D: Fetching primary data for 2025-07. [2025-09-16T18:01:21.720753] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T18:01:21.968593] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-04 (2.39 seconds). [2025-09-16T18:01:21.979185] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-09. [2025-09-16T18:01:22.774876] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-08. [2025-09-16T18:01:22.899588] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-09-16T18:01:23.336180] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T18:01:23.337838] [INFO ] [REMOTE ] DAC: Full fetch complete (14211 records). [2025-09-16T18:01:23.337905] [DEBUG] [VALIDATOR ] DAC: Validating primary data. [2025-09-16T18:01:23.579975] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-05. [2025-09-16T18:01:23.696297] [DEBUG] [VALIDATOR ] DAC: All expected trading days are present. [2025-09-16T18:01:23.696370] [DEBUG] [VALIDATOR ] DAC: Primary data validation passed. [2025-09-16T18:01:23.696714] [INFO ] [MAIN ] DAC: Fetched primary data is valid. [2025-09-16T18:01:23.696784] [DEBUG] [LOCAL ] DAC: Saving primary data. [2025-09-16T18:01:23.697251] [DEBUG] [LOCAL ] DAC: Trimmed 14211 records to 9052 (keeping 150 days of data) [2025-09-16T18:01:23.754221] [DEBUG] [LOCAL ] DAC: Primary data saved (9052 records). [2025-09-16T18:01:23.754346] [INFO ] [MAIN ] DAC: Full history saved. [2025-09-16T18:01:23.754420] [INFO ] [MAIN ] Update Progress: 582/2369 symbols completed. DAC took 21.0s. Estimated remaining: 3h 8m 21s. Rate: 9.5 symbols/min [2025-09-16T18:01:23.928884] [INFO ] [MAIN ] DAN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:23.966896] [DEBUG] [LOCAL ] DAN: Loaded 36208 existing records from primary storage. [2025-09-16T18:01:23.967042] [INFO ] [MAIN ] DAN: Loaded 36208 existing records. [2025-09-16T18:01:24.141208] [WARN ] [MAIN ] DAN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:01:24.141458] [INFO ] [MAIN ] DAN: Wiping existing primary data before full refresh. [2025-09-16T18:01:24.141520] [INFO ] [LOCAL ] DAN: Starting deletion of primary data. [2025-09-16T18:01:24.142301] [DEBUG] [LOCAL ] DAN: Deleted primary data file. [2025-09-16T18:01:24.142355] [INFO ] [MAIN ] DAN: Starting full primary refresh. [2025-09-16T18:01:24.381778] [INFO ] [REMOTE ] D: Fetching primary data for 2025-08. [2025-09-16T18:01:24.520972] [INFO ] [REMOTE ] DAN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:24.547851] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T18:01:25.174902] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-09. [2025-09-16T18:01:25.974882] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-02. [2025-09-16T18:01:26.340509] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-05 (2.76 seconds). [2025-09-16T18:01:26.380089] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-09-16T18:01:26.600428] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-09-16T18:01:26.605984] [INFO ] [REMOTE ] CYH: Full fetch complete (52653 records). [2025-09-16T18:01:26.606083] [DEBUG] [VALIDATOR ] CYH: Validating primary data. [2025-09-16T18:01:26.782730] [INFO ] [REMOTE ] D: Fetching primary data for 2025-09. [2025-09-16T18:01:26.996831] [DEBUG] [VALIDATOR ] CYH: All expected trading days are present. [2025-09-16T18:01:26.996929] [DEBUG] [VALIDATOR ] CYH: Primary data validation passed. [2025-09-16T18:01:26.997637] [INFO ] [MAIN ] CYH: Fetched primary data is valid. [2025-09-16T18:01:26.997717] [DEBUG] [LOCAL ] CYH: Saving primary data. [2025-09-16T18:01:26.999038] [DEBUG] [LOCAL ] CYH: Trimmed 52653 records to 35832 (keeping 150 days of data) [2025-09-16T18:01:27.202248] [DEBUG] [LOCAL ] CYH: Primary data saved (35832 records). [2025-09-16T18:01:27.202408] [INFO ] [MAIN ] CYH: Full history saved. [2025-09-16T18:01:27.202495] [INFO ] [MAIN ] Update Progress: 583/2369 symbols completed. CYH took 29.1s. Estimated remaining: 3h 8m 6s. Rate: 9.5 symbols/min [2025-09-16T18:01:27.385370] [INFO ] [MAIN ] DAO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:27.399156] [DEBUG] [LOCAL ] DAO: Loaded 13357 existing records from primary storage. [2025-09-16T18:01:27.399276] [INFO ] [MAIN ] DAO: Loaded 13357 existing records. [2025-09-16T18:01:27.580702] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-06. [2025-09-16T18:01:27.580890] [WARN ] [MAIN ] DAO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:01:27.582231] [INFO ] [MAIN ] DAO: Wiping existing primary data before full refresh. [2025-09-16T18:01:27.582289] [INFO ] [LOCAL ] DAO: Starting deletion of primary data. [2025-09-16T18:01:27.582682] [DEBUG] [LOCAL ] DAO: Deleted primary data file. [2025-09-16T18:01:27.582740] [INFO ] [MAIN ] DAO: Starting full primary refresh. [2025-09-16T18:01:27.951450] [INFO ] [REMOTE ] DAO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:27.978094] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-09-16T18:01:28.374899] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-03. [2025-09-16T18:01:28.932304] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-09 (2.15 seconds). [2025-09-16T18:01:28.939070] [INFO ] [REMOTE ] D: Full fetch complete (58325 records). [2025-09-16T18:01:28.939218] [DEBUG] [VALIDATOR ] D: Validating primary data. [2025-09-16T18:01:29.181976] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-02. [2025-09-16T18:01:29.342889] [DEBUG] [VALIDATOR ] D: All expected trading days are present. [2025-09-16T18:01:29.343139] [DEBUG] [VALIDATOR ] D: Primary data validation passed. [2025-09-16T18:01:29.344148] [INFO ] [MAIN ] D: Fetched primary data is valid. [2025-09-16T18:01:29.344307] [DEBUG] [LOCAL ] D: Saving primary data. [2025-09-16T18:01:29.346145] [DEBUG] [LOCAL ] D: Trimmed 58325 records to 40007 (keeping 150 days of data) [2025-09-16T18:01:29.469440] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-09-16T18:01:29.641524] [DEBUG] [LOCAL ] D: Primary data saved (40007 records). [2025-09-16T18:01:29.641674] [INFO ] [MAIN ] D: Full history saved. [2025-09-16T18:01:29.641752] [INFO ] [MAIN ] Update Progress: 584/2369 symbols completed. D took 29.5s. Estimated remaining: 3h 7m 48s. Rate: 9.5 symbols/min [2025-09-16T18:01:29.812632] [INFO ] [MAIN ] DAR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:29.812826] [DEBUG] [LOCAL ] DAR: No existing data file found in primary storage. [2025-09-16T18:01:29.812865] [WARN ] [MAIN ] DAR: No existing data found, performing full refresh instead. [2025-09-16T18:01:29.812981] [INFO ] [MAIN ] DAR: Wiping existing primary data before full refresh. [2025-09-16T18:01:29.813021] [INFO ] [LOCAL ] DAR: Starting deletion of primary data. [2025-09-16T18:01:29.813062] [DEBUG] [LOCAL ] DAR: No primary data file found to delete. [2025-09-16T18:01:29.813094] [INFO ] [MAIN ] DAR: Starting full primary refresh. [2025-09-16T18:01:29.980203] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-07. [2025-09-16T18:01:30.190282] [INFO ] [REMOTE ] DAR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:30.249357] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-09-16T18:01:30.483349] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-09-16T18:01:30.774924] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-02. [2025-09-16T18:01:31.574920] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-03. [2025-09-16T18:01:32.310533] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-09-16T18:01:32.374855] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-03. [2025-09-16T18:01:32.872084] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-07 (2.89 seconds). [2025-09-16T18:01:32.919145] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-09-16T18:01:33.174928] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-04. [2025-09-16T18:01:33.640120] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T18:01:33.974944] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-04. [2025-09-16T18:01:34.014081] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-09-16T18:01:34.774915] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-04. [2025-09-16T18:01:35.327849] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T18:01:35.574913] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-05. [2025-09-16T18:01:36.073118] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-09-16T18:01:36.374925] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-08. [2025-09-16T18:01:36.885144] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T18:01:37.174930] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-05. [2025-09-16T18:01:37.971239] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-09-16T18:01:37.975544] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-05. [2025-09-16T18:01:38.321554] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-09-16T18:01:38.774974] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-06. [2025-09-16T18:01:39.589348] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-06. [2025-09-16T18:01:39.592101] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T18:01:40.277990] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-09-16T18:01:40.374851] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-06. [2025-09-16T18:01:40.815273] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-06 (2.04 seconds). [2025-09-16T18:01:41.174893] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-09. [2025-09-16T18:01:41.979957] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-07. [2025-09-16T18:01:41.995997] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T18:01:42.774928] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-07. [2025-09-16T18:01:43.314471] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-09-16T18:01:43.320699] [INFO ] [REMOTE ] DAL: Full fetch complete (58433 records). [2025-09-16T18:01:43.320817] [DEBUG] [VALIDATOR ] DAL: Validating primary data. [2025-09-16T18:01:43.589958] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-07. [2025-09-16T18:01:43.718311] [DEBUG] [VALIDATOR ] DAL: All expected trading days are present. [2025-09-16T18:01:43.718424] [DEBUG] [VALIDATOR ] DAL: Primary data validation passed. [2025-09-16T18:01:43.719187] [INFO ] [MAIN ] DAL: Fetched primary data is valid. [2025-09-16T18:01:43.719240] [DEBUG] [LOCAL ] DAL: Saving primary data. [2025-09-16T18:01:43.720520] [DEBUG] [LOCAL ] DAL: Trimmed 58433 records to 40103 (keeping 150 days of data) [2025-09-16T18:01:43.988089] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-09-16T18:01:44.016627] [DEBUG] [LOCAL ] DAL: Primary data saved (40103 records). [2025-09-16T18:01:44.016772] [INFO ] [MAIN ] DAL: Full history saved. [2025-09-16T18:01:44.016871] [INFO ] [MAIN ] Update Progress: 585/2369 symbols completed. DAL took 32.2s. Estimated remaining: 3h 8m 6s. Rate: 9.5 symbols/min [2025-09-16T18:01:44.187222] [INFO ] [MAIN ] DAVA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:44.218369] [DEBUG] [LOCAL ] DAVA: Loaded 30318 existing records from primary storage. [2025-09-16T18:01:44.218499] [INFO ] [MAIN ] DAVA: Loaded 30318 existing records. [2025-09-16T18:01:44.378140] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-08. [2025-09-16T18:01:44.402134] [WARN ] [MAIN ] DAVA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:01:44.403373] [INFO ] [MAIN ] DAVA: Wiping existing primary data before full refresh. [2025-09-16T18:01:44.403476] [INFO ] [LOCAL ] DAVA: Starting deletion of primary data. [2025-09-16T18:01:44.404288] [DEBUG] [LOCAL ] DAVA: Deleted primary data file. [2025-09-16T18:01:44.404345] [INFO ] [MAIN ] DAVA: Starting full primary refresh. [2025-09-16T18:01:44.665695] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-09-16T18:01:44.757564] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-09-16T18:01:44.842317] [INFO ] [REMOTE ] DAVA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:45.174904] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-08. [2025-09-16T18:01:45.535033] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T18:01:45.974904] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-02. [2025-09-16T18:01:46.458161] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-09-16T18:01:46.774946] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-09. [2025-09-16T18:01:47.049102] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-09-16T18:01:47.574946] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-09. [2025-09-16T18:01:48.154526] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T18:01:48.160534] [INFO ] [REMOTE ] DAR: Full fetch complete (55935 records). [2025-09-16T18:01:48.160711] [DEBUG] [VALIDATOR ] DAR: Validating primary data. [2025-09-16T18:01:48.161445] [FATAL] [VALIDATOR ] DAR: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:01:48.161532] [FATAL] [MAIN ] DAR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:01:48.161569] [WARN ] [MAIN ] DAR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:01:48.161616] [INFO ] [LOCAL ] DAR: Starting deletion of primary data. [2025-09-16T18:01:48.161705] [DEBUG] [LOCAL ] DAR: No primary data file found to delete. [2025-09-16T18:01:48.161788] [INFO ] [LOCAL ] DAR: Starting deletion of derivative data. [2025-09-16T18:01:48.161843] [DEBUG] [LOCAL ] DAR: No derivative data file found to delete. [2025-09-16T18:01:48.161909] [INFO ] [MAIN ] Update Progress: 586/2369 symbols completed. DAR took 18.5s. Estimated remaining: 3h 7m 53s. Rate: 9.5 symbols/min [2025-09-16T18:01:48.334367] [INFO ] [MAIN ] DAY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:48.368479] [DEBUG] [LOCAL ] DAY: Loaded 36701 existing records from primary storage. [2025-09-16T18:01:48.368673] [INFO ] [MAIN ] DAY: Loaded 36701 existing records. [2025-09-16T18:01:48.380076] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-08. [2025-09-16T18:01:48.550090] [WARN ] [MAIN ] DAY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:01:48.550307] [INFO ] [MAIN ] DAY: Wiping existing primary data before full refresh. [2025-09-16T18:01:48.550355] [INFO ] [LOCAL ] DAY: Starting deletion of primary data. [2025-09-16T18:01:48.551353] [DEBUG] [LOCAL ] DAY: Deleted primary data file. [2025-09-16T18:01:48.551405] [INFO ] [MAIN ] DAY: Starting full primary refresh. [2025-09-16T18:01:48.910127] [INFO ] [REMOTE ] DAY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:48.968888] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T18:01:48.975123] [INFO ] [REMOTE ] DAN: Full fetch complete (53293 records). [2025-09-16T18:01:48.975306] [DEBUG] [VALIDATOR ] DAN: Validating primary data. [2025-09-16T18:01:49.175271] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-02. [2025-09-16T18:01:49.385625] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T18:01:49.503228] [DEBUG] [VALIDATOR ] DAN: All expected trading days are present. [2025-09-16T18:01:49.503334] [DEBUG] [VALIDATOR ] DAN: Primary data validation passed. [2025-09-16T18:01:49.504353] [INFO ] [MAIN ] DAN: Fetched primary data is valid. [2025-09-16T18:01:49.504450] [DEBUG] [LOCAL ] DAN: Saving primary data. [2025-09-16T18:01:49.506004] [DEBUG] [LOCAL ] DAN: Trimmed 53293 records to 36555 (keeping 150 days of data) [2025-09-16T18:01:49.871000] [DEBUG] [LOCAL ] DAN: Primary data saved (36555 records). [2025-09-16T18:01:49.871253] [INFO ] [MAIN ] DAN: Full history saved. [2025-09-16T18:01:49.871395] [INFO ] [MAIN ] Update Progress: 587/2369 symbols completed. DAN took 26.1s. Estimated remaining: 3h 7m 33s. Rate: 9.5 symbols/min [2025-09-16T18:01:49.980104] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-09. [2025-09-16T18:01:50.119259] [INFO ] [MAIN ] DB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:50.173769] [DEBUG] [LOCAL ] DB: Loaded 38694 existing records from primary storage. [2025-09-16T18:01:50.174127] [INFO ] [MAIN ] DB: Loaded 38694 existing records. [2025-09-16T18:01:50.375366] [WARN ] [MAIN ] DB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:01:50.375613] [INFO ] [MAIN ] DB: Wiping existing primary data before full refresh. [2025-09-16T18:01:50.375682] [INFO ] [LOCAL ] DB: Starting deletion of primary data. [2025-09-16T18:01:50.376553] [DEBUG] [LOCAL ] DB: Deleted primary data file. [2025-09-16T18:01:50.376623] [INFO ] [MAIN ] DB: Starting full primary refresh. [2025-09-16T18:01:50.759155] [INFO ] [REMOTE ] DB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:50.780240] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-02. [2025-09-16T18:01:50.793556] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T18:01:50.970070] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T18:01:50.972627] [INFO ] [REMOTE ] DAO: Full fetch complete (24373 records). [2025-09-16T18:01:50.972726] [DEBUG] [VALIDATOR ] DAO: Validating primary data. [2025-09-16T18:01:51.332753] [DEBUG] [VALIDATOR ] DAO: All expected trading days are present. [2025-09-16T18:01:51.332828] [DEBUG] [VALIDATOR ] DAO: Primary data validation passed. [2025-09-16T18:01:51.333248] [INFO ] [MAIN ] DAO: Fetched primary data is valid. [2025-09-16T18:01:51.333288] [DEBUG] [LOCAL ] DAO: Saving primary data. [2025-09-16T18:01:51.333940] [DEBUG] [LOCAL ] DAO: Trimmed 24373 records to 13384 (keeping 150 days of data) [2025-09-16T18:01:51.409966] [DEBUG] [LOCAL ] DAO: Primary data saved (13384 records). [2025-09-16T18:01:51.410115] [INFO ] [MAIN ] DAO: Full history saved. [2025-09-16T18:01:51.410201] [INFO ] [MAIN ] Update Progress: 588/2369 symbols completed. DAO took 24.2s. Estimated remaining: 3h 7m 12s. Rate: 9.5 symbols/min [2025-09-16T18:01:51.577766] [INFO ] [MAIN ] DBD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:01:51.577866] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-03. [2025-09-16T18:01:51.579169] [DEBUG] [LOCAL ] DBD: No existing data file found in primary storage. [2025-09-16T18:01:51.579232] [WARN ] [MAIN ] DBD: No existing data found, performing full refresh instead. [2025-09-16T18:01:51.579364] [INFO ] [MAIN ] DBD: Wiping existing primary data before full refresh. [2025-09-16T18:01:51.579403] [INFO ] [LOCAL ] DBD: Starting deletion of primary data. [2025-09-16T18:01:51.579444] [DEBUG] [LOCAL ] DBD: No primary data file found to delete. [2025-09-16T18:01:51.579477] [INFO ] [MAIN ] DBD: Starting full primary refresh. [2025-09-16T18:01:51.971055] [INFO ] [REMOTE ] DBD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:01:52.374914] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-02. [2025-09-16T18:01:53.128890] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-09-16T18:01:53.174446] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T18:01:53.184622] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-03. [2025-09-16T18:01:53.195812] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T18:01:53.974898] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-04. [2025-09-16T18:01:54.759264] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-09-16T18:01:54.774866] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-03. [2025-09-16T18:01:55.263801] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-09-16T18:01:55.574912] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-05. [2025-09-16T18:01:56.374895] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-04. [2025-09-16T18:01:56.804016] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-09-16T18:01:56.900582] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T18:01:57.174902] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-03. [2025-09-16T18:01:57.543053] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-09-16T18:01:57.974904] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-04. [2025-09-16T18:01:58.312541] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T18:01:58.774919] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-06. [2025-09-16T18:01:59.574908] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-05. [2025-09-16T18:01:59.661856] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T18:02:00.257336] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T18:02:00.374903] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-05. [2025-09-16T18:02:01.174899] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-04. [2025-09-16T18:02:01.344222] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-09-16T18:02:01.974911] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-07. [2025-09-16T18:02:02.081732] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-09-16T18:02:02.454403] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-09-16T18:02:02.774927] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-06. [2025-09-16T18:02:03.574917] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-06. [2025-09-16T18:02:03.847986] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T18:02:04.374933] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-05. [2025-09-16T18:02:04.411201] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T18:02:04.882037] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T18:02:05.175044] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-08. [2025-09-16T18:02:05.497559] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T18:02:05.974917] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-06. [2025-09-16T18:02:06.774908] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-07. [2025-09-16T18:02:06.810058] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T18:02:06.869224] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T18:02:07.574928] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-07. [2025-09-16T18:02:07.972701] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T18:02:08.374900] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-07. [2025-09-16T18:02:08.831888] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T18:02:09.174912] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-08. [2025-09-16T18:02:09.974910] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-09. [2025-09-16T18:02:10.199462] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T18:02:10.363222] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-09-16T18:02:10.774913] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-09. [2025-09-16T18:02:11.512796] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T18:02:11.518260] [INFO ] [REMOTE ] DAY: Full fetch complete (54138 records). [2025-09-16T18:02:11.518419] [DEBUG] [VALIDATOR ] DAY: Validating primary data. [2025-09-16T18:02:11.519225] [FATAL] [VALIDATOR ] DAY: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:02:11.519315] [FATAL] [MAIN ] DAY: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:02:11.519353] [WARN ] [MAIN ] DAY: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:02:11.519407] [INFO ] [LOCAL ] DAY: Starting deletion of primary data. [2025-09-16T18:02:11.519470] [DEBUG] [LOCAL ] DAY: No primary data file found to delete. [2025-09-16T18:02:11.519506] [INFO ] [LOCAL ] DAY: Starting deletion of derivative data. [2025-09-16T18:02:11.519892] [DEBUG] [LOCAL ] DAY: Deleted derivative data file. [2025-09-16T18:02:11.519987] [INFO ] [MAIN ] Update Progress: 589/2369 symbols completed. DAY took 23.4s. Estimated remaining: 3h 7m 48s. Rate: 9.5 symbols/min [2025-09-16T18:02:11.584589] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-08. [2025-09-16T18:02:11.696398] [INFO ] [MAIN ] DBI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:02:11.726455] [DEBUG] [LOCAL ] DBI: Loaded 32715 existing records from primary storage. [2025-09-16T18:02:11.726596] [INFO ] [MAIN ] DBI: Loaded 32715 existing records. [2025-09-16T18:02:11.918412] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T18:02:11.917866] [WARN ] [MAIN ] DBI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:02:11.921611] [INFO ] [MAIN ] DBI: Wiping existing primary data before full refresh. [2025-09-16T18:02:11.921703] [INFO ] [REMOTE ] DBD: Full fetch complete (24936 records). [2025-09-16T18:02:11.921862] [INFO ] [LOCAL ] DBI: Starting deletion of primary data. [2025-09-16T18:02:11.921821] [DEBUG] [VALIDATOR ] DBD: Validating primary data. [2025-09-16T18:02:11.922466] [FATAL] [VALIDATOR ] DBD: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:02:11.922572] [FATAL] [MAIN ] DBD: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:02:11.922672] [WARN ] [MAIN ] DBD: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:02:11.922741] [INFO ] [LOCAL ] DBD: Starting deletion of primary data. [2025-09-16T18:02:11.922794] [DEBUG] [LOCAL ] DBI: Deleted primary data file. [2025-09-16T18:02:11.922843] [DEBUG] [LOCAL ] DBD: No primary data file found to delete. [2025-09-16T18:02:11.922928] [INFO ] [MAIN ] DBI: Starting full primary refresh. [2025-09-16T18:02:11.922964] [INFO ] [LOCAL ] DBD: Starting deletion of derivative data. [2025-09-16T18:02:11.928792] [DEBUG] [LOCAL ] DBD: No derivative data file found to delete. [2025-09-16T18:02:11.993453] [INFO ] [MAIN ] Update Progress: 590/2369 symbols completed. DBD took 20.6s. Estimated remaining: 3h 7m 24s. Rate: 9.5 symbols/min [2025-09-16T18:02:12.385997] [INFO ] [MAIN ] DBL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:02:12.380187] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-08. [2025-09-16T18:02:12.469419] [DEBUG] [LOCAL ] DBL: Loaded 7034 existing records from primary storage. [2025-09-16T18:02:12.469215] [INFO ] [REMOTE ] DBI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:02:12.469776] [INFO ] [MAIN ] DBL: Loaded 7034 existing records. [2025-09-16T18:02:12.650170] [WARN ] [MAIN ] DBL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:02:12.650405] [INFO ] [MAIN ] DBL: Wiping existing primary data before full refresh. [2025-09-16T18:02:12.650451] [INFO ] [LOCAL ] DBL: Starting deletion of primary data. [2025-09-16T18:02:12.650843] [DEBUG] [LOCAL ] DBL: Deleted primary data file. [2025-09-16T18:02:12.650904] [INFO ] [MAIN ] DBL: Starting full primary refresh. [2025-09-16T18:02:13.015634] [INFO ] [REMOTE ] DBL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:02:13.046366] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-09-16T18:02:13.174912] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-02. [2025-09-16T18:02:13.865915] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T18:02:13.974916] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-02. [2025-09-16T18:02:14.017775] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T18:02:14.774918] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-09. [2025-09-16T18:02:15.574913] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-09. [2025-09-16T18:02:15.702801] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T18:02:16.374917] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-03. [2025-09-16T18:02:16.560223] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-09-16T18:02:16.566395] [INFO ] [REMOTE ] DAVA: Full fetch complete (43462 records). [2025-09-16T18:02:16.566564] [DEBUG] [VALIDATOR ] DAVA: Validating primary data. [2025-09-16T18:02:16.969166] [DEBUG] [VALIDATOR ] DAVA: All expected trading days are present. [2025-09-16T18:02:16.969268] [DEBUG] [VALIDATOR ] DAVA: Primary data validation passed. [2025-09-16T18:02:16.969990] [INFO ] [MAIN ] DAVA: Fetched primary data is valid. [2025-09-16T18:02:16.970059] [DEBUG] [LOCAL ] DAVA: Saving primary data. [2025-09-16T18:02:16.972195] [DEBUG] [LOCAL ] DAVA: Trimmed 43462 records to 30657 (keeping 150 days of data) [2025-09-16T18:02:17.167507] [DEBUG] [LOCAL ] DAVA: Primary data saved (30657 records). [2025-09-16T18:02:17.167688] [INFO ] [MAIN ] DAVA: Full history saved. [2025-09-16T18:02:17.168456] [INFO ] [MAIN ] Update Progress: 591/2369 symbols completed. DAVA took 33.2s. Estimated remaining: 3h 7m 14s. Rate: 9.5 symbols/min [2025-09-16T18:02:17.201092] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-03. [2025-09-16T18:02:17.346070] [INFO ] [MAIN ] DBRG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:02:17.387668] [DEBUG] [LOCAL ] DBRG: Loaded 36678 existing records from primary storage. [2025-09-16T18:02:17.387828] [INFO ] [MAIN ] DBRG: Loaded 36678 existing records. [2025-09-16T18:02:17.571944] [WARN ] [MAIN ] DBRG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:02:17.572216] [INFO ] [MAIN ] DBRG: Wiping existing primary data before full refresh. [2025-09-16T18:02:17.572263] [INFO ] [LOCAL ] DBRG: Starting deletion of primary data. [2025-09-16T18:02:17.573097] [DEBUG] [LOCAL ] DBRG: Deleted primary data file. [2025-09-16T18:02:17.573157] [INFO ] [MAIN ] DBRG: Starting full primary refresh. [2025-09-16T18:02:17.860968] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-09-16T18:02:17.958706] [INFO ] [REMOTE ] DBRG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:02:17.976625] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-04. [2025-09-16T18:02:17.980027] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-09 (2.41 seconds). [2025-09-16T18:02:17.988658] [INFO ] [REMOTE ] DB: Full fetch complete (57116 records). [2025-09-16T18:02:17.988842] [DEBUG] [VALIDATOR ] DB: Validating primary data. [2025-09-16T18:02:18.122765] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T18:02:18.410455] [DEBUG] [VALIDATOR ] DB: All expected trading days are present. [2025-09-16T18:02:18.410535] [DEBUG] [VALIDATOR ] DB: Primary data validation passed. [2025-09-16T18:02:18.411229] [INFO ] [MAIN ] DB: Fetched primary data is valid. [2025-09-16T18:02:18.411293] [DEBUG] [LOCAL ] DB: Saving primary data. [2025-09-16T18:02:18.412754] [DEBUG] [LOCAL ] DB: Trimmed 57116 records to 39078 (keeping 150 days of data) [2025-09-16T18:02:18.650387] [DEBUG] [LOCAL ] DB: Primary data saved (39078 records). [2025-09-16T18:02:18.650514] [INFO ] [MAIN ] DB: Full history saved. [2025-09-16T18:02:18.650623] [INFO ] [MAIN ] Update Progress: 592/2369 symbols completed. DB took 28.8s. Estimated remaining: 3h 6m 53s. Rate: 9.5 symbols/min [2025-09-16T18:02:18.780047] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-02. [2025-09-16T18:02:18.838408] [INFO ] [MAIN ] DCI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:02:18.864726] [DEBUG] [LOCAL ] DCI: Loaded 27192 existing records from primary storage. [2025-09-16T18:02:18.864864] [INFO ] [MAIN ] DCI: Loaded 27192 existing records. [2025-09-16T18:02:19.032496] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-09-16T18:02:19.056751] [WARN ] [MAIN ] DCI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:02:19.057196] [INFO ] [MAIN ] DCI: Wiping existing primary data before full refresh. [2025-09-16T18:02:19.057260] [INFO ] [LOCAL ] DCI: Starting deletion of primary data. [2025-09-16T18:02:19.057989] [DEBUG] [LOCAL ] DCI: Deleted primary data file. [2025-09-16T18:02:19.058053] [INFO ] [MAIN ] DCI: Starting full primary refresh. [2025-09-16T18:02:19.405900] [INFO ] [REMOTE ] DCI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:02:19.574899] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-02. [2025-09-16T18:02:20.374904] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-04. [2025-09-16T18:02:20.727498] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-09-16T18:02:21.064251] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T18:02:21.174915] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-03. [2025-09-16T18:02:21.755748] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-09-16T18:02:21.974940] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-05. [2025-09-16T18:02:22.742006] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-09-16T18:02:22.774925] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-03. [2025-09-16T18:02:23.259560] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-09-16T18:02:23.574901] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-05. [2025-09-16T18:02:23.966002] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-09-16T18:02:24.374907] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-04. [2025-09-16T18:02:24.482066] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T18:02:25.174910] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-06. [2025-09-16T18:02:25.721123] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-09-16T18:02:25.974929] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-06. [2025-09-16T18:02:26.284031] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-09-16T18:02:26.774932] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-05. [2025-09-16T18:02:27.574921] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-04. [2025-09-16T18:02:27.754080] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-09-16T18:02:28.374909] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-07. [2025-09-16T18:02:28.615942] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T18:02:28.990038] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-09-16T18:02:29.081132] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T18:02:29.174894] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-05. [2025-09-16T18:02:29.974898] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-07. [2025-09-16T18:02:30.592075] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-09-16T18:02:30.774939] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-06. [2025-09-16T18:02:31.141293] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T18:02:31.574934] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-06. [2025-09-16T18:02:32.223734] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T18:02:32.374941] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-07. [2025-09-16T18:02:32.853505] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T18:02:33.174905] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-08. [2025-09-16T18:02:33.974905] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-08. [2025-09-16T18:02:34.257162] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T18:02:34.783619] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-09. [2025-09-16T18:02:34.789545] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-07 (2.41 seconds). [2025-09-16T18:02:34.851120] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-09-16T18:02:35.574899] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-09. [2025-09-16T18:02:36.374894] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-07. [2025-09-16T18:02:36.509304] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-09-16T18:02:36.515047] [INFO ] [REMOTE ] DBI: Full fetch complete (49335 records). [2025-09-16T18:02:36.515284] [DEBUG] [VALIDATOR ] DBI: Validating primary data. [2025-09-16T18:02:36.629719] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T18:02:36.657230] [INFO ] [REMOTE ] DBL: Full fetch complete (9808 records). [2025-09-16T18:02:36.681260] [DEBUG] [VALIDATOR ] DBL: Validating primary data. [2025-09-16T18:02:36.723619] [FATAL] [VALIDATOR ] DBL: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:02:36.788828] [FATAL] [MAIN ] DBL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:02:36.820628] [WARN ] [MAIN ] DBL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:02:36.820761] [INFO ] [LOCAL ] DBL: Starting deletion of primary data. [2025-09-16T18:02:36.821073] [DEBUG] [LOCAL ] DBL: No primary data file found to delete. [2025-09-16T18:02:36.836618] [INFO ] [LOCAL ] DBL: Starting deletion of derivative data. [2025-09-16T18:02:36.877941] [DEBUG] [LOCAL ] DBL: Deleted derivative data file. [2025-09-16T18:02:36.897010] [INFO ] [MAIN ] Update Progress: 593/2369 symbols completed. DBL took 24.9s. Estimated remaining: 3h 7m 22s. Rate: 9.5 symbols/min [2025-09-16T18:02:36.908964] [DEBUG] [VALIDATOR ] DBI: All expected trading days are present. [2025-09-16T18:02:36.909141] [DEBUG] [VALIDATOR ] DBI: Primary data validation passed. [2025-09-16T18:02:36.941981] [INFO ] [MAIN ] DBI: Fetched primary data is valid. [2025-09-16T18:02:36.974247] [DEBUG] [LOCAL ] DBI: Saving primary data. [2025-09-16T18:02:37.012521] [DEBUG] [LOCAL ] DBI: Trimmed 49335 records to 33046 (keeping 150 days of data) [2025-09-16T18:02:37.139325] [INFO ] [MAIN ] DCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:02:37.176776] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-08. [2025-09-16T18:02:37.194027] [DEBUG] [LOCAL ] DCO: Loaded 10699 existing records from primary storage. [2025-09-16T18:02:37.197209] [INFO ] [MAIN ] DCO: Loaded 10699 existing records. [2025-09-16T18:02:37.456968] [WARN ] [MAIN ] DCO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:02:37.463583] [INFO ] [MAIN ] DCO: Wiping existing primary data before full refresh. [2025-09-16T18:02:37.464225] [INFO ] [LOCAL ] DCO: Starting deletion of primary data. [2025-09-16T18:02:37.465790] [DEBUG] [LOCAL ] DCO: Deleted primary data file. [2025-09-16T18:02:37.466402] [INFO ] [MAIN ] DCO: Starting full primary refresh. [2025-09-16T18:02:37.632727] [DEBUG] [LOCAL ] DBI: Primary data saved (33046 records). [2025-09-16T18:02:37.649012] [INFO ] [MAIN ] DBI: Full history saved. [2025-09-16T18:02:37.665820] [INFO ] [MAIN ] Update Progress: 594/2369 symbols completed. DBI took 26.1s. Estimated remaining: 3h 6m 59s. Rate: 9.5 symbols/min [2025-09-16T18:02:38.004958] [INFO ] [REMOTE ] DCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:02:38.033899] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-09-16T18:02:38.016063] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-02. [2025-09-16T18:02:38.074175] [INFO ] [MAIN ] DD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:02:38.114021] [DEBUG] [LOCAL ] DD: Loaded 38986 existing records from primary storage. [2025-09-16T18:02:38.114178] [INFO ] [MAIN ] DD: Loaded 38986 existing records. [2025-09-16T18:02:38.291201] [WARN ] [MAIN ] DD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:02:38.291465] [INFO ] [MAIN ] DD: Wiping existing primary data before full refresh. [2025-09-16T18:02:38.291511] [INFO ] [LOCAL ] DD: Starting deletion of primary data. [2025-09-16T18:02:38.292485] [DEBUG] [LOCAL ] DD: Deleted primary data file. [2025-09-16T18:02:38.292535] [INFO ] [MAIN ] DD: Starting full primary refresh. [2025-09-16T18:02:38.640279] [INFO ] [REMOTE ] DD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:02:38.774854] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-02. [2025-09-16T18:02:38.950983] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T18:02:38.968784] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-09-16T18:02:39.574914] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-09. [2025-09-16T18:02:40.374907] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-03. [2025-09-16T18:02:40.558538] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T18:02:41.174949] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-03. [2025-09-16T18:02:41.196097] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-09-16T18:02:41.325682] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-09-16T18:02:41.331549] [INFO ] [REMOTE ] DBRG: Full fetch complete (54306 records). [2025-09-16T18:02:41.331683] [DEBUG] [VALIDATOR ] DBRG: Validating primary data. [2025-09-16T18:02:41.719440] [DEBUG] [VALIDATOR ] DBRG: All expected trading days are present. [2025-09-16T18:02:41.719538] [DEBUG] [VALIDATOR ] DBRG: Primary data validation passed. [2025-09-16T18:02:41.720291] [INFO ] [MAIN ] DBRG: Fetched primary data is valid. [2025-09-16T18:02:41.720389] [DEBUG] [LOCAL ] DBRG: Saving primary data. [2025-09-16T18:02:41.721593] [DEBUG] [LOCAL ] DBRG: Trimmed 54306 records to 36985 (keeping 150 days of data) [2025-09-16T18:02:41.949442] [DEBUG] [LOCAL ] DBRG: Primary data saved (36985 records). [2025-09-16T18:02:41.949590] [INFO ] [MAIN ] DBRG: Full history saved. [2025-09-16T18:02:41.951155] [INFO ] [MAIN ] Update Progress: 595/2369 symbols completed. DBRG took 24.8s. Estimated remaining: 3h 6m 47s. Rate: 9.5 symbols/min [2025-09-16T18:02:41.981943] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-08. [2025-09-16T18:02:42.125353] [INFO ] [MAIN ] DDD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:02:42.164546] [DEBUG] [LOCAL ] DDD: Loaded 37346 existing records from primary storage. [2025-09-16T18:02:42.164709] [INFO ] [MAIN ] DDD: Loaded 37346 existing records. [2025-09-16T18:02:42.347171] [WARN ] [MAIN ] DDD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:02:42.347469] [INFO ] [MAIN ] DDD: Wiping existing primary data before full refresh. [2025-09-16T18:02:42.347517] [INFO ] [LOCAL ] DDD: Starting deletion of primary data. [2025-09-16T18:02:42.348330] [DEBUG] [LOCAL ] DDD: Deleted primary data file. [2025-09-16T18:02:42.348386] [INFO ] [MAIN ] DDD: Starting full primary refresh. [2025-09-16T18:02:42.734546] [INFO ] [REMOTE ] DDD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:02:42.774955] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-04. [2025-09-16T18:02:43.574973] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-02. [2025-09-16T18:02:43.778443] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-09-16T18:02:44.048010] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-09-16T18:02:44.374884] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-05. [2025-09-16T18:02:44.499468] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-03 (3.32 seconds). [2025-09-16T18:02:45.174847] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-04. [2025-09-16T18:02:45.190992] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T18:02:45.271132] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-09-16T18:02:45.974897] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-03. [2025-09-16T18:02:46.536200] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-09-16T18:02:46.774900] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-09. [2025-09-16T18:02:47.307525] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T18:02:47.574916] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-05. [2025-09-16T18:02:47.897064] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-09-16T18:02:47.902113] [INFO ] [REMOTE ] DCI: Full fetch complete (40928 records). [2025-09-16T18:02:47.902245] [DEBUG] [VALIDATOR ] DCI: Validating primary data. [2025-09-16T18:02:48.289512] [DEBUG] [VALIDATOR ] DCI: All expected trading days are present. [2025-09-16T18:02:48.289590] [DEBUG] [VALIDATOR ] DCI: Primary data validation passed. [2025-09-16T18:02:48.290143] [INFO ] [MAIN ] DCI: Fetched primary data is valid. [2025-09-16T18:02:48.290188] [DEBUG] [LOCAL ] DCI: Saving primary data. [2025-09-16T18:02:48.291092] [DEBUG] [LOCAL ] DCI: Trimmed 40928 records to 27450 (keeping 150 days of data) [2025-09-16T18:02:48.383224] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-04. [2025-09-16T18:02:48.462967] [DEBUG] [LOCAL ] DCI: Primary data saved (27450 records). [2025-09-16T18:02:48.463106] [INFO ] [MAIN ] DCI: Full history saved. [2025-09-16T18:02:48.463194] [INFO ] [MAIN ] Update Progress: 596/2369 symbols completed. DCI took 29.8s. Estimated remaining: 3h 6m 41s. Rate: 9.5 symbols/min [2025-09-16T18:02:48.632576] [INFO ] [MAIN ] DDL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:02:48.656830] [DEBUG] [LOCAL ] DDL: Loaded 25348 existing records from primary storage. [2025-09-16T18:02:48.656961] [INFO ] [MAIN ] DDL: Loaded 25348 existing records. [2025-09-16T18:02:48.836959] [WARN ] [MAIN ] DDL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:02:48.837380] [INFO ] [MAIN ] DDL: Wiping existing primary data before full refresh. [2025-09-16T18:02:48.837488] [INFO ] [LOCAL ] DDL: Starting deletion of primary data. [2025-09-16T18:02:48.838123] [DEBUG] [LOCAL ] DDL: Deleted primary data file. [2025-09-16T18:02:48.838176] [INFO ] [MAIN ] DDL: Starting full primary refresh. [2025-09-16T18:02:49.196579] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-06. [2025-09-16T18:02:49.223160] [INFO ] [REMOTE ] DDL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:02:49.228040] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T18:02:49.887282] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T18:02:49.909478] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-09-16T18:02:49.974864] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-05. [2025-09-16T18:02:50.774904] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-02. [2025-09-16T18:02:51.552778] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T18:02:51.574860] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-06. [2025-09-16T18:02:52.003778] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-09-16T18:02:52.374942] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-03. [2025-09-16T18:02:52.714695] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:02:53.174921] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-06. [2025-09-16T18:02:53.491604] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-09-16T18:02:53.974922] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-04. [2025-09-16T18:02:54.413347] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T18:02:54.774932] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-07. [2025-09-16T18:02:55.133334] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-09-16T18:02:55.574896] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-05. [2025-09-16T18:02:56.375006] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-07. [2025-09-16T18:02:56.533009] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T18:02:56.856461] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-09-16T18:02:57.174919] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-07. [2025-09-16T18:02:57.974893] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-06. [2025-09-16T18:02:58.028549] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T18:02:58.065267] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-09-16T18:02:58.719960] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-09-16T18:02:58.774866] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-07. [2025-09-16T18:02:59.574899] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-08. [2025-09-16T18:02:59.716546] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-09-16T18:03:00.374906] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-08. [2025-09-16T18:03:01.174918] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-08. [2025-09-16T18:03:01.384933] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-09-16T18:03:01.974921] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-09. [2025-09-16T18:03:02.774908] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-08. [2025-09-16T18:03:03.513663] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-08 (2.34 seconds). [2025-09-16T18:03:03.574848] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-09. [2025-09-16T18:03:04.006452] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-08 (3.63 seconds). [2025-09-16T18:03:04.374913] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-09. [2025-09-16T18:03:04.673143] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-09-16T18:03:05.174895] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-09. [2025-09-16T18:03:05.347788] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-09 (3.37 seconds). [2025-09-16T18:03:05.353888] [INFO ] [REMOTE ] DDD: Full fetch complete (55233 records). [2025-09-16T18:03:05.354028] [DEBUG] [VALIDATOR ] DDD: Validating primary data. [2025-09-16T18:03:05.739928] [DEBUG] [VALIDATOR ] DDD: All expected trading days are present. [2025-09-16T18:03:05.740041] [DEBUG] [VALIDATOR ] DDD: Primary data validation passed. [2025-09-16T18:03:05.740834] [INFO ] [MAIN ] DDD: Fetched primary data is valid. [2025-09-16T18:03:05.740885] [DEBUG] [LOCAL ] DDD: Saving primary data. [2025-09-16T18:03:05.742301] [DEBUG] [LOCAL ] DDD: Trimmed 55233 records to 37715 (keeping 150 days of data) [2025-09-16T18:03:05.954549] [DEBUG] [LOCAL ] DDD: Primary data saved (37715 records). [2025-09-16T18:03:05.954693] [INFO ] [MAIN ] DDD: Full history saved. [2025-09-16T18:03:05.955707] [INFO ] [MAIN ] Update Progress: 597/2369 symbols completed. DDD took 24.0s. Estimated remaining: 3h 7m 8s. Rate: 9.5 symbols/min [2025-09-16T18:03:06.125449] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T18:03:06.128101] [INFO ] [REMOTE ] DCO: Full fetch complete (14502 records). [2025-09-16T18:03:06.128178] [DEBUG] [VALIDATOR ] DCO: Validating primary data. [2025-09-16T18:03:06.128839] [FATAL] [VALIDATOR ] DCO: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:03:06.128916] [FATAL] [MAIN ] DCO: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:03:06.128953] [WARN ] [MAIN ] DCO: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:03:06.128994] [INFO ] [LOCAL ] DCO: Starting deletion of primary data. [2025-09-16T18:03:06.129051] [DEBUG] [LOCAL ] DCO: No primary data file found to delete. [2025-09-16T18:03:06.129086] [INFO ] [LOCAL ] DCO: Starting deletion of derivative data. [2025-09-16T18:03:06.129610] [DEBUG] [LOCAL ] DCO: Deleted derivative data file. [2025-09-16T18:03:06.129708] [INFO ] [MAIN ] Update Progress: 598/2369 symbols completed. DCO took 29.2s. Estimated remaining: 3h 6m 44s. Rate: 9.5 symbols/min [2025-09-16T18:03:06.461608] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-09 (2.89 seconds). [2025-09-16T18:03:06.473577] [INFO ] [REMOTE ] DD: Full fetch complete (57376 records). [2025-09-16T18:03:06.479111] [DEBUG] [VALIDATOR ] DD: Validating primary data. [2025-09-16T18:03:06.479988] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-09 (2.11 seconds). [2025-09-16T18:03:06.554264] [INFO ] [REMOTE ] DDL: Full fetch complete (39202 records). [2025-09-16T18:03:06.585790] [DEBUG] [VALIDATOR ] DDL: Validating primary data. [2025-09-16T18:03:06.589159] [FATAL] [VALIDATOR ] DDL: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:03:06.610318] [FATAL] [MAIN ] DDL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:03:06.647948] [WARN ] [MAIN ] DDL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:03:06.667835] [INFO ] [LOCAL ] DDL: Starting deletion of primary data. [2025-09-16T18:03:06.679439] [DEBUG] [LOCAL ] DDL: No primary data file found to delete. [2025-09-16T18:03:06.730103] [INFO ] [LOCAL ] DDL: Starting deletion of derivative data. [2025-09-16T18:03:06.768171] [DEBUG] [LOCAL ] DDL: Deleted derivative data file. [2025-09-16T18:03:06.787571] [INFO ] [MAIN ] Update Progress: 599/2369 symbols completed. DDL took 18.3s. Estimated remaining: 3h 6m 21s. Rate: 9.5 symbols/min [2025-09-16T18:03:06.878214] [DEBUG] [VALIDATOR ] DD: All expected trading days are present. [2025-09-16T18:03:06.878286] [DEBUG] [VALIDATOR ] DD: Primary data validation passed. [2025-09-16T18:03:06.879064] [INFO ] [MAIN ] DD: Fetched primary data is valid. [2025-09-16T18:03:06.879111] [DEBUG] [LOCAL ] DD: Saving primary data. [2025-09-16T18:03:06.880256] [DEBUG] [LOCAL ] DD: Trimmed 57376 records to 39373 (keeping 150 days of data) [2025-09-16T18:03:07.127751] [DEBUG] [LOCAL ] DD: Primary data saved (39373 records). [2025-09-16T18:03:07.127893] [INFO ] [MAIN ] DD: Full history saved. [2025-09-16T18:03:07.127977] [INFO ] [MAIN ] Update Progress: 600/2369 symbols completed. DD took 29.5s. Estimated remaining: 3h 5m 57s. Rate: 9.5 symbols/min [2025-09-16T18:03:07.737679] [INFO ] [MAIN ] DEA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:07.700366] [INFO ] [MAIN ] DDS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:07.775552] [INFO ] [MAIN ] DDT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:07.848365] [INFO ] [MAIN ] DE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:07.848730] [DEBUG] [LOCAL ] DE: No existing data file found in primary storage. [2025-09-16T18:03:07.848921] [WARN ] [MAIN ] DE: No existing data found, performing full refresh instead. [2025-09-16T18:03:07.849523] [INFO ] [MAIN ] DE: Wiping existing primary data before full refresh. [2025-09-16T18:03:07.851465] [DEBUG] [LOCAL ] DDT: Loaded 1355 existing records from primary storage. [2025-09-16T18:03:07.857549] [INFO ] [MAIN ] DDT: Loaded 1355 existing records. [2025-09-16T18:03:07.849736] [INFO ] [LOCAL ] DE: Starting deletion of primary data. [2025-09-16T18:03:07.874796] [DEBUG] [LOCAL ] DE: No primary data file found to delete. [2025-09-16T18:03:07.863690] [DEBUG] [LOCAL ] DDS: Loaded 9514 existing records from primary storage. [2025-09-16T18:03:07.880201] [INFO ] [MAIN ] DDS: Loaded 9514 existing records. [2025-09-16T18:03:07.888929] [INFO ] [MAIN ] DE: Starting full primary refresh. [2025-09-16T18:03:08.281654] [DEBUG] [LOCAL ] DEA: Loaded 29061 existing records from primary storage. [2025-09-16T18:03:08.312320] [WARN ] [MAIN ] DDT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:03:08.321328] [INFO ] [MAIN ] DEA: Loaded 29061 existing records. [2025-09-16T18:03:08.401201] [WARN ] [MAIN ] DDS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:03:08.412671] [INFO ] [MAIN ] DDT: Wiping existing primary data before full refresh. [2025-09-16T18:03:08.444616] [INFO ] [LOCAL ] DDT: Starting deletion of primary data. [2025-09-16T18:03:08.507272] [DEBUG] [LOCAL ] DDT: Deleted primary data file. [2025-09-16T18:03:08.521708] [INFO ] [MAIN ] DDT: Starting full primary refresh. [2025-09-16T18:03:08.493603] [INFO ] [MAIN ] DDS: Wiping existing primary data before full refresh. [2025-09-16T18:03:08.569730] [INFO ] [LOCAL ] DDS: Starting deletion of primary data. [2025-09-16T18:03:08.631070] [DEBUG] [LOCAL ] DDS: Deleted primary data file. [2025-09-16T18:03:08.647124] [INFO ] [MAIN ] DDS: Starting full primary refresh. [2025-09-16T18:03:09.002435] [INFO ] [REMOTE ] DE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:09.035555] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-02. [2025-09-16T18:03:09.035283] [WARN ] [MAIN ] DEA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:03:09.059714] [INFO ] [MAIN ] DEA: Wiping existing primary data before full refresh. [2025-09-16T18:03:09.084679] [INFO ] [LOCAL ] DEA: Starting deletion of primary data. [2025-09-16T18:03:09.203050] [DEBUG] [LOCAL ] DEA: Deleted primary data file. [2025-09-16T18:03:09.252685] [INFO ] [MAIN ] DEA: Starting full primary refresh. [2025-09-16T18:03:09.574802] [INFO ] [REMOTE ] DDT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:09.585439] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-02. [2025-09-16T18:03:09.661132] [INFO ] [REMOTE ] DDS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:09.684873] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-02. [2025-09-16T18:03:09.868800] [INFO ] [REMOTE ] DEA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:09.869102] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-02. [2025-09-16T18:03:10.248588] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-09-16T18:03:10.249637] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-03. [2025-09-16T18:03:10.549294] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-03 (0.30 seconds). [2025-09-16T18:03:10.550831] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-04. [2025-09-16T18:03:10.559458] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T18:03:10.774939] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-03. [2025-09-16T18:03:11.044993] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-09-16T18:03:11.235255] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T18:03:11.385568] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-09-16T18:03:11.574905] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-05. [2025-09-16T18:03:11.958220] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-09-16T18:03:12.375150] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-06. [2025-09-16T18:03:12.380594] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-02 (2.51 seconds). [2025-09-16T18:03:12.744540] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-09-16T18:03:13.174906] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-03. [2025-09-16T18:03:13.974914] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-07. [2025-09-16T18:03:14.482155] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-09-16T18:03:14.532940] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T18:03:14.774907] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-08. [2025-09-16T18:03:15.166515] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-09-16T18:03:15.574908] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-09. [2025-09-16T18:03:16.066373] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-09-16T18:03:16.067179] [INFO ] [REMOTE ] DDT: Full fetch complete (2198 records). [2025-09-16T18:03:16.067246] [DEBUG] [VALIDATOR ] DDT: Validating primary data. [2025-09-16T18:03:16.383764] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-04. [2025-09-16T18:03:16.420867] [DEBUG] [VALIDATOR ] DDT: All expected trading days are present. [2025-09-16T18:03:16.420949] [DEBUG] [VALIDATOR ] DDT: Primary data validation passed. [2025-09-16T18:03:16.421073] [INFO ] [MAIN ] DDT: Fetched primary data is valid. [2025-09-16T18:03:16.421113] [DEBUG] [LOCAL ] DDT: Saving primary data. [2025-09-16T18:03:16.421458] [DEBUG] [LOCAL ] DDT: Trimmed 2198 records to 1367 (keeping 150 days of data) [2025-09-16T18:03:16.430429] [DEBUG] [LOCAL ] DDT: Primary data saved (1367 records). [2025-09-16T18:03:16.430491] [INFO ] [MAIN ] DDT: Full history saved. [2025-09-16T18:03:16.430561] [INFO ] [MAIN ] Update Progress: 601/2369 symbols completed. DDT took 9.3s. Estimated remaining: 3h 5m 59s. Rate: 9.5 symbols/min [2025-09-16T18:03:16.599229] [INFO ] [MAIN ] DEC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:16.620778] [DEBUG] [LOCAL ] DEC: Loaded 21123 existing records from primary storage. [2025-09-16T18:03:16.620912] [INFO ] [MAIN ] DEC: Loaded 21123 existing records. [2025-09-16T18:03:16.796626] [WARN ] [MAIN ] DEC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:03:16.796926] [INFO ] [MAIN ] DEC: Wiping existing primary data before full refresh. [2025-09-16T18:03:16.796971] [INFO ] [LOCAL ] DEC: Starting deletion of primary data. [2025-09-16T18:03:16.797628] [DEBUG] [LOCAL ] DEC: Deleted primary data file. [2025-09-16T18:03:16.797697] [INFO ] [MAIN ] DEC: Starting full primary refresh. [2025-09-16T18:03:17.143958] [INFO ] [REMOTE ] DEC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:17.174846] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-02. [2025-09-16T18:03:17.573285] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T18:03:17.974889] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-05. [2025-09-16T18:03:18.440230] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T18:03:18.774907] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-03. [2025-09-16T18:03:18.918006] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-09-16T18:03:19.574900] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-04. [2025-09-16T18:03:20.198958] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T18:03:20.374863] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-06. [2025-09-16T18:03:21.174874] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-04. [2025-09-16T18:03:21.451769] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T18:03:21.562411] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-09-16T18:03:21.974903] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-03. [2025-09-16T18:03:22.546894] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T18:03:22.774883] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-05. [2025-09-16T18:03:23.574897] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-05. [2025-09-16T18:03:23.727788] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T18:03:23.793868] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-09-16T18:03:24.374903] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-06. [2025-09-16T18:03:25.174862] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-07. [2025-09-16T18:03:25.401959] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-09-16T18:03:25.618087] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T18:03:25.835566] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-09-16T18:03:25.974922] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-06. [2025-09-16T18:03:26.774899] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-08. [2025-09-16T18:03:27.520056] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-09-16T18:03:27.537780] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-09-16T18:03:27.574842] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-09. [2025-09-16T18:03:28.374948] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-07. [2025-09-16T18:03:28.502980] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T18:03:28.505368] [INFO ] [REMOTE ] DDS: Full fetch complete (14723 records). [2025-09-16T18:03:28.505504] [DEBUG] [VALIDATOR ] DDS: Validating primary data. [2025-09-16T18:03:28.506158] [FATAL] [VALIDATOR ] DDS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:03:28.506238] [FATAL] [MAIN ] DDS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:03:28.506282] [WARN ] [MAIN ] DDS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:03:28.506329] [INFO ] [LOCAL ] DDS: Starting deletion of primary data. [2025-09-16T18:03:28.506397] [DEBUG] [LOCAL ] DDS: No primary data file found to delete. [2025-09-16T18:03:28.506435] [INFO ] [LOCAL ] DDS: Starting deletion of derivative data. [2025-09-16T18:03:28.506993] [DEBUG] [LOCAL ] DDS: Deleted derivative data file. [2025-09-16T18:03:28.507314] [INFO ] [MAIN ] Update Progress: 602/2369 symbols completed. DDS took 21.4s. Estimated remaining: 3h 6m 10s. Rate: 9.5 symbols/min [2025-09-16T18:03:28.678400] [INFO ] [MAIN ] DECK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:28.720740] [DEBUG] [LOCAL ] DECK: Loaded 39340 existing records from primary storage. [2025-09-16T18:03:28.720948] [INFO ] [MAIN ] DECK: Loaded 39340 existing records. [2025-09-16T18:03:28.902951] [WARN ] [MAIN ] DECK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:03:28.903249] [INFO ] [MAIN ] DECK: Wiping existing primary data before full refresh. [2025-09-16T18:03:28.903296] [INFO ] [LOCAL ] DECK: Starting deletion of primary data. [2025-09-16T18:03:28.904361] [DEBUG] [LOCAL ] DECK: Deleted primary data file. [2025-09-16T18:03:28.904413] [INFO ] [MAIN ] DECK: Starting full primary refresh. [2025-09-16T18:03:29.179714] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-07. [2025-09-16T18:03:29.265695] [INFO ] [REMOTE ] DECK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:29.949830] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T18:03:29.974840] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-08. [2025-09-16T18:03:30.233338] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-09-16T18:03:30.774902] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-08. [2025-09-16T18:03:31.568692] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T18:03:31.578133] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-04. [2025-09-16T18:03:31.986309] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-09-16T18:03:32.374917] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-09. [2025-09-16T18:03:33.174909] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-02. [2025-09-16T18:03:33.564057] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-09-16T18:03:33.974897] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-05. [2025-09-16T18:03:34.149435] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-09-16T18:03:34.155923] [INFO ] [REMOTE ] DE: Full fetch complete (52306 records). [2025-09-16T18:03:34.156070] [DEBUG] [VALIDATOR ] DE: Validating primary data. [2025-09-16T18:03:34.552319] [DEBUG] [VALIDATOR ] DE: All expected trading days are present. [2025-09-16T18:03:34.552445] [DEBUG] [VALIDATOR ] DE: Primary data validation passed. [2025-09-16T18:03:34.553187] [INFO ] [MAIN ] DE: Fetched primary data is valid. [2025-09-16T18:03:34.553242] [DEBUG] [LOCAL ] DE: Saving primary data. [2025-09-16T18:03:34.554539] [DEBUG] [LOCAL ] DE: Trimmed 52306 records to 35191 (keeping 150 days of data) [2025-09-16T18:03:34.776299] [DEBUG] [LOCAL ] DE: Primary data saved (35191 records). [2025-09-16T18:03:34.776417] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-09. [2025-09-16T18:03:34.776615] [INFO ] [MAIN ] DE: Full history saved. [2025-09-16T18:03:34.777844] [INFO ] [MAIN ] Update Progress: 603/2369 symbols completed. DE took 27.6s. Estimated remaining: 3h 6m 3s. Rate: 9.5 symbols/min [2025-09-16T18:03:34.951354] [INFO ] [MAIN ] DEI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:34.986573] [DEBUG] [LOCAL ] DEI: Loaded 35546 existing records from primary storage. [2025-09-16T18:03:34.986743] [INFO ] [MAIN ] DEI: Loaded 35546 existing records. [2025-09-16T18:03:35.169032] [WARN ] [MAIN ] DEI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:03:35.169361] [INFO ] [MAIN ] DEI: Wiping existing primary data before full refresh. [2025-09-16T18:03:35.169413] [INFO ] [LOCAL ] DEI: Starting deletion of primary data. [2025-09-16T18:03:35.170301] [DEBUG] [LOCAL ] DEI: Deleted primary data file. [2025-09-16T18:03:35.170351] [INFO ] [MAIN ] DEI: Starting full primary refresh. [2025-09-16T18:03:35.534600] [INFO ] [REMOTE ] DEI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:35.564842] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-02 (2.39 seconds). [2025-09-16T18:03:35.575356] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-02. [2025-09-16T18:03:35.599803] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T18:03:35.837786] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-09-16T18:03:35.841042] [INFO ] [REMOTE ] DEC: Full fetch complete (34004 records). [2025-09-16T18:03:35.841139] [DEBUG] [VALIDATOR ] DEC: Validating primary data. [2025-09-16T18:03:36.239605] [DEBUG] [VALIDATOR ] DEC: All expected trading days are present. [2025-09-16T18:03:36.239708] [DEBUG] [VALIDATOR ] DEC: Primary data validation passed. [2025-09-16T18:03:36.240215] [INFO ] [MAIN ] DEC: Fetched primary data is valid. [2025-09-16T18:03:36.240257] [DEBUG] [LOCAL ] DEC: Saving primary data. [2025-09-16T18:03:36.241083] [DEBUG] [LOCAL ] DEC: Trimmed 34004 records to 21218 (keeping 150 days of data) [2025-09-16T18:03:36.373814] [DEBUG] [LOCAL ] DEC: Primary data saved (21218 records). [2025-09-16T18:03:36.373969] [INFO ] [MAIN ] DEC: Full history saved. [2025-09-16T18:03:36.374052] [INFO ] [MAIN ] Update Progress: 604/2369 symbols completed. DEC took 19.9s. Estimated remaining: 3h 5m 43s. Rate: 9.5 symbols/min [2025-09-16T18:03:36.385598] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-06. [2025-09-16T18:03:36.551599] [INFO ] [MAIN ] DELL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:36.552279] [DEBUG] [LOCAL ] DELL: No existing data file found in primary storage. [2025-09-16T18:03:36.552322] [WARN ] [MAIN ] DELL: No existing data found, performing full refresh instead. [2025-09-16T18:03:36.552499] [INFO ] [MAIN ] DELL: Wiping existing primary data before full refresh. [2025-09-16T18:03:36.552543] [INFO ] [LOCAL ] DELL: Starting deletion of primary data. [2025-09-16T18:03:36.552586] [DEBUG] [LOCAL ] DELL: No primary data file found to delete. [2025-09-16T18:03:36.552619] [INFO ] [MAIN ] DELL: Starting full primary refresh. [2025-09-16T18:03:36.906256] [INFO ] [REMOTE ] DELL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:36.960737] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T18:03:37.174924] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-02. [2025-09-16T18:03:37.974905] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-03. [2025-09-16T18:03:38.141961] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T18:03:38.774925] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-07. [2025-09-16T18:03:39.432941] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-02 (2.26 seconds). [2025-09-16T18:03:39.574892] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-03. [2025-09-16T18:03:39.625747] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-09-16T18:03:39.690445] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-09-16T18:03:40.374943] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-04. [2025-09-16T18:03:41.174930] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-08. [2025-09-16T18:03:41.960723] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-03 (2.39 seconds). [2025-09-16T18:03:41.974844] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-04. [2025-09-16T18:03:42.181979] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T18:03:42.564083] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-09-16T18:03:42.774915] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-05. [2025-09-16T18:03:43.574918] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-03. [2025-09-16T18:03:43.953703] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-09-16T18:03:44.374941] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-05. [2025-09-16T18:03:44.511295] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T18:03:45.174961] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-06. [2025-09-16T18:03:45.646719] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-09-16T18:03:45.974927] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-09. [2025-09-16T18:03:46.605149] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-05 (2.23 seconds). [2025-09-16T18:03:46.666892] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T18:03:46.774877] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-07. [2025-09-16T18:03:47.521288] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-09-16T18:03:47.526867] [INFO ] [REMOTE ] DEA: Full fetch complete (46144 records). [2025-09-16T18:03:47.526987] [DEBUG] [VALIDATOR ] DEA: Validating primary data. [2025-09-16T18:03:47.580049] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-04. [2025-09-16T18:03:47.915675] [DEBUG] [VALIDATOR ] DEA: All expected trading days are present. [2025-09-16T18:03:47.915757] [DEBUG] [VALIDATOR ] DEA: Primary data validation passed. [2025-09-16T18:03:47.916397] [INFO ] [MAIN ] DEA: Fetched primary data is valid. [2025-09-16T18:03:47.916440] [DEBUG] [LOCAL ] DEA: Saving primary data. [2025-09-16T18:03:47.917434] [DEBUG] [LOCAL ] DEA: Trimmed 46144 records to 29272 (keeping 150 days of data) [2025-09-16T18:03:48.092836] [DEBUG] [LOCAL ] DEA: Primary data saved (29272 records). [2025-09-16T18:03:48.092982] [INFO ] [MAIN ] DEA: Full history saved. [2025-09-16T18:03:48.093082] [INFO ] [MAIN ] Update Progress: 605/2369 symbols completed. DEA took 41.0s. Estimated remaining: 3h 5m 53s. Rate: 9.5 symbols/min [2025-09-16T18:03:48.270365] [INFO ] [MAIN ] DEO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:48.299773] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-09-16T18:03:48.340111] [DEBUG] [LOCAL ] DEO: Loaded 35760 existing records from primary storage. [2025-09-16T18:03:48.340249] [INFO ] [MAIN ] DEO: Loaded 35760 existing records. [2025-09-16T18:03:48.382324] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-06. [2025-09-16T18:03:48.518996] [WARN ] [MAIN ] DEO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:03:48.519233] [INFO ] [MAIN ] DEO: Wiping existing primary data before full refresh. [2025-09-16T18:03:48.519282] [INFO ] [LOCAL ] DEO: Starting deletion of primary data. [2025-09-16T18:03:48.520168] [DEBUG] [LOCAL ] DEO: Deleted primary data file. [2025-09-16T18:03:48.520219] [INFO ] [MAIN ] DEO: Starting full primary refresh. [2025-09-16T18:03:48.872948] [INFO ] [REMOTE ] DEO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:49.174901] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-08. [2025-09-16T18:03:49.562345] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-09-16T18:03:49.974957] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-05. [2025-09-16T18:03:50.259964] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-09-16T18:03:50.774938] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-02. [2025-09-16T18:03:50.974170] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-06 (2.59 seconds). [2025-09-16T18:03:51.574954] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-09. [2025-09-16T18:03:51.930907] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-09-16T18:03:52.374972] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-06. [2025-09-16T18:03:52.532532] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-09-16T18:03:53.174867] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-03. [2025-09-16T18:03:53.851552] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-09 (2.28 seconds). [2025-09-16T18:03:53.859008] [INFO ] [REMOTE ] DEI: Full fetch complete (52134 records). [2025-09-16T18:03:53.859254] [DEBUG] [VALIDATOR ] DEI: Validating primary data. [2025-09-16T18:03:53.870979] [FATAL] [VALIDATOR ] DEI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:03:53.889572] [FATAL] [MAIN ] DEI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:03:53.889931] [WARN ] [MAIN ] DEI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:03:53.891809] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T18:03:53.890459] [INFO ] [LOCAL ] DEI: Starting deletion of primary data. [2025-09-16T18:03:53.901989] [DEBUG] [LOCAL ] DEI: No primary data file found to delete. [2025-09-16T18:03:53.902434] [INFO ] [LOCAL ] DEI: Starting deletion of derivative data. [2025-09-16T18:03:53.903042] [DEBUG] [LOCAL ] DEI: Deleted derivative data file. [2025-09-16T18:03:53.904973] [INFO ] [MAIN ] Update Progress: 606/2369 symbols completed. DEI took 19.1s. Estimated remaining: 3h 5m 45s. Rate: 9.5 symbols/min [2025-09-16T18:03:53.983622] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-07. [2025-09-16T18:03:54.084110] [INFO ] [MAIN ] DFH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:03:54.110477] [DEBUG] [LOCAL ] DFH: Loaded 23634 existing records from primary storage. [2025-09-16T18:03:54.110626] [INFO ] [MAIN ] DFH: Loaded 23634 existing records. [2025-09-16T18:03:54.291094] [WARN ] [MAIN ] DFH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:03:54.291372] [INFO ] [MAIN ] DFH: Wiping existing primary data before full refresh. [2025-09-16T18:03:54.291613] [INFO ] [LOCAL ] DFH: Starting deletion of primary data. [2025-09-16T18:03:54.292220] [DEBUG] [LOCAL ] DFH: Deleted primary data file. [2025-09-16T18:03:54.292269] [INFO ] [MAIN ] DFH: Starting full primary refresh. [2025-09-16T18:03:54.652562] [INFO ] [REMOTE ] DFH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:03:54.695723] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T18:03:54.774862] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-04. [2025-09-16T18:03:55.574938] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-07. [2025-09-16T18:03:55.792993] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T18:03:56.318250] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-09-16T18:03:56.374970] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-05. [2025-09-16T18:03:57.174945] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-02. [2025-09-16T18:03:57.494528] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T18:03:57.688842] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T18:03:57.974902] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-06. [2025-09-16T18:03:58.268510] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T18:03:58.774910] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-03. [2025-09-16T18:03:59.457874] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T18:03:59.574904] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-07. [2025-09-16T18:03:59.826123] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-09-16T18:04:00.374905] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-04. [2025-09-16T18:04:01.174926] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-08. [2025-09-16T18:04:01.610743] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-09-16T18:04:01.974944] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-08. [2025-09-16T18:04:02.226670] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-09-16T18:04:02.774908] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-05. [2025-09-16T18:04:03.520505] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-08 (2.35 seconds). [2025-09-16T18:04:03.574862] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-09. [2025-09-16T18:04:03.677686] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-09-16T18:04:03.719777] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-09-16T18:04:04.374930] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-08. [2025-09-16T18:04:05.119377] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T18:04:05.125675] [INFO ] [REMOTE ] DECK: Full fetch complete (57994 records). [2025-09-16T18:04:05.125823] [DEBUG] [VALIDATOR ] DECK: Validating primary data. [2025-09-16T18:04:05.185579] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-09. [2025-09-16T18:04:05.535241] [DEBUG] [VALIDATOR ] DECK: All expected trading days are present. [2025-09-16T18:04:05.535352] [DEBUG] [VALIDATOR ] DECK: Primary data validation passed. [2025-09-16T18:04:05.536186] [INFO ] [MAIN ] DECK: Fetched primary data is valid. [2025-09-16T18:04:05.536262] [DEBUG] [LOCAL ] DECK: Saving primary data. [2025-09-16T18:04:05.537770] [DEBUG] [LOCAL ] DECK: Trimmed 57994 records to 39727 (keeping 150 days of data) [2025-09-16T18:04:05.779865] [DEBUG] [LOCAL ] DECK: Primary data saved (39727 records). [2025-09-16T18:04:05.780018] [INFO ] [MAIN ] DECK: Full history saved. [2025-09-16T18:04:05.781024] [INFO ] [MAIN ] Update Progress: 607/2369 symbols completed. DECK took 37.3s. Estimated remaining: 3h 5m 55s. Rate: 9.5 symbols/min [2025-09-16T18:04:05.962528] [INFO ] [MAIN ] DFIN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:04:05.974900] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-06. [2025-09-16T18:04:05.982383] [DEBUG] [LOCAL ] DFIN: Loaded 15977 existing records from primary storage. [2025-09-16T18:04:05.982522] [INFO ] [MAIN ] DFIN: Loaded 15977 existing records. [2025-09-16T18:04:06.165327] [WARN ] [MAIN ] DFIN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:04:06.165595] [INFO ] [MAIN ] DFIN: Wiping existing primary data before full refresh. [2025-09-16T18:04:06.165663] [INFO ] [LOCAL ] DFIN: Starting deletion of primary data. [2025-09-16T18:04:06.166102] [DEBUG] [LOCAL ] DFIN: Deleted primary data file. [2025-09-16T18:04:06.166149] [INFO ] [MAIN ] DFIN: Starting full primary refresh. [2025-09-16T18:04:06.528061] [INFO ] [REMOTE ] DFIN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:04:06.545246] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T18:04:06.551375] [INFO ] [REMOTE ] DEO: Full fetch complete (53252 records). [2025-09-16T18:04:06.551621] [DEBUG] [VALIDATOR ] DEO: Validating primary data. [2025-09-16T18:04:06.690668] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-08 (2.32 seconds). [2025-09-16T18:04:06.790399] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-02. [2025-09-16T18:04:06.980405] [DEBUG] [VALIDATOR ] DEO: All expected trading days are present. [2025-09-16T18:04:06.980493] [DEBUG] [VALIDATOR ] DEO: Primary data validation passed. [2025-09-16T18:04:06.981199] [INFO ] [MAIN ] DEO: Fetched primary data is valid. [2025-09-16T18:04:06.981248] [DEBUG] [LOCAL ] DEO: Saving primary data. [2025-09-16T18:04:06.982440] [DEBUG] [LOCAL ] DEO: Trimmed 53252 records to 36147 (keeping 150 days of data) [2025-09-16T18:04:07.209401] [DEBUG] [LOCAL ] DEO: Primary data saved (36147 records). [2025-09-16T18:04:07.209543] [INFO ] [MAIN ] DEO: Full history saved. [2025-09-16T18:04:07.209654] [INFO ] [MAIN ] Update Progress: 608/2369 symbols completed. DEO took 19.1s. Estimated remaining: 3h 5m 34s. Rate: 9.5 symbols/min [2025-09-16T18:04:07.389907] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T18:04:07.415101] [INFO ] [MAIN ] DFP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:04:07.415818] [DEBUG] [LOCAL ] DFP: No existing data file found in primary storage. [2025-09-16T18:04:07.415863] [WARN ] [MAIN ] DFP: No existing data found, performing full refresh instead. [2025-09-16T18:04:07.416413] [INFO ] [MAIN ] DFP: Wiping existing primary data before full refresh. [2025-09-16T18:04:07.416500] [INFO ] [LOCAL ] DFP: Starting deletion of primary data. [2025-09-16T18:04:07.416550] [DEBUG] [LOCAL ] DFP: No primary data file found to delete. [2025-09-16T18:04:07.416583] [INFO ] [MAIN ] DFP: Starting full primary refresh. [2025-09-16T18:04:07.590440] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-09. [2025-09-16T18:04:07.780819] [INFO ] [REMOTE ] DFP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:04:07.796325] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-09-16T18:04:08.374947] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-03. [2025-09-16T18:04:09.174889] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-07. [2025-09-16T18:04:09.499984] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-09-16T18:04:09.505980] [INFO ] [REMOTE ] DELL: Full fetch complete (58026 records). [2025-09-16T18:04:09.506134] [DEBUG] [VALIDATOR ] DELL: Validating primary data. [2025-09-16T18:04:09.506948] [FATAL] [VALIDATOR ] DELL: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:04:09.507036] [FATAL] [MAIN ] DELL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:04:09.507075] [WARN ] [MAIN ] DELL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:04:09.507121] [INFO ] [LOCAL ] DELL: Starting deletion of primary data. [2025-09-16T18:04:09.507180] [DEBUG] [LOCAL ] DELL: No primary data file found to delete. [2025-09-16T18:04:09.507230] [INFO ] [LOCAL ] DELL: Starting deletion of derivative data. [2025-09-16T18:04:09.507283] [DEBUG] [LOCAL ] DELL: No derivative data file found to delete. [2025-09-16T18:04:09.507348] [INFO ] [MAIN ] Update Progress: 609/2369 symbols completed. DELL took 33.1s. Estimated remaining: 3h 5m 16s. Rate: 9.5 symbols/min [2025-09-16T18:04:09.689407] [INFO ] [MAIN ] DG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:04:09.707199] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T18:04:09.748671] [DEBUG] [LOCAL ] DG: Loaded 39311 existing records from primary storage. [2025-09-16T18:04:09.748821] [INFO ] [MAIN ] DG: Loaded 39311 existing records. [2025-09-16T18:04:09.936498] [WARN ] [MAIN ] DG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:04:09.936787] [INFO ] [MAIN ] DG: Wiping existing primary data before full refresh. [2025-09-16T18:04:09.936833] [INFO ] [LOCAL ] DG: Starting deletion of primary data. [2025-09-16T18:04:09.937740] [DEBUG] [LOCAL ] DG: Deleted primary data file. [2025-09-16T18:04:09.937828] [INFO ] [MAIN ] DG: Starting full primary refresh. [2025-09-16T18:04:09.982517] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-02. [2025-09-16T18:04:10.310194] [INFO ] [REMOTE ] DG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:04:10.774892] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-04. [2025-09-16T18:04:10.835804] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-09-16T18:04:10.895815] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-09-16T18:04:11.574909] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-03. [2025-09-16T18:04:11.969035] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T18:04:12.374910] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-08. [2025-09-16T18:04:12.450837] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-09-16T18:04:13.174907] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-05. [2025-09-16T18:04:13.674193] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-09-16T18:04:13.974916] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-09. [2025-09-16T18:04:14.319940] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T18:04:14.774941] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-04. [2025-09-16T18:04:14.993293] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T18:04:14.997069] [INFO ] [REMOTE ] DFH: Full fetch complete (36003 records). [2025-09-16T18:04:14.997176] [DEBUG] [VALIDATOR ] DFH: Validating primary data. [2025-09-16T18:04:14.997787] [FATAL] [VALIDATOR ] DFH: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:04:14.997867] [FATAL] [MAIN ] DFH: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:04:14.997907] [WARN ] [MAIN ] DFH: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:04:14.997948] [INFO ] [LOCAL ] DFH: Starting deletion of primary data. [2025-09-16T18:04:14.998009] [DEBUG] [LOCAL ] DFH: No primary data file found to delete. [2025-09-16T18:04:14.998047] [INFO ] [LOCAL ] DFH: Starting deletion of derivative data. [2025-09-16T18:04:14.998284] [DEBUG] [LOCAL ] DFH: Deleted derivative data file. [2025-09-16T18:04:14.998367] [INFO ] [MAIN ] Update Progress: 610/2369 symbols completed. DFH took 21.1s. Estimated remaining: 3h 5m 7s. Rate: 9.5 symbols/min [2025-09-16T18:04:15.179239] [INFO ] [MAIN ] DGX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:04:15.214917] [DEBUG] [LOCAL ] DGX: Loaded 32994 existing records from primary storage. [2025-09-16T18:04:15.215089] [INFO ] [MAIN ] DGX: Loaded 32994 existing records. [2025-09-16T18:04:15.405442] [WARN ] [MAIN ] DGX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:04:15.406375] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-09-16T18:04:15.406711] [INFO ] [MAIN ] DGX: Wiping existing primary data before full refresh. [2025-09-16T18:04:15.409503] [INFO ] [LOCAL ] DGX: Starting deletion of primary data. [2025-09-16T18:04:15.410986] [DEBUG] [LOCAL ] DGX: Deleted primary data file. [2025-09-16T18:04:15.411057] [INFO ] [MAIN ] DGX: Starting full primary refresh. [2025-09-16T18:04:15.595106] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-06. [2025-09-16T18:04:15.776694] [INFO ] [REMOTE ] DGX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:04:16.374874] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-05. [2025-09-16T18:04:16.484786] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T18:04:17.174906] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-02. [2025-09-16T18:04:17.203131] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T18:04:17.974902] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-07. [2025-09-16T18:04:18.740377] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T18:04:18.774869] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-03. [2025-09-16T18:04:18.873301] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-09-16T18:04:19.574915] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-08. [2025-09-16T18:04:20.374906] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-02. [2025-09-16T18:04:20.505076] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T18:04:20.548256] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-09-16T18:04:21.174910] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-04. [2025-09-16T18:04:21.974888] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-09. [2025-09-16T18:04:22.217035] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-09-16T18:04:22.774904] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-03. [2025-09-16T18:04:22.941219] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T18:04:22.944180] [INFO ] [REMOTE ] DFIN: Full fetch complete (25898 records). [2025-09-16T18:04:22.944295] [DEBUG] [VALIDATOR ] DFIN: Validating primary data. [2025-09-16T18:04:22.944910] [FATAL] [VALIDATOR ] DFIN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:04:22.944993] [FATAL] [MAIN ] DFIN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:04:22.945031] [WARN ] [MAIN ] DFIN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:04:22.945084] [INFO ] [LOCAL ] DFIN: Starting deletion of primary data. [2025-09-16T18:04:22.945159] [DEBUG] [LOCAL ] DFIN: No primary data file found to delete. [2025-09-16T18:04:22.945208] [INFO ] [LOCAL ] DFIN: Starting deletion of derivative data. [2025-09-16T18:04:22.945518] [DEBUG] [LOCAL ] DFIN: Deleted derivative data file. [2025-09-16T18:04:22.945672] [INFO ] [MAIN ] Update Progress: 611/2369 symbols completed. DFIN took 17.2s. Estimated remaining: 3h 5m 6s. Rate: 9.5 symbols/min [2025-09-16T18:04:23.132719] [INFO ] [MAIN ] DHF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:04:23.170482] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-09-16T18:04:23.186559] [DEBUG] [LOCAL ] DHF: Loaded 15803 existing records from primary storage. [2025-09-16T18:04:23.186951] [INFO ] [MAIN ] DHF: Loaded 15803 existing records. [2025-09-16T18:04:23.371580] [WARN ] [MAIN ] DHF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:04:23.371858] [INFO ] [MAIN ] DHF: Wiping existing primary data before full refresh. [2025-09-16T18:04:23.371932] [INFO ] [LOCAL ] DHF: Starting deletion of primary data. [2025-09-16T18:04:23.372381] [DEBUG] [LOCAL ] DHF: Deleted primary data file. [2025-09-16T18:04:23.372431] [INFO ] [MAIN ] DHF: Starting full primary refresh. [2025-09-16T18:04:23.586020] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-05. [2025-09-16T18:04:23.738769] [INFO ] [REMOTE ] DHF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:04:24.374861] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-02. [2025-09-16T18:04:24.493783] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T18:04:25.185603] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-04. [2025-09-16T18:04:25.190750] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-09-16T18:04:25.449174] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-09-16T18:04:25.974886] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-06. [2025-09-16T18:04:26.774945] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-03. [2025-09-16T18:04:27.004540] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T18:04:27.091636] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T18:04:27.574910] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-07. [2025-09-16T18:04:27.817881] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-09-16T18:04:28.374931] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-05. [2025-09-16T18:04:29.174920] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-04. [2025-09-16T18:04:29.512597] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-09-16T18:04:29.974864] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-08. [2025-09-16T18:04:30.279830] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-09-16T18:04:30.537446] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-05 (2.16 seconds). [2025-09-16T18:04:30.774959] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-06. [2025-09-16T18:04:31.353068] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T18:04:31.574935] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-09. [2025-09-16T18:04:32.290246] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T18:04:32.374869] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-07. [2025-09-16T18:04:32.912727] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T18:04:32.917381] [INFO ] [REMOTE ] DGX: Full fetch complete (49577 records). [2025-09-16T18:04:32.917494] [DEBUG] [VALIDATOR ] DGX: Validating primary data. [2025-09-16T18:04:32.918199] [FATAL] [VALIDATOR ] DGX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:04:32.918287] [FATAL] [MAIN ] DGX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:04:32.918332] [WARN ] [MAIN ] DGX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:04:32.918385] [INFO ] [LOCAL ] DGX: Starting deletion of primary data. [2025-09-16T18:04:32.918444] [DEBUG] [LOCAL ] DGX: No primary data file found to delete. [2025-09-16T18:04:32.918483] [INFO ] [LOCAL ] DGX: Starting deletion of derivative data. [2025-09-16T18:04:32.918727] [DEBUG] [LOCAL ] DGX: Deleted derivative data file. [2025-09-16T18:04:32.918810] [INFO ] [MAIN ] Update Progress: 612/2369 symbols completed. DGX took 17.9s. Estimated remaining: 3h 5m 10s. Rate: 9.5 symbols/min [2025-09-16T18:04:33.088025] [INFO ] [MAIN ] DHI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:04:33.130888] [DEBUG] [LOCAL ] DHI: Loaded 39418 existing records from primary storage. [2025-09-16T18:04:33.131028] [INFO ] [MAIN ] DHI: Loaded 39418 existing records. [2025-09-16T18:04:33.186161] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-05. [2025-09-16T18:04:33.318684] [WARN ] [MAIN ] DHI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:04:33.318965] [INFO ] [MAIN ] DHI: Wiping existing primary data before full refresh. [2025-09-16T18:04:33.319014] [INFO ] [LOCAL ] DHI: Starting deletion of primary data. [2025-09-16T18:04:33.320022] [DEBUG] [LOCAL ] DHI: Deleted primary data file. [2025-09-16T18:04:33.320079] [INFO ] [MAIN ] DHI: Starting full primary refresh. [2025-09-16T18:04:33.673328] [INFO ] [REMOTE ] DHI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:04:33.974881] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-06. [2025-09-16T18:04:34.196599] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T18:04:34.232291] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-09-16T18:04:34.774908] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-08. [2025-09-16T18:04:35.007205] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-09-16T18:04:35.574912] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-07. [2025-09-16T18:04:36.366474] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T18:04:36.376926] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-02. [2025-09-16T18:04:36.445711] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T18:04:37.174933] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-06. [2025-09-16T18:04:37.974872] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-08. [2025-09-16T18:04:38.092614] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-09-16T18:04:38.302507] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-09-16T18:04:38.774896] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-03. [2025-09-16T18:04:38.810356] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-09-16T18:04:39.574906] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-09. [2025-09-16T18:04:40.374894] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-07. [2025-09-16T18:04:40.554353] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T18:04:40.556564] [INFO ] [REMOTE ] DFP: Full fetch complete (12830 records). [2025-09-16T18:04:40.556632] [DEBUG] [VALIDATOR ] DFP: Validating primary data. [2025-09-16T18:04:40.965326] [DEBUG] [VALIDATOR ] DFP: All expected trading days are present. [2025-09-16T18:04:40.965423] [DEBUG] [VALIDATOR ] DFP: Primary data validation passed. [2025-09-16T18:04:40.965854] [INFO ] [MAIN ] DFP: Fetched primary data is valid. [2025-09-16T18:04:40.965966] [DEBUG] [LOCAL ] DFP: Saving primary data. [2025-09-16T18:04:40.966622] [DEBUG] [LOCAL ] DFP: Trimmed 12830 records to 8662 (keeping 150 days of data) [2025-09-16T18:04:41.039271] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-03 (2.26 seconds). [2025-09-16T18:04:41.064454] [DEBUG] [LOCAL ] DFP: Primary data saved (8662 records). [2025-09-16T18:04:41.064572] [INFO ] [MAIN ] DFP: Full history saved. [2025-09-16T18:04:41.064663] [INFO ] [MAIN ] Update Progress: 613/2369 symbols completed. DFP took 33.9s. Estimated remaining: 3h 5m 9s. Rate: 9.5 symbols/min [2025-09-16T18:04:41.176635] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-04. [2025-09-16T18:04:41.283622] [INFO ] [MAIN ] DHR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:04:41.284773] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-09-16T18:04:41.334342] [DEBUG] [LOCAL ] DHR: Loaded 39395 existing records from primary storage. [2025-09-16T18:04:41.334492] [INFO ] [MAIN ] DHR: Loaded 39395 existing records. [2025-09-16T18:04:41.516755] [WARN ] [MAIN ] DHR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:04:41.517021] [INFO ] [MAIN ] DHR: Wiping existing primary data before full refresh. [2025-09-16T18:04:41.517069] [INFO ] [LOCAL ] DHR: Starting deletion of primary data. [2025-09-16T18:04:41.517928] [DEBUG] [LOCAL ] DHR: Deleted primary data file. [2025-09-16T18:04:41.518027] [INFO ] [MAIN ] DHR: Starting full primary refresh. [2025-09-16T18:04:41.884863] [INFO ] [REMOTE ] DHR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:04:41.974869] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-02. [2025-09-16T18:04:42.774895] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-09. [2025-09-16T18:04:42.909846] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T18:04:43.166332] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T18:04:43.574924] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-08. [2025-09-16T18:04:44.374917] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-03. [2025-09-16T18:04:44.426082] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-09-16T18:04:44.661831] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-09-16T18:04:44.669662] [INFO ] [REMOTE ] DG: Full fetch complete (57958 records). [2025-09-16T18:04:44.669810] [DEBUG] [VALIDATOR ] DG: Validating primary data. [2025-09-16T18:04:45.055497] [DEBUG] [VALIDATOR ] DG: All expected trading days are present. [2025-09-16T18:04:45.055581] [DEBUG] [VALIDATOR ] DG: Primary data validation passed. [2025-09-16T18:04:45.056402] [INFO ] [MAIN ] DG: Fetched primary data is valid. [2025-09-16T18:04:45.056462] [DEBUG] [LOCAL ] DG: Saving primary data. [2025-09-16T18:04:45.058786] [DEBUG] [LOCAL ] DG: Trimmed 57958 records to 39688 (keeping 150 days of data) [2025-09-16T18:04:45.177743] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-05. [2025-09-16T18:04:45.312492] [DEBUG] [LOCAL ] DG: Primary data saved (39688 records). [2025-09-16T18:04:45.312655] [INFO ] [MAIN ] DG: Full history saved. [2025-09-16T18:04:45.312767] [INFO ] [MAIN ] Update Progress: 614/2369 symbols completed. DG took 35.8s. Estimated remaining: 3h 4m 57s. Rate: 9.5 symbols/min [2025-09-16T18:04:45.483848] [INFO ] [MAIN ] DHT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:04:45.484036] [DEBUG] [LOCAL ] DHT: No existing data file found in primary storage. [2025-09-16T18:04:45.484075] [WARN ] [MAIN ] DHT: No existing data found, performing full refresh instead. [2025-09-16T18:04:45.484206] [INFO ] [MAIN ] DHT: Wiping existing primary data before full refresh. [2025-09-16T18:04:45.484246] [INFO ] [LOCAL ] DHT: Starting deletion of primary data. [2025-09-16T18:04:45.484288] [DEBUG] [LOCAL ] DHT: No primary data file found to delete. [2025-09-16T18:04:45.484321] [INFO ] [MAIN ] DHT: Starting full primary refresh. [2025-09-16T18:04:45.845169] [INFO ] [REMOTE ] DHT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:04:45.974866] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-02. [2025-09-16T18:04:46.088421] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T18:04:46.774914] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-04. [2025-09-16T18:04:46.985581] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T18:04:47.574893] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-09. [2025-09-16T18:04:47.608531] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T18:04:48.374954] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-06. [2025-09-16T18:04:48.607700] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T18:04:48.773250] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-09-16T18:04:48.777309] [INFO ] [REMOTE ] DHF: Full fetch complete (22577 records). [2025-09-16T18:04:48.777492] [DEBUG] [VALIDATOR ] DHF: Validating primary data. [2025-09-16T18:04:49.180084] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-03. [2025-09-16T18:04:49.359420] [DEBUG] [VALIDATOR ] DHF: All expected trading days are present. [2025-09-16T18:04:49.359509] [DEBUG] [VALIDATOR ] DHF: Primary data validation passed. [2025-09-16T18:04:49.360022] [INFO ] [MAIN ] DHF: Fetched primary data is valid. [2025-09-16T18:04:49.360068] [DEBUG] [LOCAL ] DHF: Saving primary data. [2025-09-16T18:04:49.361003] [DEBUG] [LOCAL ] DHF: Trimmed 22577 records to 15920 (keeping 150 days of data) [2025-09-16T18:04:49.464678] [DEBUG] [LOCAL ] DHF: Primary data saved (15920 records). [2025-09-16T18:04:49.464843] [INFO ] [MAIN ] DHF: Full history saved. [2025-09-16T18:04:49.464957] [INFO ] [MAIN ] Update Progress: 615/2369 symbols completed. DHF took 26.5s. Estimated remaining: 3h 4m 44s. Rate: 9.5 symbols/min [2025-09-16T18:04:49.638032] [INFO ] [MAIN ] DHX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:04:49.647610] [DEBUG] [LOCAL ] DHX: Loaded 9039 existing records from primary storage. [2025-09-16T18:04:49.647726] [INFO ] [MAIN ] DHX: Loaded 9039 existing records. [2025-09-16T18:04:49.823708] [WARN ] [MAIN ] DHX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:04:49.824229] [INFO ] [MAIN ] DHX: Wiping existing primary data before full refresh. [2025-09-16T18:04:49.824337] [INFO ] [LOCAL ] DHX: Starting deletion of primary data. [2025-09-16T18:04:49.824776] [DEBUG] [LOCAL ] DHX: Deleted primary data file. [2025-09-16T18:04:49.824876] [INFO ] [MAIN ] DHX: Starting full primary refresh. [2025-09-16T18:04:49.982603] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-05. [2025-09-16T18:04:50.218610] [INFO ] [REMOTE ] DHX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:04:50.224594] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-09-16T18:04:50.774934] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-02. [2025-09-16T18:04:50.904778] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T18:04:51.574871] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-04. [2025-09-16T18:04:51.939219] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-09-16T18:04:52.049930] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-09-16T18:04:52.374889] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-03. [2025-09-16T18:04:53.179982] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-06. [2025-09-16T18:04:53.198068] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T18:04:53.242299] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-09-16T18:04:53.974887] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-04. [2025-09-16T18:04:54.774916] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-07. [2025-09-16T18:04:54.919035] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-09-16T18:04:55.014128] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T18:04:55.574898] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-07. [2025-09-16T18:04:56.374914] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-05. [2025-09-16T18:04:56.591679] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T18:04:57.104731] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-09-16T18:04:57.174881] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-08. [2025-09-16T18:04:57.974928] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-08. [2025-09-16T18:04:58.097312] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-09-16T18:04:58.774910] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-06. [2025-09-16T18:04:59.081886] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-09-16T18:04:59.574945] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-05. [2025-09-16T18:05:00.194223] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-09-16T18:05:00.226988] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-09-16T18:05:00.375073] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-06. [2025-09-16T18:05:00.382270] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-09-16T18:05:01.174900] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-09. [2025-09-16T18:05:01.974962] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-07. [2025-09-16T18:05:02.774911] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-09. [2025-09-16T18:05:03.541196] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T18:05:03.574868] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-08. [2025-09-16T18:05:04.646282] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-06 (4.27 seconds). [2025-09-16T18:05:04.649369] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-07. [2025-09-16T18:05:04.899083] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-09-16T18:05:05.066492] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-09 (2.29 seconds). [2025-09-16T18:05:05.073714] [INFO ] [REMOTE ] DHI: Full fetch complete (58039 records). [2025-09-16T18:05:05.073971] [DEBUG] [VALIDATOR ] DHI: Validating primary data. [2025-09-16T18:05:05.176523] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-09. [2025-09-16T18:05:05.480181] [DEBUG] [VALIDATOR ] DHI: All expected trading days are present. [2025-09-16T18:05:05.480428] [DEBUG] [VALIDATOR ] DHI: Primary data validation passed. [2025-09-16T18:05:05.481462] [INFO ] [MAIN ] DHI: Fetched primary data is valid. [2025-09-16T18:05:05.481765] [DEBUG] [LOCAL ] DHI: Saving primary data. [2025-09-16T18:05:05.483492] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-09-16T18:05:05.488518] [DEBUG] [LOCAL ] DHI: Trimmed 58039 records to 39803 (keeping 150 days of data) [2025-09-16T18:05:05.577310] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-09 (4.40 seconds). [2025-09-16T18:05:05.623112] [INFO ] [REMOTE ] DHR: Full fetch complete (57653 records). [2025-09-16T18:05:05.627180] [DEBUG] [VALIDATOR ] DHR: Validating primary data. [2025-09-16T18:05:05.629379] [FATAL] [VALIDATOR ] DHR: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:05:05.630905] [FATAL] [MAIN ] DHR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:05:05.631800] [WARN ] [MAIN ] DHR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:05:05.632630] [INFO ] [LOCAL ] DHR: Starting deletion of primary data. [2025-09-16T18:05:05.634951] [DEBUG] [LOCAL ] DHR: No primary data file found to delete. [2025-09-16T18:05:05.636028] [INFO ] [LOCAL ] DHR: Starting deletion of derivative data. [2025-09-16T18:05:05.637326] [DEBUG] [LOCAL ] DHR: Deleted derivative data file. [2025-09-16T18:05:05.637920] [INFO ] [MAIN ] Update Progress: 616/2369 symbols completed. DHR took 24.6s. Estimated remaining: 3h 5m 6s. Rate: 9.5 symbols/min [2025-09-16T18:05:05.832033] [INFO ] [MAIN ] DIAX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:05.833729] [DEBUG] [LOCAL ] DIAX: No existing data file found in primary storage. [2025-09-16T18:05:05.834045] [WARN ] [MAIN ] DIAX: No existing data found, performing full refresh instead. [2025-09-16T18:05:05.834737] [INFO ] [MAIN ] DIAX: Wiping existing primary data before full refresh. [2025-09-16T18:05:05.835742] [INFO ] [LOCAL ] DIAX: Starting deletion of primary data. [2025-09-16T18:05:05.836972] [DEBUG] [LOCAL ] DIAX: No primary data file found to delete. [2025-09-16T18:05:05.837302] [INFO ] [MAIN ] DIAX: Starting full primary refresh. [2025-09-16T18:05:05.979950] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-08. [2025-09-16T18:05:06.316676] [INFO ] [REMOTE ] DIAX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:06.347866] [DEBUG] [LOCAL ] DHI: Primary data saved (39803 records). [2025-09-16T18:05:06.348029] [INFO ] [MAIN ] DHI: Full history saved. [2025-09-16T18:05:06.350380] [INFO ] [MAIN ] Update Progress: 617/2369 symbols completed. DHI took 33.4s. Estimated remaining: 3h 4m 43s. Rate: 9.5 symbols/min [2025-09-16T18:05:06.526384] [INFO ] [MAIN ] DIN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:06.556742] [DEBUG] [LOCAL ] DIN: Loaded 26800 existing records from primary storage. [2025-09-16T18:05:06.556885] [INFO ] [MAIN ] DIN: Loaded 26800 existing records. [2025-09-16T18:05:06.738260] [WARN ] [MAIN ] DIN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:05:06.738504] [INFO ] [MAIN ] DIN: Wiping existing primary data before full refresh. [2025-09-16T18:05:06.738551] [INFO ] [LOCAL ] DIN: Starting deletion of primary data. [2025-09-16T18:05:06.739368] [DEBUG] [LOCAL ] DIN: Deleted primary data file. [2025-09-16T18:05:06.739429] [INFO ] [MAIN ] DIN: Starting full primary refresh. [2025-09-16T18:05:06.783404] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-02. [2025-09-16T18:05:07.055287] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T18:05:07.113291] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-09-16T18:05:07.153263] [INFO ] [REMOTE ] DHT: Full fetch complete (51967 records). [2025-09-16T18:05:07.153171] [INFO ] [REMOTE ] DIN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:07.153668] [DEBUG] [VALIDATOR ] DHT: Validating primary data. [2025-09-16T18:05:07.154334] [FATAL] [VALIDATOR ] DHT: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:05:07.154417] [FATAL] [MAIN ] DHT: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:05:07.154456] [WARN ] [MAIN ] DHT: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:05:07.154502] [INFO ] [LOCAL ] DHT: Starting deletion of primary data. [2025-09-16T18:05:07.154561] [DEBUG] [LOCAL ] DHT: No primary data file found to delete. [2025-09-16T18:05:07.154598] [INFO ] [LOCAL ] DHT: Starting deletion of derivative data. [2025-09-16T18:05:07.154669] [DEBUG] [LOCAL ] DHT: No derivative data file found to delete. [2025-09-16T18:05:07.154746] [INFO ] [MAIN ] Update Progress: 618/2369 symbols completed. DHT took 21.8s. Estimated remaining: 3h 4m 22s. Rate: 9.5 symbols/min [2025-09-16T18:05:07.327903] [INFO ] [MAIN ] DINO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:07.366381] [DEBUG] [LOCAL ] DINO: Loaded 39092 existing records from primary storage. [2025-09-16T18:05:07.366531] [INFO ] [MAIN ] DINO: Loaded 39092 existing records. [2025-09-16T18:05:07.550012] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-09-16T18:05:07.553318] [WARN ] [MAIN ] DINO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:05:07.556031] [INFO ] [MAIN ] DINO: Wiping existing primary data before full refresh. [2025-09-16T18:05:07.556118] [INFO ] [LOCAL ] DINO: Starting deletion of primary data. [2025-09-16T18:05:07.557099] [DEBUG] [LOCAL ] DINO: Deleted primary data file. [2025-09-16T18:05:07.557150] [INFO ] [MAIN ] DINO: Starting full primary refresh. [2025-09-16T18:05:07.579946] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-02. [2025-09-16T18:05:07.907082] [INFO ] [REMOTE ] DINO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:08.374913] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-03. [2025-09-16T18:05:09.021435] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-09-16T18:05:09.174893] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-04. [2025-09-16T18:05:09.408564] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T18:05:09.974891] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-09. [2025-09-16T18:05:10.141421] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-09-16T18:05:10.774942] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-03. [2025-09-16T18:05:10.965927] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T18:05:10.968426] [INFO ] [REMOTE ] DHX: Full fetch complete (15104 records). [2025-09-16T18:05:10.968507] [DEBUG] [VALIDATOR ] DHX: Validating primary data. [2025-09-16T18:05:10.969003] [FATAL] [VALIDATOR ] DHX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:05:10.969084] [FATAL] [MAIN ] DHX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:05:10.969123] [WARN ] [MAIN ] DHX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:05:10.969165] [INFO ] [LOCAL ] DHX: Starting deletion of primary data. [2025-09-16T18:05:10.969223] [DEBUG] [LOCAL ] DHX: No primary data file found to delete. [2025-09-16T18:05:10.969260] [INFO ] [LOCAL ] DHX: Starting deletion of derivative data. [2025-09-16T18:05:10.969519] [DEBUG] [LOCAL ] DHX: Deleted derivative data file. [2025-09-16T18:05:10.969600] [INFO ] [MAIN ] Update Progress: 619/2369 symbols completed. DHX took 21.5s. Estimated remaining: 3h 4m 8s. Rate: 9.5 symbols/min [2025-09-16T18:05:11.140659] [INFO ] [MAIN ] DIS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:11.180873] [DEBUG] [LOCAL ] DIS: Loaded 39748 existing records from primary storage. [2025-09-16T18:05:11.181053] [INFO ] [MAIN ] DIS: Loaded 39748 existing records. [2025-09-16T18:05:11.358144] [WARN ] [MAIN ] DIS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:05:11.358383] [INFO ] [MAIN ] DIS: Wiping existing primary data before full refresh. [2025-09-16T18:05:11.358430] [INFO ] [LOCAL ] DIS: Starting deletion of primary data. [2025-09-16T18:05:11.359396] [DEBUG] [LOCAL ] DIS: Deleted primary data file. [2025-09-16T18:05:11.359446] [INFO ] [MAIN ] DIS: Starting full primary refresh. [2025-09-16T18:05:11.580073] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-02. [2025-09-16T18:05:11.725469] [INFO ] [REMOTE ] DIS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:12.374908] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-05. [2025-09-16T18:05:12.514154] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T18:05:13.123909] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-09-16T18:05:13.174914] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-02. [2025-09-16T18:05:13.289566] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-09-16T18:05:13.974953] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-06. [2025-09-16T18:05:14.774949] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-04. [2025-09-16T18:05:14.805622] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-09-16T18:05:15.364014] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-02 (2.19 seconds). [2025-09-16T18:05:15.574905] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-03. [2025-09-16T18:05:16.372321] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T18:05:16.380475] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-07. [2025-09-16T18:05:17.096148] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-09-16T18:05:17.174924] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-05. [2025-09-16T18:05:17.943672] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-03 (2.37 seconds). [2025-09-16T18:05:17.974852] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-04. [2025-09-16T18:05:18.578942] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T18:05:18.774905] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-08. [2025-09-16T18:05:19.473700] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-09-16T18:05:19.574894] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-09. [2025-09-16T18:05:20.374919] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-06. [2025-09-16T18:05:20.531803] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T18:05:20.533862] [INFO ] [REMOTE ] DIAX: Full fetch complete (12998 records). [2025-09-16T18:05:20.534009] [DEBUG] [VALIDATOR ] DIAX: Validating primary data. [2025-09-16T18:05:20.534488] [FATAL] [VALIDATOR ] DIAX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:05:20.534622] [FATAL] [MAIN ] DIAX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:05:20.534687] [WARN ] [MAIN ] DIAX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:05:20.534737] [INFO ] [LOCAL ] DIAX: Starting deletion of primary data. [2025-09-16T18:05:20.534857] [DEBUG] [LOCAL ] DIAX: No primary data file found to delete. [2025-09-16T18:05:20.534902] [INFO ] [LOCAL ] DIAX: Starting deletion of derivative data. [2025-09-16T18:05:20.534985] [DEBUG] [LOCAL ] DIAX: No derivative data file found to delete. [2025-09-16T18:05:20.535111] [INFO ] [MAIN ] Update Progress: 620/2369 symbols completed. DIAX took 14.9s. Estimated remaining: 3h 4m 11s. Rate: 9.5 symbols/min [2025-09-16T18:05:20.678776] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-04 (2.70 seconds). [2025-09-16T18:05:20.761525] [INFO ] [MAIN ] DIVY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:20.761786] [DEBUG] [LOCAL ] DIVY: No existing data file found in primary storage. [2025-09-16T18:05:20.761855] [WARN ] [MAIN ] DIVY: No existing data found, performing full refresh instead. [2025-09-16T18:05:20.761987] [INFO ] [MAIN ] DIVY: Wiping existing primary data before full refresh. [2025-09-16T18:05:20.762028] [INFO ] [LOCAL ] DIVY: Starting deletion of primary data. [2025-09-16T18:05:20.762072] [DEBUG] [LOCAL ] DIVY: No primary data file found to delete. [2025-09-16T18:05:20.762105] [INFO ] [MAIN ] DIVY: Starting full primary refresh. [2025-09-16T18:05:21.115582] [INFO ] [REMOTE ] DIVY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:21.174919] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-03. [2025-09-16T18:05:21.705727] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T18:05:21.974880] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-07. [2025-09-16T18:05:22.774904] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-05. [2025-09-16T18:05:22.983497] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T18:05:23.254308] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-09-16T18:05:23.574898] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-08. [2025-09-16T18:05:24.374901] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-02. [2025-09-16T18:05:24.879334] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-09-16T18:05:25.053581] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T18:05:25.174858] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-09. [2025-09-16T18:05:25.253766] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-05 (2.48 seconds). [2025-09-16T18:05:25.974876] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-04. [2025-09-16T18:05:26.668861] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T18:05:26.673317] [INFO ] [REMOTE ] DIN: Full fetch complete (40362 records). [2025-09-16T18:05:26.673415] [DEBUG] [VALIDATOR ] DIN: Validating primary data. [2025-09-16T18:05:26.674038] [FATAL] [VALIDATOR ] DIN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:05:26.674118] [FATAL] [MAIN ] DIN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:05:26.674161] [WARN ] [MAIN ] DIN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:05:26.674222] [INFO ] [LOCAL ] DIN: Starting deletion of primary data. [2025-09-16T18:05:26.674294] [DEBUG] [LOCAL ] DIN: No primary data file found to delete. [2025-09-16T18:05:26.674344] [INFO ] [LOCAL ] DIN: Starting deletion of derivative data. [2025-09-16T18:05:26.674603] [DEBUG] [LOCAL ] DIN: Deleted derivative data file. [2025-09-16T18:05:26.674714] [INFO ] [MAIN ] Update Progress: 621/2369 symbols completed. DIN took 20.3s. Estimated remaining: 3h 4m 4s. Rate: 9.5 symbols/min [2025-09-16T18:05:26.779954] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-03. [2025-09-16T18:05:26.854730] [INFO ] [MAIN ] DK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:26.898371] [DEBUG] [LOCAL ] DK: Loaded 36616 existing records from primary storage. [2025-09-16T18:05:26.898522] [INFO ] [MAIN ] DK: Loaded 36616 existing records. [2025-09-16T18:05:27.081188] [WARN ] [MAIN ] DK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:05:27.081439] [INFO ] [MAIN ] DK: Wiping existing primary data before full refresh. [2025-09-16T18:05:27.081485] [INFO ] [LOCAL ] DK: Starting deletion of primary data. [2025-09-16T18:05:27.082393] [DEBUG] [LOCAL ] DK: Deleted primary data file. [2025-09-16T18:05:27.082444] [INFO ] [MAIN ] DK: Starting full primary refresh. [2025-09-16T18:05:27.342332] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-09-16T18:05:27.462694] [INFO ] [REMOTE ] DK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:27.574857] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-04. [2025-09-16T18:05:27.660628] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T18:05:27.924547] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-09-16T18:05:28.374911] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-05. [2025-09-16T18:05:28.705236] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-09-16T18:05:29.174912] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-02. [2025-09-16T18:05:29.974884] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-06. [2025-09-16T18:05:30.301265] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-09-16T18:05:30.530688] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T18:05:30.774942] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-05. [2025-09-16T18:05:31.574934] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-07. [2025-09-16T18:05:31.999911] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-09-16T18:05:32.374937] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-06. [2025-09-16T18:05:32.533921] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T18:05:33.174938] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-06. [2025-09-16T18:05:33.974888] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-03. [2025-09-16T18:05:34.528605] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-06 (2.15 seconds). [2025-09-16T18:05:34.663309] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T18:05:34.774887] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-08. [2025-09-16T18:05:35.188252] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-09-16T18:05:35.574909] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-09. [2025-09-16T18:05:35.662700] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-09-16T18:05:35.976488] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-09-16T18:05:35.977222] [INFO ] [REMOTE ] DIVY: Full fetch complete (1274 records). [2025-09-16T18:05:35.977289] [DEBUG] [VALIDATOR ] DIVY: Validating primary data. [2025-09-16T18:05:36.159013] [WARN ] [VALIDATOR ] DIVY: Validation failed - insufficient historical data. Have 148 unique days, need 150 days [2025-09-16T18:05:36.159205] [FATAL] [MAIN ] DIVY: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:05:36.159254] [WARN ] [MAIN ] DIVY: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:05:36.159295] [INFO ] [LOCAL ] DIVY: Starting deletion of primary data. [2025-09-16T18:05:36.159361] [DEBUG] [LOCAL ] DIVY: No primary data file found to delete. [2025-09-16T18:05:36.159398] [INFO ] [LOCAL ] DIVY: Starting deletion of derivative data. [2025-09-16T18:05:36.159439] [DEBUG] [LOCAL ] DIVY: No derivative data file found to delete. [2025-09-16T18:05:36.159496] [INFO ] [MAIN ] Update Progress: 622/2369 symbols completed. DIVY took 15.6s. Estimated remaining: 3h 4m 7s. Rate: 9.5 symbols/min [2025-09-16T18:05:36.368908] [INFO ] [MAIN ] DKL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:36.375867] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-04. [2025-09-16T18:05:36.383105] [DEBUG] [LOCAL ] DKL: Loaded 12610 existing records from primary storage. [2025-09-16T18:05:36.383272] [INFO ] [MAIN ] DKL: Loaded 12610 existing records. [2025-09-16T18:05:36.564980] [WARN ] [MAIN ] DKL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:05:36.565226] [INFO ] [MAIN ] DKL: Wiping existing primary data before full refresh. [2025-09-16T18:05:36.565272] [INFO ] [LOCAL ] DKL: Starting deletion of primary data. [2025-09-16T18:05:36.565735] [DEBUG] [LOCAL ] DKL: Deleted primary data file. [2025-09-16T18:05:36.565787] [INFO ] [MAIN ] DKL: Starting full primary refresh. [2025-09-16T18:05:36.939610] [INFO ] [REMOTE ] DKL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:37.174900] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-02. [2025-09-16T18:05:37.974943] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-07. [2025-09-16T18:05:38.111739] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-09-16T18:05:38.774919] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-07. [2025-09-16T18:05:38.802193] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T18:05:39.574955] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-03. [2025-09-16T18:05:40.188900] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-07 (2.21 seconds). [2025-09-16T18:05:40.374900] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-08. [2025-09-16T18:05:40.739445] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T18:05:40.899280] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-09-16T18:05:41.174973] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-08. [2025-09-16T18:05:41.974936] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-05. [2025-09-16T18:05:42.086558] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-09-16T18:05:42.774912] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-04. [2025-09-16T18:05:43.574951] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-09. [2025-09-16T18:05:43.574893] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-08 (2.40 seconds). [2025-09-16T18:05:43.765608] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T18:05:43.795550] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-09-16T18:05:44.374913] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-09. [2025-09-16T18:05:45.174915] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-05. [2025-09-16T18:05:45.262811] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-09-16T18:05:45.269092] [INFO ] [REMOTE ] DINO: Full fetch complete (57632 records). [2025-09-16T18:05:45.269196] [DEBUG] [VALIDATOR ] DINO: Validating primary data. [2025-09-16T18:05:45.654478] [DEBUG] [VALIDATOR ] DINO: All expected trading days are present. [2025-09-16T18:05:45.654575] [DEBUG] [VALIDATOR ] DINO: Primary data validation passed. [2025-09-16T18:05:45.655326] [INFO ] [MAIN ] DINO: Fetched primary data is valid. [2025-09-16T18:05:45.655379] [DEBUG] [LOCAL ] DINO: Saving primary data. [2025-09-16T18:05:45.656712] [DEBUG] [LOCAL ] DINO: Trimmed 57632 records to 39480 (keeping 150 days of data) [2025-09-16T18:05:45.908242] [DEBUG] [LOCAL ] DINO: Primary data saved (39480 records). [2025-09-16T18:05:45.908392] [INFO ] [MAIN ] DINO: Full history saved. [2025-09-16T18:05:45.908482] [INFO ] [MAIN ] Update Progress: 623/2369 symbols completed. DINO took 38.8s. Estimated remaining: 3h 4m 10s. Rate: 9.5 symbols/min [2025-09-16T18:05:45.982663] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-06. [2025-09-16T18:05:46.085554] [INFO ] [MAIN ] DKS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:46.101565] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-09-16T18:05:46.151256] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T18:05:46.161271] [INFO ] [REMOTE ] DIS: Full fetch complete (58468 records). [2025-09-16T18:05:46.162350] [DEBUG] [VALIDATOR ] DIS: Validating primary data. [2025-09-16T18:05:46.173677] [DEBUG] [LOCAL ] DKS: Loaded 36782 existing records from primary storage. [2025-09-16T18:05:46.173967] [INFO ] [MAIN ] DKS: Loaded 36782 existing records. [2025-09-16T18:05:46.597140] [WARN ] [MAIN ] DKS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:05:46.637593] [INFO ] [MAIN ] DKS: Wiping existing primary data before full refresh. [2025-09-16T18:05:46.650292] [INFO ] [LOCAL ] DKS: Starting deletion of primary data. [2025-09-16T18:05:46.670575] [DEBUG] [LOCAL ] DKS: Deleted primary data file. [2025-09-16T18:05:46.694775] [INFO ] [MAIN ] DKS: Starting full primary refresh. [2025-09-16T18:05:46.781019] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-06. [2025-09-16T18:05:46.904880] [DEBUG] [VALIDATOR ] DIS: All expected trading days are present. [2025-09-16T18:05:46.926031] [DEBUG] [VALIDATOR ] DIS: Primary data validation passed. [2025-09-16T18:05:46.967033] [INFO ] [MAIN ] DIS: Fetched primary data is valid. [2025-09-16T18:05:47.025424] [DEBUG] [LOCAL ] DIS: Saving primary data. [2025-09-16T18:05:47.098133] [DEBUG] [LOCAL ] DIS: Trimmed 58468 records to 40138 (keeping 150 days of data) [2025-09-16T18:05:47.269885] [INFO ] [REMOTE ] DKS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:47.457898] [DEBUG] [LOCAL ] DIS: Primary data saved (40138 records). [2025-09-16T18:05:47.458118] [INFO ] [MAIN ] DIS: Full history saved. [2025-09-16T18:05:47.459587] [INFO ] [MAIN ] Update Progress: 624/2369 symbols completed. DIS took 36.5s. Estimated remaining: 3h 3m 50s. Rate: 9.5 symbols/min [2025-09-16T18:05:47.462358] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T18:05:47.576409] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-07. [2025-09-16T18:05:47.648538] [INFO ] [MAIN ] DLB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:47.673555] [DEBUG] [LOCAL ] DLB: Loaded 23670 existing records from primary storage. [2025-09-16T18:05:47.673709] [INFO ] [MAIN ] DLB: Loaded 23670 existing records. [2025-09-16T18:05:47.852876] [WARN ] [MAIN ] DLB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:05:47.853137] [INFO ] [MAIN ] DLB: Wiping existing primary data before full refresh. [2025-09-16T18:05:47.853184] [INFO ] [LOCAL ] DLB: Starting deletion of primary data. [2025-09-16T18:05:47.853846] [DEBUG] [LOCAL ] DLB: Deleted primary data file. [2025-09-16T18:05:47.853900] [INFO ] [MAIN ] DLB: Starting full primary refresh. [2025-09-16T18:05:48.157411] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T18:05:48.240087] [INFO ] [REMOTE ] DLB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:48.374917] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-02. [2025-09-16T18:05:49.174952] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-07. [2025-09-16T18:05:49.328985] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-09-16T18:05:49.807415] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-09-16T18:05:49.903154] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-09-16T18:05:49.974895] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-08. [2025-09-16T18:05:50.774908] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-03. [2025-09-16T18:05:51.574877] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-02. [2025-09-16T18:05:51.668745] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T18:05:52.121722] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T18:05:52.374913] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-04. [2025-09-16T18:05:53.174955] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-09. [2025-09-16T18:05:53.341021] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-09-16T18:05:53.409015] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T18:05:53.974948] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-05. [2025-09-16T18:05:54.774903] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-03. [2025-09-16T18:05:55.138985] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-09-16T18:05:55.148381] [INFO ] [REMOTE ] DK: Full fetch complete (53801 records). [2025-09-16T18:05:55.148580] [DEBUG] [VALIDATOR ] DK: Validating primary data. [2025-09-16T18:05:55.432660] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T18:05:55.576429] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-06. [2025-09-16T18:05:55.576714] [DEBUG] [VALIDATOR ] DK: All expected trading days are present. [2025-09-16T18:05:55.577468] [DEBUG] [VALIDATOR ] DK: Primary data validation passed. [2025-09-16T18:05:55.578205] [INFO ] [MAIN ] DK: Fetched primary data is valid. [2025-09-16T18:05:55.578252] [DEBUG] [LOCAL ] DK: Saving primary data. [2025-09-16T18:05:55.579523] [DEBUG] [LOCAL ] DK: Trimmed 53801 records to 36998 (keeping 150 days of data) [2025-09-16T18:05:55.811161] [DEBUG] [LOCAL ] DK: Primary data saved (36998 records). [2025-09-16T18:05:55.811309] [INFO ] [MAIN ] DK: Full history saved. [2025-09-16T18:05:55.811398] [INFO ] [MAIN ] Update Progress: 625/2369 symbols completed. DK took 29.1s. Estimated remaining: 3h 3m 50s. Rate: 9.5 symbols/min [2025-09-16T18:05:55.991960] [INFO ] [MAIN ] DLNG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:55.998013] [DEBUG] [LOCAL ] DLNG: Loaded 4259 existing records from primary storage. [2025-09-16T18:05:55.998123] [INFO ] [MAIN ] DLNG: Loaded 4259 existing records. [2025-09-16T18:05:56.180901] [WARN ] [MAIN ] DLNG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:05:56.181176] [INFO ] [MAIN ] DLNG: Wiping existing primary data before full refresh. [2025-09-16T18:05:56.181227] [INFO ] [LOCAL ] DLNG: Starting deletion of primary data. [2025-09-16T18:05:56.181518] [DEBUG] [LOCAL ] DLNG: Deleted primary data file. [2025-09-16T18:05:56.181567] [INFO ] [MAIN ] DLNG: Starting full primary refresh. [2025-09-16T18:05:56.396763] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-08. [2025-09-16T18:05:56.558393] [INFO ] [REMOTE ] DLNG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:05:56.590026] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-09-16T18:05:56.987092] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T18:05:57.174937] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-07. [2025-09-16T18:05:57.368182] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-09-16T18:05:57.974922] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-09. [2025-09-16T18:05:58.186668] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-09-16T18:05:58.774900] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-08. [2025-09-16T18:05:58.903400] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T18:05:58.905735] [INFO ] [REMOTE ] DKL: Full fetch complete (22002 records). [2025-09-16T18:05:58.905819] [DEBUG] [VALIDATOR ] DKL: Validating primary data. [2025-09-16T18:05:59.275799] [DEBUG] [VALIDATOR ] DKL: All expected trading days are present. [2025-09-16T18:05:59.275874] [DEBUG] [VALIDATOR ] DKL: Primary data validation passed. [2025-09-16T18:05:59.276282] [INFO ] [MAIN ] DKL: Fetched primary data is valid. [2025-09-16T18:05:59.276323] [DEBUG] [LOCAL ] DKL: Saving primary data. [2025-09-16T18:05:59.276896] [DEBUG] [LOCAL ] DKL: Trimmed 22002 records to 12673 (keeping 150 days of data) [2025-09-16T18:05:59.353247] [DEBUG] [LOCAL ] DKL: Primary data saved (12673 records). [2025-09-16T18:05:59.353389] [INFO ] [MAIN ] DKL: Full history saved. [2025-09-16T18:05:59.353477] [INFO ] [MAIN ] Update Progress: 626/2369 symbols completed. DKL took 23.2s. Estimated remaining: 3h 3m 36s. Rate: 9.5 symbols/min [2025-09-16T18:05:59.525435] [INFO ] [MAIN ] DLR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:05:59.566954] [DEBUG] [LOCAL ] DLR: Loaded 37449 existing records from primary storage. [2025-09-16T18:05:59.567103] [INFO ] [MAIN ] DLR: Loaded 37449 existing records. [2025-09-16T18:05:59.580094] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-04. [2025-09-16T18:05:59.755017] [WARN ] [MAIN ] DLR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:05:59.755296] [INFO ] [MAIN ] DLR: Wiping existing primary data before full refresh. [2025-09-16T18:05:59.755344] [INFO ] [LOCAL ] DLR: Starting deletion of primary data. [2025-09-16T18:05:59.756175] [DEBUG] [LOCAL ] DLR: Deleted primary data file. [2025-09-16T18:05:59.756229] [INFO ] [MAIN ] DLR: Starting full primary refresh. [2025-09-16T18:06:00.124704] [INFO ] [REMOTE ] DLR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:06:00.251566] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T18:06:00.374887] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-02. [2025-09-16T18:06:01.174945] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-09. [2025-09-16T18:06:01.974932] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-02. [2025-09-16T18:06:02.169007] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-04 (2.59 seconds). [2025-09-16T18:06:02.255048] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-09-16T18:06:02.774955] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-05. [2025-09-16T18:06:02.810615] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T18:06:02.814931] [INFO ] [REMOTE ] DLB: Full fetch complete (35159 records). [2025-09-16T18:06:02.815065] [DEBUG] [VALIDATOR ] DLB: Validating primary data. [2025-09-16T18:06:03.149713] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-09-16T18:06:03.214363] [DEBUG] [VALIDATOR ] DLB: All expected trading days are present. [2025-09-16T18:06:03.214439] [DEBUG] [VALIDATOR ] DLB: Primary data validation passed. [2025-09-16T18:06:03.214961] [INFO ] [MAIN ] DLB: Fetched primary data is valid. [2025-09-16T18:06:03.215004] [DEBUG] [LOCAL ] DLB: Saving primary data. [2025-09-16T18:06:03.215762] [DEBUG] [LOCAL ] DLB: Trimmed 35159 records to 23929 (keeping 150 days of data) [2025-09-16T18:06:03.361702] [DEBUG] [LOCAL ] DLB: Primary data saved (23929 records). [2025-09-16T18:06:03.361828] [INFO ] [MAIN ] DLB: Full history saved. [2025-09-16T18:06:03.361910] [INFO ] [MAIN ] Update Progress: 627/2369 symbols completed. DLB took 15.9s. Estimated remaining: 3h 3m 23s. Rate: 9.5 symbols/min [2025-09-16T18:06:03.529978] [INFO ] [MAIN ] DLX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:06:03.550870] [DEBUG] [LOCAL ] DLX: Loaded 21432 existing records from primary storage. [2025-09-16T18:06:03.551006] [INFO ] [MAIN ] DLX: Loaded 21432 existing records. [2025-09-16T18:06:03.581327] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-03. [2025-09-16T18:06:03.735300] [WARN ] [MAIN ] DLX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:06:03.735633] [INFO ] [MAIN ] DLX: Wiping existing primary data before full refresh. [2025-09-16T18:06:03.735699] [INFO ] [LOCAL ] DLX: Starting deletion of primary data. [2025-09-16T18:06:03.736288] [DEBUG] [LOCAL ] DLX: Deleted primary data file. [2025-09-16T18:06:03.736334] [INFO ] [MAIN ] DLX: Starting full primary refresh. [2025-09-16T18:06:04.093422] [INFO ] [REMOTE ] DLX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:06:04.374904] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-03. [2025-09-16T18:06:04.473120] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T18:06:04.474928] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-09-16T18:06:05.174912] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-06. [2025-09-16T18:06:05.862058] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-09-16T18:06:05.974901] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-02. [2025-09-16T18:06:06.774895] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-04. [2025-09-16T18:06:06.803152] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T18:06:06.883178] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-09-16T18:06:07.574902] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-03. [2025-09-16T18:06:07.734794] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-09-16T18:06:08.374901] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-04. [2025-09-16T18:06:08.615069] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-09-16T18:06:09.174904] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-05. [2025-09-16T18:06:09.948635] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-09-16T18:06:09.974846] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-06. [2025-09-16T18:06:10.206119] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T18:06:10.692063] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-09-16T18:06:10.774872] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-07. [2025-09-16T18:06:11.574909] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-04. [2025-09-16T18:06:12.374899] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-05. [2025-09-16T18:06:12.615446] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-09-16T18:06:12.659251] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-09-16T18:06:13.174895] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-05. [2025-09-16T18:06:13.974878] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-08. [2025-09-16T18:06:14.074109] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T18:06:14.538251] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-09-16T18:06:14.774901] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-06. [2025-09-16T18:06:15.574880] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-07. [2025-09-16T18:06:15.638431] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T18:06:16.093898] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-09-16T18:06:16.181604] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T18:06:16.374902] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-09. [2025-09-16T18:06:17.174900] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-08. [2025-09-16T18:06:17.831206] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-09-16T18:06:17.974884] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-06. [2025-09-16T18:06:18.207864] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-09-16T18:06:18.214015] [INFO ] [REMOTE ] DKS: Full fetch complete (54251 records). [2025-09-16T18:06:18.214127] [DEBUG] [VALIDATOR ] DKS: Validating primary data. [2025-09-16T18:06:18.605030] [DEBUG] [VALIDATOR ] DKS: All expected trading days are present. [2025-09-16T18:06:18.605267] [DEBUG] [VALIDATOR ] DKS: Primary data validation passed. [2025-09-16T18:06:18.606149] [INFO ] [MAIN ] DKS: Fetched primary data is valid. [2025-09-16T18:06:18.606227] [DEBUG] [LOCAL ] DKS: Saving primary data. [2025-09-16T18:06:18.607672] [DEBUG] [LOCAL ] DKS: Trimmed 54251 records to 37152 (keeping 150 days of data) [2025-09-16T18:06:18.794710] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-09. [2025-09-16T18:06:18.857107] [DEBUG] [LOCAL ] DKS: Primary data saved (37152 records). [2025-09-16T18:06:18.857270] [INFO ] [MAIN ] DKS: Full history saved. [2025-09-16T18:06:18.857360] [INFO ] [MAIN ] Update Progress: 628/2369 symbols completed. DKS took 32.9s. Estimated remaining: 3h 3m 42s. Rate: 9.5 symbols/min [2025-09-16T18:06:19.028592] [INFO ] [MAIN ] DLY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:06:19.028805] [DEBUG] [LOCAL ] DLY: No existing data file found in primary storage. [2025-09-16T18:06:19.028844] [WARN ] [MAIN ] DLY: No existing data found, performing full refresh instead. [2025-09-16T18:06:19.029008] [INFO ] [MAIN ] DLY: Wiping existing primary data before full refresh. [2025-09-16T18:06:19.029048] [INFO ] [LOCAL ] DLY: Starting deletion of primary data. [2025-09-16T18:06:19.029102] [DEBUG] [LOCAL ] DLY: No primary data file found to delete. [2025-09-16T18:06:19.029165] [INFO ] [MAIN ] DLY: Starting full primary refresh. [2025-09-16T18:06:19.398287] [INFO ] [REMOTE ] DLY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:06:19.410161] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-09-16T18:06:19.411423] [INFO ] [REMOTE ] DLNG: Full fetch complete (7273 records). [2025-09-16T18:06:19.411501] [DEBUG] [VALIDATOR ] DLNG: Validating primary data. [2025-09-16T18:06:19.583015] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-02. [2025-09-16T18:06:19.789493] [DEBUG] [VALIDATOR ] DLNG: All expected trading days are present. [2025-09-16T18:06:19.789712] [DEBUG] [VALIDATOR ] DLNG: Primary data validation passed. [2025-09-16T18:06:19.789966] [INFO ] [MAIN ] DLNG: Fetched primary data is valid. [2025-09-16T18:06:19.790040] [DEBUG] [LOCAL ] DLNG: Saving primary data. [2025-09-16T18:06:19.790916] [DEBUG] [LOCAL ] DLNG: Trimmed 7273 records to 4310 (keeping 150 days of data) [2025-09-16T18:06:19.817868] [DEBUG] [LOCAL ] DLNG: Primary data saved (4310 records). [2025-09-16T18:06:19.818126] [INFO ] [MAIN ] DLNG: Full history saved. [2025-09-16T18:06:19.818273] [INFO ] [MAIN ] Update Progress: 629/2369 symbols completed. DLNG took 24.0s. Estimated remaining: 3h 3m 21s. Rate: 9.5 symbols/min [2025-09-16T18:06:19.949729] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-09-16T18:06:20.030729] [INFO ] [MAIN ] DMA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:06:20.035979] [DEBUG] [LOCAL ] DMA: Loaded 3612 existing records from primary storage. [2025-09-16T18:06:20.036056] [INFO ] [MAIN ] DMA: Loaded 3612 existing records. [2025-09-16T18:06:20.211406] [WARN ] [MAIN ] DMA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:06:20.211693] [INFO ] [MAIN ] DMA: Wiping existing primary data before full refresh. [2025-09-16T18:06:20.211754] [INFO ] [LOCAL ] DMA: Starting deletion of primary data. [2025-09-16T18:06:20.212156] [DEBUG] [LOCAL ] DMA: Deleted primary data file. [2025-09-16T18:06:20.212223] [INFO ] [MAIN ] DMA: Starting full primary refresh. [2025-09-16T18:06:20.389180] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-07. [2025-09-16T18:06:20.591008] [INFO ] [REMOTE ] DMA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:06:20.854824] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T18:06:21.174887] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-02. [2025-09-16T18:06:21.911212] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-09-16T18:06:21.974857] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-03. [2025-09-16T18:06:22.538112] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-09-16T18:06:22.570004] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-07 (2.18 seconds). [2025-09-16T18:06:22.774893] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-04. [2025-09-16T18:06:23.271706] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-09-16T18:06:23.574910] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-07. [2025-09-16T18:06:24.374903] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-03. [2025-09-16T18:06:24.797029] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-09-16T18:06:25.174899] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-08. [2025-09-16T18:06:25.400304] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-09-16T18:06:25.974888] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-08. [2025-09-16T18:06:26.412894] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T18:06:26.774897] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-04. [2025-09-16T18:06:27.574860] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-05. [2025-09-16T18:06:27.733163] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-09-16T18:06:27.757179] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-09-16T18:06:27.968277] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-09-16T18:06:28.374882] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-06. [2025-09-16T18:06:28.771214] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-09-16T18:06:29.174946] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-09. [2025-09-16T18:06:29.974876] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-09. [2025-09-16T18:06:30.464371] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T18:06:30.468267] [INFO ] [REMOTE ] DLX: Full fetch complete (32338 records). [2025-09-16T18:06:30.468403] [DEBUG] [VALIDATOR ] DLX: Validating primary data. [2025-09-16T18:06:30.780093] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-05. [2025-09-16T18:06:30.844400] [DEBUG] [VALIDATOR ] DLX: All expected trading days are present. [2025-09-16T18:06:30.844484] [DEBUG] [VALIDATOR ] DLX: Primary data validation passed. [2025-09-16T18:06:30.845062] [INFO ] [MAIN ] DLX: Fetched primary data is valid. [2025-09-16T18:06:30.845364] [DEBUG] [LOCAL ] DLX: Saving primary data. [2025-09-16T18:06:30.846473] [DEBUG] [LOCAL ] DLX: Trimmed 32338 records to 21590 (keeping 150 days of data) [2025-09-16T18:06:30.980084] [DEBUG] [LOCAL ] DLX: Primary data saved (21590 records). [2025-09-16T18:06:30.980210] [INFO ] [MAIN ] DLX: Full history saved. [2025-09-16T18:06:30.980296] [INFO ] [MAIN ] Update Progress: 630/2369 symbols completed. DLX took 27.6s. Estimated remaining: 3h 3m 28s. Rate: 9.5 symbols/min [2025-09-16T18:06:31.160534] [INFO ] [MAIN ] DMB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:06:31.169909] [DEBUG] [LOCAL ] DMB: Loaded 7871 existing records from primary storage. [2025-09-16T18:06:31.170005] [INFO ] [MAIN ] DMB: Loaded 7871 existing records. [2025-09-16T18:06:31.347664] [WARN ] [MAIN ] DMB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:06:31.347932] [INFO ] [MAIN ] DMB: Wiping existing primary data before full refresh. [2025-09-16T18:06:31.348000] [INFO ] [LOCAL ] DMB: Starting deletion of primary data. [2025-09-16T18:06:31.348336] [DEBUG] [LOCAL ] DMB: Deleted primary data file. [2025-09-16T18:06:31.348383] [INFO ] [MAIN ] DMB: Starting full primary refresh. [2025-09-16T18:06:31.577213] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-07. [2025-09-16T18:06:31.747171] [INFO ] [REMOTE ] DMB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:06:31.748274] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T18:06:31.749836] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-09-16T18:06:31.760473] [INFO ] [REMOTE ] DLR: Full fetch complete (55863 records). [2025-09-16T18:06:31.760579] [DEBUG] [VALIDATOR ] DLR: Validating primary data. [2025-09-16T18:06:32.144492] [DEBUG] [VALIDATOR ] DLR: All expected trading days are present. [2025-09-16T18:06:32.144570] [DEBUG] [VALIDATOR ] DLR: Primary data validation passed. [2025-09-16T18:06:32.145289] [INFO ] [MAIN ] DLR: Fetched primary data is valid. [2025-09-16T18:06:32.145352] [DEBUG] [LOCAL ] DLR: Saving primary data. [2025-09-16T18:06:32.146430] [DEBUG] [LOCAL ] DLR: Trimmed 55863 records to 37822 (keeping 150 days of data) [2025-09-16T18:06:32.364756] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T18:06:32.374954] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-08. [2025-09-16T18:06:32.400964] [DEBUG] [LOCAL ] DLR: Primary data saved (37822 records). [2025-09-16T18:06:32.401092] [INFO ] [MAIN ] DLR: Full history saved. [2025-09-16T18:06:32.401174] [INFO ] [MAIN ] Update Progress: 631/2369 symbols completed. DLR took 33.0s. Estimated remaining: 3h 3m 8s. Rate: 9.5 symbols/min [2025-09-16T18:06:32.570565] [INFO ] [MAIN ] DMO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:06:32.578791] [DEBUG] [LOCAL ] DMO: Loaded 6850 existing records from primary storage. [2025-09-16T18:06:32.578905] [INFO ] [MAIN ] DMO: Loaded 6850 existing records. [2025-09-16T18:06:32.757954] [WARN ] [MAIN ] DMO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:06:32.758202] [INFO ] [MAIN ] DMO: Wiping existing primary data before full refresh. [2025-09-16T18:06:32.758251] [INFO ] [LOCAL ] DMO: Starting deletion of primary data. [2025-09-16T18:06:32.758563] [DEBUG] [LOCAL ] DMO: Deleted primary data file. [2025-09-16T18:06:32.758609] [INFO ] [MAIN ] DMO: Starting full primary refresh. [2025-09-16T18:06:33.126295] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T18:06:33.129593] [INFO ] [REMOTE ] DMO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:06:33.174852] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-09. [2025-09-16T18:06:33.883291] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-09-16T18:06:33.884802] [INFO ] [REMOTE ] DMA: Full fetch complete (5032 records). [2025-09-16T18:06:33.884904] [DEBUG] [VALIDATOR ] DMA: Validating primary data. [2025-09-16T18:06:33.980209] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-06. [2025-09-16T18:06:34.378783] [DEBUG] [VALIDATOR ] DMA: All expected trading days are present. [2025-09-16T18:06:34.378902] [DEBUG] [VALIDATOR ] DMA: Primary data validation passed. [2025-09-16T18:06:34.379187] [INFO ] [MAIN ] DMA: Fetched primary data is valid. [2025-09-16T18:06:34.379244] [DEBUG] [LOCAL ] DMA: Saving primary data. [2025-09-16T18:06:34.379865] [DEBUG] [LOCAL ] DMA: Trimmed 5032 records to 3636 (keeping 150 days of data) [2025-09-16T18:06:34.414873] [DEBUG] [LOCAL ] DMA: Primary data saved (3636 records). [2025-09-16T18:06:34.415069] [INFO ] [MAIN ] DMA: Full history saved. [2025-09-16T18:06:34.415233] [INFO ] [MAIN ] Update Progress: 632/2369 symbols completed. DMA took 14.6s. Estimated remaining: 3h 2m 50s. Rate: 9.5 symbols/min [2025-09-16T18:06:34.705393] [INFO ] [MAIN ] DNA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:06:34.745800] [DEBUG] [LOCAL ] DNA: Loaded 35541 existing records from primary storage. [2025-09-16T18:06:34.745980] [INFO ] [MAIN ] DNA: Loaded 35541 existing records. [2025-09-16T18:06:34.780159] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-02. [2025-09-16T18:06:34.946274] [WARN ] [MAIN ] DNA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:06:34.946591] [INFO ] [MAIN ] DNA: Wiping existing primary data before full refresh. [2025-09-16T18:06:34.946776] [INFO ] [LOCAL ] DNA: Starting deletion of primary data. [2025-09-16T18:06:34.952736] [DEBUG] [LOCAL ] DNA: Deleted primary data file. [2025-09-16T18:06:34.958675] [INFO ] [MAIN ] DNA: Starting full primary refresh. [2025-09-16T18:06:34.994363] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T18:06:35.355190] [INFO ] [REMOTE ] DNA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:06:35.550206] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-09-16T18:06:35.574851] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-03. [2025-09-16T18:06:36.340859] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-09-16T18:06:36.374847] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-04. [2025-09-16T18:06:37.099157] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-09-16T18:06:37.174875] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-05. [2025-09-16T18:06:37.935936] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-09-16T18:06:37.974862] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-06. [2025-09-16T18:06:38.743313] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T18:06:38.774887] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-07. [2025-09-16T18:06:39.544873] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T18:06:39.574857] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-08. [2025-09-16T18:06:40.272676] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-09-16T18:06:40.374877] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-09. [2025-09-16T18:06:41.174913] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-02. [2025-09-16T18:06:41.194539] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-09-16T18:06:41.196036] [INFO ] [REMOTE ] DMO: Full fetch complete (10437 records). [2025-09-16T18:06:41.196104] [DEBUG] [VALIDATOR ] DMO: Validating primary data. [2025-09-16T18:06:41.554898] [DEBUG] [VALIDATOR ] DMO: All expected trading days are present. [2025-09-16T18:06:41.554980] [DEBUG] [VALIDATOR ] DMO: Primary data validation passed. [2025-09-16T18:06:41.555313] [INFO ] [MAIN ] DMO: Fetched primary data is valid. [2025-09-16T18:06:41.555372] [DEBUG] [LOCAL ] DMO: Saving primary data. [2025-09-16T18:06:41.555856] [DEBUG] [LOCAL ] DMO: Trimmed 10437 records to 6948 (keeping 150 days of data) [2025-09-16T18:06:41.603580] [DEBUG] [LOCAL ] DMO: Primary data saved (6948 records). [2025-09-16T18:06:41.603772] [INFO ] [MAIN ] DMO: Full history saved. [2025-09-16T18:06:41.603899] [INFO ] [MAIN ] Update Progress: 633/2369 symbols completed. DMO took 9.2s. Estimated remaining: 3h 2m 46s. Rate: 9.5 symbols/min [2025-09-16T18:06:41.777493] [INFO ] [MAIN ] DNOW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:06:41.777690] [DEBUG] [LOCAL ] DNOW: No existing data file found in primary storage. [2025-09-16T18:06:41.777768] [WARN ] [MAIN ] DNOW: No existing data found, performing full refresh instead. [2025-09-16T18:06:41.777894] [INFO ] [MAIN ] DNOW: Wiping existing primary data before full refresh. [2025-09-16T18:06:41.777934] [INFO ] [LOCAL ] DNOW: Starting deletion of primary data. [2025-09-16T18:06:41.777977] [DEBUG] [LOCAL ] DNOW: No primary data file found to delete. [2025-09-16T18:06:41.778011] [INFO ] [MAIN ] DNOW: Starting full primary refresh. [2025-09-16T18:06:41.980289] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-02. [2025-09-16T18:06:42.077834] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-09-16T18:06:42.152620] [INFO ] [REMOTE ] DNOW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:06:42.774921] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-02. [2025-09-16T18:06:43.574911] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-03. [2025-09-16T18:06:44.056847] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-02 (2.08 seconds). [2025-09-16T18:06:44.083430] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-09-16T18:06:44.175839] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-09-16T18:06:44.374952] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-03. [2025-09-16T18:06:45.174930] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-04. [2025-09-16T18:06:45.908127] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T18:06:45.974857] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-04. [2025-09-16T18:06:45.997320] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-09-16T18:06:46.774991] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-05. [2025-09-16T18:06:47.533392] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-09-16T18:06:47.583246] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-03. [2025-09-16T18:06:47.588095] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-09-16T18:06:48.374939] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-05. [2025-09-16T18:06:49.174913] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-07. [2025-09-16T18:06:49.512227] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-09-16T18:06:49.736554] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-09-16T18:06:49.974888] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-06. [2025-09-16T18:06:50.331130] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-09-16T18:06:50.771779] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-09-16T18:06:50.774845] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-07. [2025-09-16T18:06:51.383302] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-09-16T18:06:51.574905] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-06. [2025-09-16T18:06:52.374901] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-04. [2025-09-16T18:06:53.076730] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T18:06:53.174915] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-07. [2025-09-16T18:06:53.974919] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-08. [2025-09-16T18:06:54.240090] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-09-16T18:06:54.774928] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-05. [2025-09-16T18:06:54.893105] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-09-16T18:06:55.052288] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T18:06:55.574915] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-09. [2025-09-16T18:06:56.356608] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T18:06:56.374844] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-06. [2025-09-16T18:06:56.421509] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T18:06:56.423537] [INFO ] [REMOTE ] DMB: Full fetch complete (10829 records). [2025-09-16T18:06:56.423611] [DEBUG] [VALIDATOR ] DMB: Validating primary data. [2025-09-16T18:06:56.783120] [DEBUG] [VALIDATOR ] DMB: All expected trading days are present. [2025-09-16T18:06:56.783196] [DEBUG] [VALIDATOR ] DMB: Primary data validation passed. [2025-09-16T18:06:56.783435] [INFO ] [MAIN ] DMB: Fetched primary data is valid. [2025-09-16T18:06:56.783474] [DEBUG] [LOCAL ] DMB: Saving primary data. [2025-09-16T18:06:56.783935] [DEBUG] [LOCAL ] DMB: Trimmed 10829 records to 7953 (keeping 150 days of data) [2025-09-16T18:06:56.831974] [DEBUG] [LOCAL ] DMB: Primary data saved (7953 records). [2025-09-16T18:06:56.832090] [INFO ] [MAIN ] DMB: Full history saved. [2025-09-16T18:06:56.832179] [INFO ] [MAIN ] Update Progress: 634/2369 symbols completed. DMB took 25.9s. Estimated remaining: 3h 3m 4s. Rate: 9.5 symbols/min [2025-09-16T18:06:57.006351] [INFO ] [MAIN ] DNP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:06:57.028790] [DEBUG] [LOCAL ] DNP: Loaded 24357 existing records from primary storage. [2025-09-16T18:06:57.028931] [INFO ] [MAIN ] DNP: Loaded 24357 existing records. [2025-09-16T18:06:57.179974] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-08. [2025-09-16T18:06:57.207026] [WARN ] [MAIN ] DNP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:06:57.207228] [INFO ] [MAIN ] DNP: Wiping existing primary data before full refresh. [2025-09-16T18:06:57.207296] [INFO ] [LOCAL ] DNP: Starting deletion of primary data. [2025-09-16T18:06:57.207991] [DEBUG] [LOCAL ] DNP: Deleted primary data file. [2025-09-16T18:06:57.208049] [INFO ] [MAIN ] DNP: Starting full primary refresh. [2025-09-16T18:06:57.555470] [INFO ] [REMOTE ] DNP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:06:57.974891] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-08. [2025-09-16T18:06:58.143808] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-09-16T18:06:58.190484] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T18:06:58.774890] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-07. [2025-09-16T18:06:59.259005] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-09-16T18:06:59.574914] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-02. [2025-09-16T18:07:00.374868] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-09. [2025-09-16T18:07:00.885283] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-09-16T18:07:01.049124] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T18:07:01.174861] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-08. [2025-09-16T18:07:01.563019] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-09-16T18:07:01.570079] [INFO ] [REMOTE ] DLY: Full fetch complete (20513 records). [2025-09-16T18:07:01.570217] [DEBUG] [VALIDATOR ] DLY: Validating primary data. [2025-09-16T18:07:01.933981] [DEBUG] [VALIDATOR ] DLY: All expected trading days are present. [2025-09-16T18:07:01.934061] [DEBUG] [VALIDATOR ] DLY: Primary data validation passed. [2025-09-16T18:07:01.934467] [INFO ] [MAIN ] DLY: Fetched primary data is valid. [2025-09-16T18:07:01.934544] [DEBUG] [LOCAL ] DLY: Saving primary data. [2025-09-16T18:07:01.935214] [DEBUG] [LOCAL ] DLY: Trimmed 20513 records to 13860 (keeping 150 days of data) [2025-09-16T18:07:01.979306] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-09. [2025-09-16T18:07:02.021253] [DEBUG] [LOCAL ] DLY: Primary data saved (13860 records). [2025-09-16T18:07:02.021355] [INFO ] [MAIN ] DLY: Full history saved. [2025-09-16T18:07:02.021444] [INFO ] [MAIN ] Update Progress: 635/2369 symbols completed. DLY took 43.2s. Estimated remaining: 3h 2m 55s. Rate: 9.5 symbols/min [2025-09-16T18:07:02.194442] [INFO ] [MAIN ] DOC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:02.232494] [DEBUG] [LOCAL ] DOC: Loaded 39535 existing records from primary storage. [2025-09-16T18:07:02.232627] [INFO ] [MAIN ] DOC: Loaded 39535 existing records. [2025-09-16T18:07:02.412407] [WARN ] [MAIN ] DOC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:07:02.412656] [INFO ] [MAIN ] DOC: Wiping existing primary data before full refresh. [2025-09-16T18:07:02.412714] [INFO ] [LOCAL ] DOC: Starting deletion of primary data. [2025-09-16T18:07:02.413531] [DEBUG] [LOCAL ] DOC: Deleted primary data file. [2025-09-16T18:07:02.413580] [INFO ] [MAIN ] DOC: Starting full primary refresh. [2025-09-16T18:07:02.761554] [INFO ] [REMOTE ] DOC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:02.774843] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-02. [2025-09-16T18:07:02.840502] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:07:03.176093] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-09-16T18:07:03.182008] [INFO ] [REMOTE ] DNOW: Full fetch complete (47551 records). [2025-09-16T18:07:03.182101] [DEBUG] [VALIDATOR ] DNOW: Validating primary data. [2025-09-16T18:07:03.557430] [DEBUG] [VALIDATOR ] DNOW: All expected trading days are present. [2025-09-16T18:07:03.557510] [DEBUG] [VALIDATOR ] DNOW: Primary data validation passed. [2025-09-16T18:07:03.558165] [INFO ] [MAIN ] DNOW: Fetched primary data is valid. [2025-09-16T18:07:03.558213] [DEBUG] [LOCAL ] DNOW: Saving primary data. [2025-09-16T18:07:03.559838] [DEBUG] [LOCAL ] DNOW: Trimmed 47551 records to 31921 (keeping 150 days of data) [2025-09-16T18:07:03.611058] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-09. [2025-09-16T18:07:03.748927] [DEBUG] [LOCAL ] DNOW: Primary data saved (31921 records). [2025-09-16T18:07:03.749057] [INFO ] [MAIN ] DNOW: Full history saved. [2025-09-16T18:07:03.749140] [INFO ] [MAIN ] Update Progress: 636/2369 symbols completed. DNOW took 22.1s. Estimated remaining: 3h 2m 36s. Rate: 9.5 symbols/min [2025-09-16T18:07:03.925068] [INFO ] [MAIN ] DOCN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:03.960408] [DEBUG] [LOCAL ] DOCN: Loaded 37526 existing records from primary storage. [2025-09-16T18:07:03.960549] [INFO ] [MAIN ] DOCN: Loaded 37526 existing records. [2025-09-16T18:07:04.137338] [WARN ] [MAIN ] DOCN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:07:04.137584] [INFO ] [MAIN ] DOCN: Wiping existing primary data before full refresh. [2025-09-16T18:07:04.137679] [INFO ] [LOCAL ] DOCN: Starting deletion of primary data. [2025-09-16T18:07:04.138448] [DEBUG] [LOCAL ] DOCN: Deleted primary data file. [2025-09-16T18:07:04.138520] [INFO ] [MAIN ] DOCN: Starting full primary refresh. [2025-09-16T18:07:04.383732] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-03. [2025-09-16T18:07:04.493786] [INFO ] [REMOTE ] DOCN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:04.988672] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-09-16T18:07:05.174857] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-02. [2025-09-16T18:07:05.228893] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T18:07:05.236795] [INFO ] [REMOTE ] DNA: Full fetch complete (53102 records). [2025-09-16T18:07:05.237018] [DEBUG] [VALIDATOR ] DNA: Validating primary data. [2025-09-16T18:07:05.628904] [DEBUG] [VALIDATOR ] DNA: All expected trading days are present. [2025-09-16T18:07:05.629021] [DEBUG] [VALIDATOR ] DNA: Primary data validation passed. [2025-09-16T18:07:05.629796] [INFO ] [MAIN ] DNA: Fetched primary data is valid. [2025-09-16T18:07:05.629859] [DEBUG] [LOCAL ] DNA: Saving primary data. [2025-09-16T18:07:05.632629] [DEBUG] [LOCAL ] DNA: Trimmed 53102 records to 35867 (keeping 150 days of data) [2025-09-16T18:07:05.844697] [DEBUG] [LOCAL ] DNA: Primary data saved (35867 records). [2025-09-16T18:07:05.844834] [INFO ] [MAIN ] DNA: Full history saved. [2025-09-16T18:07:05.844937] [INFO ] [MAIN ] Update Progress: 637/2369 symbols completed. DNA took 31.4s. Estimated remaining: 3h 2m 18s. Rate: 9.5 symbols/min [2025-09-16T18:07:05.893775] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-09-16T18:07:05.974842] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-04. [2025-09-16T18:07:06.785047] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-03. [2025-09-16T18:07:06.790492] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T18:07:07.303942] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T18:07:07.574876] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-03. [2025-09-16T18:07:08.374888] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-05. [2025-09-16T18:07:08.840546] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-03 (2.06 seconds). [2025-09-16T18:07:08.927571] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T18:07:09.174955] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-04. [2025-09-16T18:07:09.387360] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T18:07:09.974893] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-06. [2025-09-16T18:07:10.774939] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-04. [2025-09-16T18:07:11.156601] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-09-16T18:07:11.464253] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T18:07:11.574865] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-07. [2025-09-16T18:07:12.374966] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-05. [2025-09-16T18:07:12.721589] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-09-16T18:07:12.953515] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-09-16T18:07:13.174908] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-05. [2025-09-16T18:07:13.974934] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-08. [2025-09-16T18:07:14.179847] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-09-16T18:07:14.780220] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-06. [2025-09-16T18:07:14.783088] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T18:07:15.342297] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-09-16T18:07:15.574987] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-06. [2025-09-16T18:07:16.374944] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-09. [2025-09-16T18:07:16.880507] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-06 (2.10 seconds). [2025-09-16T18:07:17.174898] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-07. [2025-09-16T18:07:17.306172] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T18:07:17.310217] [INFO ] [REMOTE ] DNP: Full fetch complete (36420 records). [2025-09-16T18:07:17.310334] [DEBUG] [VALIDATOR ] DNP: Validating primary data. [2025-09-16T18:07:17.646941] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-06 (2.07 seconds). [2025-09-16T18:07:17.726238] [DEBUG] [VALIDATOR ] DNP: All expected trading days are present. [2025-09-16T18:07:17.726330] [DEBUG] [VALIDATOR ] DNP: Primary data validation passed. [2025-09-16T18:07:17.727013] [INFO ] [MAIN ] DNP: Fetched primary data is valid. [2025-09-16T18:07:17.727102] [DEBUG] [LOCAL ] DNP: Saving primary data. [2025-09-16T18:07:17.728183] [DEBUG] [LOCAL ] DNP: Trimmed 36420 records to 24576 (keeping 150 days of data) [2025-09-16T18:07:17.879546] [DEBUG] [LOCAL ] DNP: Primary data saved (24576 records). [2025-09-16T18:07:17.879693] [INFO ] [MAIN ] DNP: Full history saved. [2025-09-16T18:07:17.879793] [INFO ] [MAIN ] Update Progress: 638/2369 symbols completed. DNP took 21.0s. Estimated remaining: 3h 2m 27s. Rate: 9.5 symbols/min [2025-09-16T18:07:17.974891] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-07. [2025-09-16T18:07:19.369773] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-09-16T18:07:19.381550] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-08. [2025-09-16T18:07:19.478995] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T18:07:19.574897] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-08. [2025-09-16T18:07:21.327233] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-09-16T18:07:21.338860] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-09. [2025-09-16T18:07:21.557468] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-09-16T18:07:21.568821] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-09. [2025-09-16T18:07:23.088916] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-09-16T18:07:23.095304] [INFO ] [REMOTE ] DOC: Full fetch complete (58181 records). [2025-09-16T18:07:23.095424] [DEBUG] [VALIDATOR ] DOC: Validating primary data. [2025-09-16T18:07:23.502612] [DEBUG] [VALIDATOR ] DOC: All expected trading days are present. [2025-09-16T18:07:23.503917] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-09-16T18:07:23.503256] [DEBUG] [VALIDATOR ] DOC: Primary data validation passed. [2025-09-16T18:07:23.510478] [INFO ] [MAIN ] DOC: Fetched primary data is valid. [2025-09-16T18:07:23.510680] [DEBUG] [LOCAL ] DOC: Saving primary data. [2025-09-16T18:07:23.511082] [INFO ] [REMOTE ] DOCN: Full fetch complete (54717 records). [2025-09-16T18:07:23.511356] [DEBUG] [VALIDATOR ] DOCN: Validating primary data. [2025-09-16T18:07:23.595338] [DEBUG] [LOCAL ] DOC: Trimmed 58181 records to 39924 (keeping 150 days of data) [2025-09-16T18:07:24.012163] [DEBUG] [VALIDATOR ] DOCN: All expected trading days are present. [2025-09-16T18:07:24.013229] [DEBUG] [VALIDATOR ] DOCN: Primary data validation passed. [2025-09-16T18:07:24.014300] [INFO ] [MAIN ] DOCN: Fetched primary data is valid. [2025-09-16T18:07:24.014920] [DEBUG] [LOCAL ] DOCN: Saving primary data. [2025-09-16T18:07:24.027917] [DEBUG] [LOCAL ] DOCN: Trimmed 54717 records to 37915 (keeping 150 days of data) [2025-09-16T18:07:24.256256] [DEBUG] [LOCAL ] DOC: Primary data saved (39924 records). [2025-09-16T18:07:24.257887] [INFO ] [MAIN ] DOC: Full history saved. [2025-09-16T18:07:24.258864] [INFO ] [MAIN ] Update Progress: 639/2369 symbols completed. DOC took 22.2s. Estimated remaining: 3h 2m 21s. Rate: 9.5 symbols/min [2025-09-16T18:07:24.381772] [DEBUG] [LOCAL ] DOCN: Primary data saved (37915 records). [2025-09-16T18:07:24.381938] [INFO ] [MAIN ] DOCN: Full history saved. [2025-09-16T18:07:24.382046] [INFO ] [MAIN ] Update Progress: 640/2369 symbols completed. DOCN took 20.6s. Estimated remaining: 3h 1m 58s. Rate: 9.5 symbols/min [2025-09-16T18:07:24.982394] [INFO ] [MAIN ] DOCS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:25.108318] [INFO ] [MAIN ] DOUG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:25.112097] [INFO ] [MAIN ] DOV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:25.127772] [INFO ] [MAIN ] DOLE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:25.128085] [DEBUG] [LOCAL ] DOV: No existing data file found in primary storage. [2025-09-16T18:07:25.128704] [WARN ] [MAIN ] DOV: No existing data found, performing full refresh instead. [2025-09-16T18:07:25.129211] [INFO ] [MAIN ] DOV: Wiping existing primary data before full refresh. [2025-09-16T18:07:25.129256] [INFO ] [LOCAL ] DOV: Starting deletion of primary data. [2025-09-16T18:07:25.131052] [DEBUG] [LOCAL ] DOV: No primary data file found to delete. [2025-09-16T18:07:25.131119] [INFO ] [MAIN ] DOV: Starting full primary refresh. [2025-09-16T18:07:25.331319] [DEBUG] [LOCAL ] DOUG: Loaded 25700 existing records from primary storage. [2025-09-16T18:07:25.379594] [INFO ] [MAIN ] DOUG: Loaded 25700 existing records. [2025-09-16T18:07:25.403184] [DEBUG] [LOCAL ] DOLE: Loaded 28240 existing records from primary storage. [2025-09-16T18:07:25.422537] [INFO ] [MAIN ] DOLE: Loaded 28240 existing records. [2025-09-16T18:07:25.422361] [DEBUG] [LOCAL ] DOCS: Loaded 37582 existing records from primary storage. [2025-09-16T18:07:25.493205] [INFO ] [MAIN ] DOCS: Loaded 37582 existing records. [2025-09-16T18:07:26.023964] [INFO ] [REMOTE ] DOV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:26.068243] [WARN ] [MAIN ] DOUG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:07:26.106922] [INFO ] [MAIN ] DOUG: Wiping existing primary data before full refresh. [2025-09-16T18:07:26.068873] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-02. [2025-09-16T18:07:26.110105] [WARN ] [MAIN ] DOLE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:07:26.110200] [WARN ] [MAIN ] DOCS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:07:26.111369] [INFO ] [MAIN ] DOLE: Wiping existing primary data before full refresh. [2025-09-16T18:07:26.111467] [INFO ] [LOCAL ] DOLE: Starting deletion of primary data. [2025-09-16T18:07:26.111523] [INFO ] [LOCAL ] DOUG: Starting deletion of primary data. [2025-09-16T18:07:26.112189] [DEBUG] [LOCAL ] DOUG: Deleted primary data file. [2025-09-16T18:07:26.112264] [INFO ] [MAIN ] DOCS: Wiping existing primary data before full refresh. [2025-09-16T18:07:26.112458] [INFO ] [LOCAL ] DOCS: Starting deletion of primary data. [2025-09-16T18:07:26.112420] [INFO ] [MAIN ] DOUG: Starting full primary refresh. [2025-09-16T18:07:26.117913] [DEBUG] [LOCAL ] DOLE: Deleted primary data file. [2025-09-16T18:07:26.133367] [DEBUG] [LOCAL ] DOCS: Deleted primary data file. [2025-09-16T18:07:26.133461] [INFO ] [MAIN ] DOCS: Starting full primary refresh. [2025-09-16T18:07:26.166341] [INFO ] [MAIN ] DOLE: Starting full primary refresh. [2025-09-16T18:07:27.028989] [INFO ] [REMOTE ] DOUG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:27.034479] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-02. [2025-09-16T18:07:27.184000] [INFO ] [REMOTE ] DOCS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:27.194869] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-02. [2025-09-16T18:07:27.249753] [INFO ] [REMOTE ] DOLE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:27.250073] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-02. [2025-09-16T18:07:27.804562] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T18:07:27.812958] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-03. [2025-09-16T18:07:28.140769] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-09-16T18:07:28.147872] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-03. [2025-09-16T18:07:28.240010] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-09-16T18:07:28.246273] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-03. [2025-09-16T18:07:29.028484] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T18:07:29.038425] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-03. [2025-09-16T18:07:29.122340] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T18:07:29.132213] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-04. [2025-09-16T18:07:29.251287] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-09-16T18:07:29.258788] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-04. [2025-09-16T18:07:29.546067] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T18:07:29.974939] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-04. [2025-09-16T18:07:30.831469] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-09-16T18:07:30.843297] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-04. [2025-09-16T18:07:30.908912] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T18:07:31.006537] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-09-16T18:07:31.366110] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T18:07:31.574866] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-05. [2025-09-16T18:07:32.374889] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-05. [2025-09-16T18:07:32.521880] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-09-16T18:07:33.090388] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T18:07:33.174924] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-06. [2025-09-16T18:07:33.414980] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-09-16T18:07:33.974914] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-05. [2025-09-16T18:07:34.319422] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:07:34.774901] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-07. [2025-09-16T18:07:35.574883] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-05. [2025-09-16T18:07:35.615875] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T18:07:36.277172] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T18:07:36.374921] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-06. [2025-09-16T18:07:37.122561] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-09-16T18:07:37.174854] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-06. [2025-09-16T18:07:37.969093] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T18:07:37.977442] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-06. [2025-09-16T18:07:38.742399] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T18:07:38.774876] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-07. [2025-09-16T18:07:39.173032] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T18:07:39.574910] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-07. [2025-09-16T18:07:40.374978] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-08. [2025-09-16T18:07:40.657613] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-09-16T18:07:40.740272] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-09-16T18:07:41.174950] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-07. [2025-09-16T18:07:41.512211] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T18:07:41.974915] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-08. [2025-09-16T18:07:42.774920] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-09. [2025-09-16T18:07:43.074679] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-09-16T18:07:43.480249] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T18:07:43.574870] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-08. [2025-09-16T18:07:44.374966] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-09. [2025-09-16T18:07:44.717198] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-09-16T18:07:44.722607] [INFO ] [REMOTE ] DOLE: Full fetch complete (40595 records). [2025-09-16T18:07:44.722719] [DEBUG] [VALIDATOR ] DOLE: Validating primary data. [2025-09-16T18:07:45.092191] [DEBUG] [VALIDATOR ] DOLE: All expected trading days are present. [2025-09-16T18:07:45.092272] [DEBUG] [VALIDATOR ] DOLE: Primary data validation passed. [2025-09-16T18:07:45.092860] [INFO ] [MAIN ] DOLE: Fetched primary data is valid. [2025-09-16T18:07:45.092904] [DEBUG] [LOCAL ] DOLE: Saving primary data. [2025-09-16T18:07:45.093827] [DEBUG] [LOCAL ] DOLE: Trimmed 40595 records to 28584 (keeping 150 days of data) [2025-09-16T18:07:45.187337] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-08. [2025-09-16T18:07:45.276502] [DEBUG] [LOCAL ] DOLE: Primary data saved (28584 records). [2025-09-16T18:07:45.276665] [INFO ] [MAIN ] DOLE: Full history saved. [2025-09-16T18:07:45.276754] [INFO ] [MAIN ] Update Progress: 641/2369 symbols completed. DOLE took 20.9s. Estimated remaining: 3h 2m 31s. Rate: 9.5 symbols/min [2025-09-16T18:07:45.447032] [INFO ] [MAIN ] DOW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:45.489400] [DEBUG] [LOCAL ] DOW: Loaded 39693 existing records from primary storage. [2025-09-16T18:07:45.489586] [INFO ] [MAIN ] DOW: Loaded 39693 existing records. [2025-09-16T18:07:45.670034] [WARN ] [MAIN ] DOW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:07:45.670831] [INFO ] [MAIN ] DOW: Wiping existing primary data before full refresh. [2025-09-16T18:07:45.671108] [INFO ] [LOCAL ] DOW: Starting deletion of primary data. [2025-09-16T18:07:45.676557] [DEBUG] [LOCAL ] DOW: Deleted primary data file. [2025-09-16T18:07:45.679508] [INFO ] [MAIN ] DOW: Starting full primary refresh. [2025-09-16T18:07:45.726828] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T18:07:45.787516] [INFO ] [REMOTE ] DOUG: Full fetch complete (37743 records). [2025-09-16T18:07:45.790722] [DEBUG] [VALIDATOR ] DOUG: Validating primary data. [2025-09-16T18:07:45.817936] [FATAL] [VALIDATOR ] DOUG: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:07:45.842161] [FATAL] [MAIN ] DOUG: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:07:45.849657] [WARN ] [MAIN ] DOUG: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:07:45.867083] [INFO ] [LOCAL ] DOUG: Starting deletion of primary data. [2025-09-16T18:07:45.880946] [DEBUG] [LOCAL ] DOUG: No primary data file found to delete. [2025-09-16T18:07:45.890444] [INFO ] [LOCAL ] DOUG: Starting deletion of derivative data. [2025-09-16T18:07:45.938978] [DEBUG] [LOCAL ] DOUG: Deleted derivative data file. [2025-09-16T18:07:45.971607] [INFO ] [MAIN ] Update Progress: 642/2369 symbols completed. DOUG took 21.6s. Estimated remaining: 3h 2m 9s. Rate: 9.5 symbols/min [2025-09-16T18:07:46.093683] [INFO ] [REMOTE ] DOW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:46.130445] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-08 (2.56 seconds). [2025-09-16T18:07:46.129174] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-02. [2025-09-16T18:07:46.259067] [INFO ] [MAIN ] DPG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:46.270522] [DEBUG] [LOCAL ] DPG: Loaded 9724 existing records from primary storage. [2025-09-16T18:07:46.270666] [INFO ] [MAIN ] DPG: Loaded 9724 existing records. [2025-09-16T18:07:46.453220] [WARN ] [MAIN ] DPG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:07:46.453520] [INFO ] [MAIN ] DPG: Wiping existing primary data before full refresh. [2025-09-16T18:07:46.453566] [INFO ] [LOCAL ] DPG: Starting deletion of primary data. [2025-09-16T18:07:46.453997] [DEBUG] [LOCAL ] DPG: Deleted primary data file. [2025-09-16T18:07:46.454047] [INFO ] [MAIN ] DPG: Starting full primary refresh. [2025-09-16T18:07:46.781434] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-09. [2025-09-16T18:07:46.818482] [INFO ] [REMOTE ] DPG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:46.851496] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T18:07:47.574937] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-09. [2025-09-16T18:07:48.382244] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-02. [2025-09-16T18:07:48.389760] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T18:07:48.397381] [INFO ] [REMOTE ] DOV: Full fetch complete (49658 records). [2025-09-16T18:07:48.397628] [DEBUG] [VALIDATOR ] DOV: Validating primary data. [2025-09-16T18:07:48.696270] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-02 (2.57 seconds). [2025-09-16T18:07:49.044429] [DEBUG] [VALIDATOR ] DOV: All expected trading days are present. [2025-09-16T18:07:49.044554] [DEBUG] [VALIDATOR ] DOV: Primary data validation passed. [2025-09-16T18:07:49.045577] [INFO ] [MAIN ] DOV: Fetched primary data is valid. [2025-09-16T18:07:49.045680] [DEBUG] [LOCAL ] DOV: Saving primary data. [2025-09-16T18:07:49.047068] [DEBUG] [LOCAL ] DOV: Trimmed 49658 records to 33591 (keeping 150 days of data) [2025-09-16T18:07:49.185857] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-03. [2025-09-16T18:07:49.252491] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T18:07:49.294990] [INFO ] [REMOTE ] DOCS: Full fetch complete (56085 records). [2025-09-16T18:07:49.297271] [DEBUG] [VALIDATOR ] DOCS: Validating primary data. [2025-09-16T18:07:49.298073] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-09-16T18:07:49.748171] [DEBUG] [VALIDATOR ] DOCS: All expected trading days are present. [2025-09-16T18:07:49.749880] [DEBUG] [VALIDATOR ] DOCS: Primary data validation passed. [2025-09-16T18:07:49.751446] [INFO ] [MAIN ] DOCS: Fetched primary data is valid. [2025-09-16T18:07:49.751815] [DEBUG] [LOCAL ] DOCS: Saving primary data. [2025-09-16T18:07:49.757176] [DEBUG] [LOCAL ] DOV: Primary data saved (33591 records). [2025-09-16T18:07:49.757372] [INFO ] [MAIN ] DOV: Full history saved. [2025-09-16T18:07:49.757607] [INFO ] [MAIN ] Update Progress: 643/2369 symbols completed. DOV took 25.4s. Estimated remaining: 3h 1m 56s. Rate: 9.5 symbols/min [2025-09-16T18:07:49.757831] [DEBUG] [LOCAL ] DOCS: Trimmed 56085 records to 37942 (keeping 150 days of data) [2025-09-16T18:07:49.975548] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-03. [2025-09-16T18:07:50.027942] [INFO ] [MAIN ] DQ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:50.077483] [DEBUG] [LOCAL ] DQ: Loaded 31306 existing records from primary storage. [2025-09-16T18:07:50.077739] [INFO ] [MAIN ] DQ: Loaded 31306 existing records. [2025-09-16T18:07:50.340810] [WARN ] [MAIN ] DQ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:07:50.343874] [INFO ] [MAIN ] DQ: Wiping existing primary data before full refresh. [2025-09-16T18:07:50.345185] [INFO ] [LOCAL ] DQ: Starting deletion of primary data. [2025-09-16T18:07:50.347441] [DEBUG] [LOCAL ] DQ: Deleted primary data file. [2025-09-16T18:07:50.347766] [INFO ] [MAIN ] DQ: Starting full primary refresh. [2025-09-16T18:07:50.722575] [INFO ] [REMOTE ] DQ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:50.727524] [DEBUG] [LOCAL ] DOCS: Primary data saved (37942 records). [2025-09-16T18:07:50.727677] [INFO ] [MAIN ] DOCS: Full history saved. [2025-09-16T18:07:50.727797] [INFO ] [MAIN ] Update Progress: 644/2369 symbols completed. DOCS took 26.3s. Estimated remaining: 3h 1m 35s. Rate: 9.5 symbols/min [2025-09-16T18:07:50.786131] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-02. [2025-09-16T18:07:50.902280] [INFO ] [MAIN ] DRD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:07:50.928524] [DEBUG] [LOCAL ] DRD: Loaded 26240 existing records from primary storage. [2025-09-16T18:07:50.928693] [INFO ] [MAIN ] DRD: Loaded 26240 existing records. [2025-09-16T18:07:51.123712] [WARN ] [MAIN ] DRD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:07:51.125022] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T18:07:51.128892] [INFO ] [MAIN ] DRD: Wiping existing primary data before full refresh. [2025-09-16T18:07:51.129026] [INFO ] [LOCAL ] DRD: Starting deletion of primary data. [2025-09-16T18:07:51.140356] [DEBUG] [LOCAL ] DRD: Deleted primary data file. [2025-09-16T18:07:51.159909] [INFO ] [MAIN ] DRD: Starting full primary refresh. [2025-09-16T18:07:51.227133] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-03 (2.04 seconds). [2025-09-16T18:07:51.526747] [INFO ] [REMOTE ] DRD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:07:51.574860] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-02. [2025-09-16T18:07:52.374958] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-04. [2025-09-16T18:07:52.607968] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-09-16T18:07:52.770078] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-09-16T18:07:53.174998] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-03. [2025-09-16T18:07:53.331773] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-09-16T18:07:53.974953] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-03. [2025-09-16T18:07:54.720585] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T18:07:54.774859] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-04. [2025-09-16T18:07:55.532953] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T18:07:55.574860] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-04. [2025-09-16T18:07:56.224799] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T18:07:56.374906] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-05. [2025-09-16T18:07:57.148762] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-09-16T18:07:57.174858] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-04. [2025-09-16T18:07:57.865602] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T18:07:57.974874] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-05. [2025-09-16T18:07:58.774910] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-06. [2025-09-16T18:07:59.574881] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-05. [2025-09-16T18:07:59.626095] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T18:07:59.713404] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-04 (2.54 seconds). [2025-09-16T18:07:59.824818] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T18:08:00.374918] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-05. [2025-09-16T18:08:00.705685] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-09-16T18:08:01.174899] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-06. [2025-09-16T18:08:01.980229] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-07. [2025-09-16T18:08:01.981967] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-09-16T18:08:02.774903] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-07. [2025-09-16T18:08:03.575029] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-06. [2025-09-16T18:08:03.775167] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T18:08:03.916022] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-09-16T18:08:04.374952] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-08. [2025-09-16T18:08:05.174955] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-08. [2025-09-16T18:08:06.055146] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T18:08:06.064745] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-09. [2025-09-16T18:08:06.125866] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-09-16T18:08:06.774934] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-09. [2025-09-16T18:08:07.728564] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T18:08:07.730493] [INFO ] [REMOTE ] DPG: Full fetch complete (14285 records). [2025-09-16T18:08:07.730563] [DEBUG] [VALIDATOR ] DPG: Validating primary data. [2025-09-16T18:08:08.110951] [DEBUG] [VALIDATOR ] DPG: All expected trading days are present. [2025-09-16T18:08:08.111147] [DEBUG] [VALIDATOR ] DPG: Primary data validation passed. [2025-09-16T18:08:08.125407] [INFO ] [MAIN ] DPG: Fetched primary data is valid. [2025-09-16T18:08:08.125992] [DEBUG] [LOCAL ] DPG: Saving primary data. [2025-09-16T18:08:08.126495] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-09-16T18:08:08.132735] [INFO ] [REMOTE ] DQ: Full fetch complete (47230 records). [2025-09-16T18:08:08.132849] [DEBUG] [VALIDATOR ] DQ: Validating primary data. [2025-09-16T18:08:08.132618] [DEBUG] [LOCAL ] DPG: Trimmed 14285 records to 9792 (keeping 150 days of data) [2025-09-16T18:08:08.398442] [DEBUG] [LOCAL ] DPG: Primary data saved (9792 records). [2025-09-16T18:08:08.412707] [INFO ] [MAIN ] DPG: Full history saved. [2025-09-16T18:08:08.458505] [INFO ] [MAIN ] Update Progress: 645/2369 symbols completed. DPG took 22.5s. Estimated remaining: 3h 1m 60s. Rate: 9.5 symbols/min [2025-09-16T18:08:08.778750] [DEBUG] [VALIDATOR ] DQ: All expected trading days are present. [2025-09-16T18:08:08.791222] [DEBUG] [VALIDATOR ] DQ: Primary data validation passed. [2025-09-16T18:08:08.810899] [INFO ] [MAIN ] DQ: Fetched primary data is valid. [2025-09-16T18:08:08.809881] [INFO ] [MAIN ] DRH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:08:08.811128] [DEBUG] [LOCAL ] DQ: Saving primary data. [2025-09-16T18:08:08.813024] [DEBUG] [LOCAL ] DQ: Trimmed 47230 records to 31641 (keeping 150 days of data) [2025-09-16T18:08:08.956781] [DEBUG] [LOCAL ] DRH: Loaded 37030 existing records from primary storage. [2025-09-16T18:08:08.958118] [INFO ] [MAIN ] DRH: Loaded 37030 existing records. [2025-09-16T18:08:09.209271] [WARN ] [MAIN ] DRH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:08:09.213701] [INFO ] [MAIN ] DRH: Wiping existing primary data before full refresh. [2025-09-16T18:08:09.214461] [INFO ] [LOCAL ] DRH: Starting deletion of primary data. [2025-09-16T18:08:09.216966] [DEBUG] [LOCAL ] DRH: Deleted primary data file. [2025-09-16T18:08:09.217834] [INFO ] [MAIN ] DRH: Starting full primary refresh. [2025-09-16T18:08:09.545960] [DEBUG] [LOCAL ] DQ: Primary data saved (31641 records). [2025-09-16T18:08:09.561498] [INFO ] [MAIN ] DQ: Full history saved. [2025-09-16T18:08:09.566753] [INFO ] [MAIN ] Update Progress: 646/2369 symbols completed. DQ took 19.8s. Estimated remaining: 3h 1m 39s. Rate: 9.5 symbols/min [2025-09-16T18:08:09.703310] [INFO ] [REMOTE ] DRH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:08:09.728417] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-02. [2025-09-16T18:08:09.806121] [INFO ] [MAIN ] DRI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:08:09.840692] [DEBUG] [LOCAL ] DRI: Loaded 34727 existing records from primary storage. [2025-09-16T18:08:09.840869] [INFO ] [MAIN ] DRI: Loaded 34727 existing records. [2025-09-16T18:08:10.023822] [WARN ] [MAIN ] DRI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:08:10.024080] [INFO ] [MAIN ] DRI: Wiping existing primary data before full refresh. [2025-09-16T18:08:10.024127] [INFO ] [LOCAL ] DRI: Starting deletion of primary data. [2025-09-16T18:08:10.025086] [DEBUG] [LOCAL ] DRI: Deleted primary data file. [2025-09-16T18:08:10.025138] [INFO ] [MAIN ] DRI: Starting full primary refresh. [2025-09-16T18:08:10.383277] [INFO ] [REMOTE ] DRI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:08:10.383660] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-02. [2025-09-16T18:08:11.396031] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T18:08:11.404857] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-03. [2025-09-16T18:08:13.061823] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-02 (2.68 seconds). [2025-09-16T18:08:13.071331] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-03. [2025-09-16T18:08:13.690495] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-03 (2.29 seconds). [2025-09-16T18:08:13.703208] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-04. [2025-09-16T18:08:14.432827] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T18:08:14.443351] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-04. [2025-09-16T18:08:15.120326] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T18:08:15.131182] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-05. [2025-09-16T18:08:15.785803] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-09-16T18:08:15.796573] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-05. [2025-09-16T18:08:16.161991] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-06 (12.59 seconds). [2025-09-16T18:08:16.170294] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-07. [2025-09-16T18:08:16.453912] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T18:08:16.464418] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-06. [2025-09-16T18:08:16.784563] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-05 (16.41 seconds). [2025-09-16T18:08:16.796049] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-06. [2025-09-16T18:08:17.144150] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T18:08:17.157915] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-06. [2025-09-16T18:08:17.358263] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-09-16T18:08:17.366067] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-08. [2025-09-16T18:08:17.725234] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T18:08:17.974960] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-07. [2025-09-16T18:08:18.449300] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T18:08:18.493245] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T18:08:18.594968] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T18:08:18.774898] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-09. [2025-09-16T18:08:19.270295] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-09-16T18:08:19.574953] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-07. [2025-09-16T18:08:19.952094] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T18:08:19.957120] [INFO ] [REMOTE ] DRD: Full fetch complete (38415 records). [2025-09-16T18:08:19.957207] [DEBUG] [VALIDATOR ] DRD: Validating primary data. [2025-09-16T18:08:19.957774] [FATAL] [VALIDATOR ] DRD: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:08:19.957864] [FATAL] [MAIN ] DRD: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:08:19.957919] [WARN ] [MAIN ] DRD: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:08:19.957986] [INFO ] [LOCAL ] DRD: Starting deletion of primary data. [2025-09-16T18:08:19.958054] [DEBUG] [LOCAL ] DRD: No primary data file found to delete. [2025-09-16T18:08:19.958093] [INFO ] [LOCAL ] DRD: Starting deletion of derivative data. [2025-09-16T18:08:19.958354] [DEBUG] [LOCAL ] DRD: Deleted derivative data file. [2025-09-16T18:08:19.958437] [INFO ] [MAIN ] Update Progress: 647/2369 symbols completed. DRD took 29.2s. Estimated remaining: 3h 1m 44s. Rate: 9.5 symbols/min [2025-09-16T18:08:20.128019] [INFO ] [MAIN ] DRLL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:08:20.135839] [DEBUG] [LOCAL ] DRLL: Loaded 6438 existing records from primary storage. [2025-09-16T18:08:20.135961] [INFO ] [MAIN ] DRLL: Loaded 6438 existing records. [2025-09-16T18:08:20.310266] [WARN ] [MAIN ] DRLL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:08:20.310538] [INFO ] [MAIN ] DRLL: Wiping existing primary data before full refresh. [2025-09-16T18:08:20.310590] [INFO ] [LOCAL ] DRLL: Starting deletion of primary data. [2025-09-16T18:08:20.310865] [DEBUG] [LOCAL ] DRLL: Deleted primary data file. [2025-09-16T18:08:20.310960] [INFO ] [MAIN ] DRLL: Starting full primary refresh. [2025-09-16T18:08:20.384457] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-07. [2025-09-16T18:08:20.663006] [INFO ] [REMOTE ] DRLL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:08:21.174907] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-02. [2025-09-16T18:08:21.449345] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T18:08:21.974909] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-08. [2025-09-16T18:08:22.057376] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-09-16T18:08:22.128042] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T18:08:22.774898] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-03. [2025-09-16T18:08:23.291181] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-09-16T18:08:23.547531] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-09-16T18:08:23.574840] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-04. [2025-09-16T18:08:24.069368] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-09-16T18:08:24.374905] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-08. [2025-09-16T18:08:25.174891] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-08. [2025-09-16T18:08:25.704311] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-09-16T18:08:25.974900] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-09. [2025-09-16T18:08:26.774905] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-09. [2025-09-16T18:08:27.517575] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T18:08:27.523595] [INFO ] [REMOTE ] DRH: Full fetch complete (54535 records). [2025-09-16T18:08:27.523719] [DEBUG] [VALIDATOR ] DRH: Validating primary data. [2025-09-16T18:08:27.588339] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-05. [2025-09-16T18:08:27.910804] [DEBUG] [VALIDATOR ] DRH: All expected trading days are present. [2025-09-16T18:08:27.910966] [DEBUG] [VALIDATOR ] DRH: Primary data validation passed. [2025-09-16T18:08:27.911733] [INFO ] [MAIN ] DRH: Fetched primary data is valid. [2025-09-16T18:08:27.911844] [DEBUG] [LOCAL ] DRH: Saving primary data. [2025-09-16T18:08:27.913221] [DEBUG] [LOCAL ] DRH: Trimmed 54535 records to 37403 (keeping 150 days of data) [2025-09-16T18:08:28.034518] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-08 (2.86 seconds). [2025-09-16T18:08:28.184322] [DEBUG] [LOCAL ] DRH: Primary data saved (37403 records). [2025-09-16T18:08:28.184606] [INFO ] [MAIN ] DRH: Full history saved. [2025-09-16T18:08:28.184737] [INFO ] [MAIN ] Update Progress: 648/2369 symbols completed. DRH took 19.7s. Estimated remaining: 3h 1m 43s. Rate: 9.5 symbols/min [2025-09-16T18:08:28.323331] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T18:08:28.374494] [INFO ] [MAIN ] DSL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:08:28.375142] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-06. [2025-09-16T18:08:28.393295] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T18:08:28.399506] [INFO ] [REMOTE ] DRI: Full fetch complete (52061 records). [2025-09-16T18:08:28.399678] [DEBUG] [VALIDATOR ] DRI: Validating primary data. [2025-09-16T18:08:28.516245] [DEBUG] [LOCAL ] DSL: Loaded 19964 existing records from primary storage. [2025-09-16T18:08:28.521994] [INFO ] [MAIN ] DSL: Loaded 19964 existing records. [2025-09-16T18:08:28.902867] [WARN ] [MAIN ] DSL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:08:28.967723] [INFO ] [MAIN ] DSL: Wiping existing primary data before full refresh. [2025-09-16T18:08:28.980672] [INFO ] [LOCAL ] DSL: Starting deletion of primary data. [2025-09-16T18:08:28.990085] [DEBUG] [LOCAL ] DSL: Deleted primary data file. [2025-09-16T18:08:28.990201] [INFO ] [MAIN ] DSL: Starting full primary refresh. [2025-09-16T18:08:29.003709] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-09-16T18:08:29.003818] [DEBUG] [VALIDATOR ] DRI: All expected trading days are present. [2025-09-16T18:08:29.040171] [DEBUG] [VALIDATOR ] DRI: Primary data validation passed. [2025-09-16T18:08:29.053301] [INFO ] [MAIN ] DRI: Fetched primary data is valid. [2025-09-16T18:08:29.073804] [DEBUG] [LOCAL ] DRI: Saving primary data. [2025-09-16T18:08:29.143451] [DEBUG] [LOCAL ] DRI: Trimmed 52061 records to 35099 (keeping 150 days of data) [2025-09-16T18:08:29.178067] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-07. [2025-09-16T18:08:29.535701] [INFO ] [REMOTE ] DSL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:08:29.587927] [DEBUG] [LOCAL ] DRI: Primary data saved (35099 records). [2025-09-16T18:08:29.588080] [INFO ] [MAIN ] DRI: Full history saved. [2025-09-16T18:08:29.588169] [INFO ] [MAIN ] Update Progress: 649/2369 symbols completed. DRI took 20.0s. Estimated remaining: 3h 1m 23s. Rate: 9.5 symbols/min [2025-09-16T18:08:29.773405] [INFO ] [MAIN ] DSM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:08:29.783770] [DEBUG] [LOCAL ] DSM: Loaded 10060 existing records from primary storage. [2025-09-16T18:08:29.783885] [INFO ] [MAIN ] DSM: Loaded 10060 existing records. [2025-09-16T18:08:29.968637] [WARN ] [MAIN ] DSM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:08:29.969017] [INFO ] [MAIN ] DSM: Wiping existing primary data before full refresh. [2025-09-16T18:08:29.969066] [INFO ] [LOCAL ] DSM: Starting deletion of primary data. [2025-09-16T18:08:29.969487] [DEBUG] [LOCAL ] DSM: Deleted primary data file. [2025-09-16T18:08:29.969536] [INFO ] [MAIN ] DSM: Starting full primary refresh. [2025-09-16T18:08:29.980622] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-09. [2025-09-16T18:08:30.329363] [INFO ] [REMOTE ] DSM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:08:30.337401] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-09-16T18:08:30.774902] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-08. [2025-09-16T18:08:31.362733] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-09-16T18:08:31.574868] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-02. [2025-09-16T18:08:32.136835] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-09-16T18:08:32.143184] [INFO ] [REMOTE ] DOW: Full fetch complete (58411 records). [2025-09-16T18:08:32.143300] [DEBUG] [VALIDATOR ] DOW: Validating primary data. [2025-09-16T18:08:32.380011] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-09. [2025-09-16T18:08:32.548906] [DEBUG] [VALIDATOR ] DOW: All expected trading days are present. [2025-09-16T18:08:32.549175] [DEBUG] [VALIDATOR ] DOW: Primary data validation passed. [2025-09-16T18:08:32.550041] [INFO ] [MAIN ] DOW: Fetched primary data is valid. [2025-09-16T18:08:32.550111] [DEBUG] [LOCAL ] DOW: Saving primary data. [2025-09-16T18:08:32.557970] [DEBUG] [LOCAL ] DOW: Trimmed 58411 records to 40083 (keeping 150 days of data) [2025-09-16T18:08:32.570487] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-09-16T18:08:32.829453] [DEBUG] [LOCAL ] DOW: Primary data saved (40083 records). [2025-09-16T18:08:32.829624] [INFO ] [MAIN ] DOW: Full history saved. [2025-09-16T18:08:32.830983] [INFO ] [MAIN ] Update Progress: 650/2369 symbols completed. DOW took 47.6s. Estimated remaining: 3h 1m 9s. Rate: 9.5 symbols/min [2025-09-16T18:08:33.000955] [INFO ] [MAIN ] DSMC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:08:33.004634] [DEBUG] [LOCAL ] DSMC: Loaded 1215 existing records from primary storage. [2025-09-16T18:08:33.004743] [INFO ] [MAIN ] DSMC: Loaded 1215 existing records. [2025-09-16T18:08:33.175106] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-03. [2025-09-16T18:08:33.178351] [WARN ] [MAIN ] DSMC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:08:33.179491] [INFO ] [MAIN ] DSMC: Wiping existing primary data before full refresh. [2025-09-16T18:08:33.179537] [INFO ] [LOCAL ] DSMC: Starting deletion of primary data. [2025-09-16T18:08:33.179808] [DEBUG] [LOCAL ] DSMC: Deleted primary data file. [2025-09-16T18:08:33.179856] [INFO ] [MAIN ] DSMC: Starting full primary refresh. [2025-09-16T18:08:33.312905] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T18:08:33.342525] [INFO ] [REMOTE ] DRLL: Full fetch complete (11134 records). [2025-09-16T18:08:33.372248] [DEBUG] [VALIDATOR ] DRLL: Validating primary data. [2025-09-16T18:08:33.786998] [INFO ] [REMOTE ] DSMC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:08:33.928886] [DEBUG] [VALIDATOR ] DRLL: All expected trading days are present. [2025-09-16T18:08:33.928966] [DEBUG] [VALIDATOR ] DRLL: Primary data validation passed. [2025-09-16T18:08:33.929200] [INFO ] [MAIN ] DRLL: Fetched primary data is valid. [2025-09-16T18:08:33.929241] [DEBUG] [LOCAL ] DRLL: Saving primary data. [2025-09-16T18:08:33.929755] [DEBUG] [LOCAL ] DRLL: Trimmed 11134 records to 6504 (keeping 150 days of data) [2025-09-16T18:08:33.970677] [DEBUG] [LOCAL ] DRLL: Primary data saved (6504 records). [2025-09-16T18:08:33.970818] [INFO ] [MAIN ] DRLL: Full history saved. [2025-09-16T18:08:33.970907] [INFO ] [MAIN ] Update Progress: 651/2369 symbols completed. DRLL took 14.0s. Estimated remaining: 3h 0m 49s. Rate: 9.5 symbols/min [2025-09-16T18:08:33.982135] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-02. [2025-09-16T18:08:34.150299] [INFO ] [MAIN ] DSU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:08:34.150517] [DEBUG] [LOCAL ] DSU: No existing data file found in primary storage. [2025-09-16T18:08:34.150564] [WARN ] [MAIN ] DSU: No existing data found, performing full refresh instead. [2025-09-16T18:08:34.150911] [INFO ] [MAIN ] DSU: Wiping existing primary data before full refresh. [2025-09-16T18:08:34.150988] [INFO ] [LOCAL ] DSU: Starting deletion of primary data. [2025-09-16T18:08:34.151071] [DEBUG] [LOCAL ] DSU: No primary data file found to delete. [2025-09-16T18:08:34.151135] [INFO ] [MAIN ] DSU: Starting full primary refresh. [2025-09-16T18:08:34.283710] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-09-16T18:08:34.537024] [INFO ] [REMOTE ] DSU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:08:34.774900] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-02. [2025-09-16T18:08:35.192329] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-09-16T18:08:35.237118] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-09-16T18:08:35.574900] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-03. [2025-09-16T18:08:36.327232] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-09-16T18:08:36.374868] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-04. [2025-09-16T18:08:37.053078] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T18:08:37.174893] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-04. [2025-09-16T18:08:37.974918] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-02. [2025-09-16T18:08:38.537876] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-09-16T18:08:38.774888] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-05. [2025-09-16T18:08:39.117941] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-09-16T18:08:39.303399] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-09-16T18:08:39.574911] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-06. [2025-09-16T18:08:40.374892] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-05. [2025-09-16T18:08:40.531128] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-09-16T18:08:41.174970] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-07. [2025-09-16T18:08:41.427387] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-09-16T18:08:41.971624] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T18:08:41.976009] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-06. [2025-09-16T18:08:42.774949] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-03. [2025-09-16T18:08:43.013059] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-09-16T18:08:43.574955] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-08. [2025-09-16T18:08:43.806159] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-09-16T18:08:44.374924] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-07. [2025-09-16T18:08:44.474127] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-09-16T18:08:45.174943] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-09. [2025-09-16T18:08:45.724247] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T18:08:45.974910] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-08. [2025-09-16T18:08:46.416302] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T18:08:46.418420] [INFO ] [REMOTE ] DSM: Full fetch complete (17923 records). [2025-09-16T18:08:46.418493] [DEBUG] [VALIDATOR ] DSM: Validating primary data. [2025-09-16T18:08:46.782684] [DEBUG] [VALIDATOR ] DSM: All expected trading days are present. [2025-09-16T18:08:46.782442] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-04. [2025-09-16T18:08:46.783132] [DEBUG] [VALIDATOR ] DSM: Primary data validation passed. [2025-09-16T18:08:46.784376] [INFO ] [MAIN ] DSM: Fetched primary data is valid. [2025-09-16T18:08:46.784453] [DEBUG] [LOCAL ] DSM: Saving primary data. [2025-09-16T18:08:46.785230] [DEBUG] [LOCAL ] DSM: Trimmed 17923 records to 10175 (keeping 150 days of data) [2025-09-16T18:08:46.845049] [DEBUG] [LOCAL ] DSM: Primary data saved (10175 records). [2025-09-16T18:08:46.845198] [INFO ] [MAIN ] DSM: Full history saved. [2025-09-16T18:08:46.845281] [INFO ] [MAIN ] Update Progress: 652/2369 symbols completed. DSM took 17.3s. Estimated remaining: 3h 0m 60s. Rate: 9.5 symbols/min [2025-09-16T18:08:47.022145] [INFO ] [MAIN ] DSX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:08:47.040772] [DEBUG] [LOCAL ] DSX: Loaded 17629 existing records from primary storage. [2025-09-16T18:08:47.040918] [INFO ] [MAIN ] DSX: Loaded 17629 existing records. [2025-09-16T18:08:47.225622] [WARN ] [MAIN ] DSX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:08:47.240800] [INFO ] [MAIN ] DSX: Wiping existing primary data before full refresh. [2025-09-16T18:08:47.240932] [INFO ] [LOCAL ] DSX: Starting deletion of primary data. [2025-09-16T18:08:47.242204] [DEBUG] [LOCAL ] DSX: Deleted primary data file. [2025-09-16T18:08:47.242144] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-09-16T18:08:47.242365] [INFO ] [MAIN ] DSX: Starting full primary refresh. [2025-09-16T18:08:47.580061] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-09. [2025-09-16T18:08:47.624563] [INFO ] [REMOTE ] DSX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:08:47.685543] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-09-16T18:08:48.374909] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-02. [2025-09-16T18:08:48.902167] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T18:08:48.905772] [INFO ] [REMOTE ] DSL: Full fetch complete (30785 records). [2025-09-16T18:08:48.905858] [DEBUG] [VALIDATOR ] DSL: Validating primary data. [2025-09-16T18:08:49.180059] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-03. [2025-09-16T18:08:49.291181] [DEBUG] [VALIDATOR ] DSL: All expected trading days are present. [2025-09-16T18:08:49.291259] [DEBUG] [VALIDATOR ] DSL: Primary data validation passed. [2025-09-16T18:08:49.291765] [INFO ] [MAIN ] DSL: Fetched primary data is valid. [2025-09-16T18:08:49.291821] [DEBUG] [LOCAL ] DSL: Saving primary data. [2025-09-16T18:08:49.292548] [DEBUG] [LOCAL ] DSL: Trimmed 30785 records to 20214 (keeping 150 days of data) [2025-09-16T18:08:49.430180] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T18:08:49.441492] [DEBUG] [LOCAL ] DSL: Primary data saved (20214 records). [2025-09-16T18:08:49.441601] [INFO ] [MAIN ] DSL: Full history saved. [2025-09-16T18:08:49.441703] [INFO ] [MAIN ] Update Progress: 653/2369 symbols completed. DSL took 21.3s. Estimated remaining: 3h 0m 44s. Rate: 9.5 symbols/min [2025-09-16T18:08:49.619439] [INFO ] [MAIN ] DT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:08:49.656664] [DEBUG] [LOCAL ] DT: Loaded 39040 existing records from primary storage. [2025-09-16T18:08:49.656837] [INFO ] [MAIN ] DT: Loaded 39040 existing records. [2025-09-16T18:08:49.832458] [WARN ] [MAIN ] DT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:08:49.832729] [INFO ] [MAIN ] DT: Wiping existing primary data before full refresh. [2025-09-16T18:08:49.832777] [INFO ] [LOCAL ] DT: Starting deletion of primary data. [2025-09-16T18:08:49.833593] [DEBUG] [LOCAL ] DT: Deleted primary data file. [2025-09-16T18:08:49.833664] [INFO ] [MAIN ] DT: Starting full primary refresh. [2025-09-16T18:08:49.936538] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-09-16T18:08:49.980018] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-03. [2025-09-16T18:08:50.226304] [INFO ] [REMOTE ] DT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:08:50.774923] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-04. [2025-09-16T18:08:51.286158] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T18:08:51.358552] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-09-16T18:08:51.574908] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-05. [2025-09-16T18:08:52.026551] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-09-16T18:08:52.374911] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-06. [2025-09-16T18:08:52.710437] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-09-16T18:08:53.174913] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-02. [2025-09-16T18:08:53.974901] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-07. [2025-09-16T18:08:54.396300] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-09-16T18:08:54.774979] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-05. [2025-09-16T18:08:54.943709] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-09-16T18:08:55.467060] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-09-16T18:08:55.574914] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-03. [2025-09-16T18:08:56.374908] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-04. [2025-09-16T18:08:57.174864] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-06. [2025-09-16T18:08:57.321574] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T18:08:57.604680] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T18:08:57.843486] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-09-16T18:08:57.974889] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-05. [2025-09-16T18:08:58.774943] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-04. [2025-09-16T18:08:59.151505] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-09-16T18:08:59.574927] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-07. [2025-09-16T18:09:00.244297] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T18:09:00.374868] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-05. [2025-09-16T18:09:00.404806] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-09-16T18:09:01.174920] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-08. [2025-09-16T18:09:01.772819] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T18:09:01.974938] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-09. [2025-09-16T18:09:02.054144] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T18:09:02.360112] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-09-16T18:09:02.361024] [INFO ] [REMOTE ] DSMC: Full fetch complete (2222 records). [2025-09-16T18:09:02.361114] [DEBUG] [VALIDATOR ] DSMC: Validating primary data. [2025-09-16T18:09:02.361576] [FATAL] [VALIDATOR ] DSMC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:09:02.361681] [FATAL] [MAIN ] DSMC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:09:02.361728] [WARN ] [MAIN ] DSMC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:09:02.361787] [INFO ] [LOCAL ] DSMC: Starting deletion of primary data. [2025-09-16T18:09:02.361846] [DEBUG] [LOCAL ] DSMC: No primary data file found to delete. [2025-09-16T18:09:02.361883] [INFO ] [LOCAL ] DSMC: Starting deletion of derivative data. [2025-09-16T18:09:02.362190] [DEBUG] [LOCAL ] DSMC: Deleted derivative data file. [2025-09-16T18:09:02.362284] [INFO ] [MAIN ] Update Progress: 654/2369 symbols completed. DSMC took 29.5s. Estimated remaining: 3h 0m 55s. Rate: 9.5 symbols/min [2025-09-16T18:09:02.540244] [INFO ] [MAIN ] DTB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:02.549716] [DEBUG] [LOCAL ] DTB: Loaded 3739 existing records from primary storage. [2025-09-16T18:09:02.549877] [INFO ] [MAIN ] DTB: Loaded 3739 existing records. [2025-09-16T18:09:02.727956] [WARN ] [MAIN ] DTB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:09:02.728198] [INFO ] [MAIN ] DTB: Wiping existing primary data before full refresh. [2025-09-16T18:09:02.728243] [INFO ] [LOCAL ] DTB: Starting deletion of primary data. [2025-09-16T18:09:02.728478] [DEBUG] [LOCAL ] DTB: Deleted primary data file. [2025-09-16T18:09:02.728535] [INFO ] [MAIN ] DTB: Starting full primary refresh. [2025-09-16T18:09:02.782071] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-06. [2025-09-16T18:09:03.082892] [INFO ] [REMOTE ] DTB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:03.574907] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-06. [2025-09-16T18:09:04.374925] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-02. [2025-09-16T18:09:04.436771] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T18:09:04.529697] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-09-16T18:09:05.027853] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-09-16T18:09:05.174867] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-03. [2025-09-16T18:09:05.520877] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-09-16T18:09:05.974907] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-08. [2025-09-16T18:09:06.774921] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-07. [2025-09-16T18:09:06.975033] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-09-16T18:09:07.574877] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-04. [2025-09-16T18:09:07.976535] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T18:09:08.184621] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-09-16T18:09:08.374892] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-07. [2025-09-16T18:09:09.174901] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-05. [2025-09-16T18:09:09.596517] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T18:09:09.974926] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-08. [2025-09-16T18:09:10.293789] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T18:09:10.782030] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-08. [2025-09-16T18:09:10.783962] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T18:09:11.574897] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-06. [2025-09-16T18:09:12.122400] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-09-16T18:09:12.374901] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-09. [2025-09-16T18:09:12.584592] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-09-16T18:09:13.174907] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-09. [2025-09-16T18:09:13.211555] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T18:09:13.214744] [INFO ] [REMOTE ] DSU: Full fetch complete (23266 records). [2025-09-16T18:09:13.214819] [DEBUG] [VALIDATOR ] DSU: Validating primary data. [2025-09-16T18:09:13.577678] [DEBUG] [VALIDATOR ] DSU: All expected trading days are present. [2025-09-16T18:09:13.577763] [DEBUG] [VALIDATOR ] DSU: Primary data validation passed. [2025-09-16T18:09:13.578248] [INFO ] [MAIN ] DSU: Fetched primary data is valid. [2025-09-16T18:09:13.578289] [DEBUG] [LOCAL ] DSU: Saving primary data. [2025-09-16T18:09:13.578904] [DEBUG] [LOCAL ] DSU: Trimmed 23266 records to 15673 (keeping 150 days of data) [2025-09-16T18:09:13.677511] [DEBUG] [LOCAL ] DSU: Primary data saved (15673 records). [2025-09-16T18:09:13.677704] [INFO ] [MAIN ] DSU: Full history saved. [2025-09-16T18:09:13.677807] [INFO ] [MAIN ] Update Progress: 655/2369 symbols completed. DSU took 39.7s. Estimated remaining: 3h 1m 1s. Rate: 9.5 symbols/min [2025-09-16T18:09:13.846683] [INFO ] [MAIN ] DTE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:13.882193] [DEBUG] [LOCAL ] DTE: Loaded 35164 existing records from primary storage. [2025-09-16T18:09:13.882359] [INFO ] [MAIN ] DTE: Loaded 35164 existing records. [2025-09-16T18:09:13.980236] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-07. [2025-09-16T18:09:14.062971] [WARN ] [MAIN ] DTE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:09:14.063200] [INFO ] [MAIN ] DTE: Wiping existing primary data before full refresh. [2025-09-16T18:09:14.063247] [INFO ] [LOCAL ] DTE: Starting deletion of primary data. [2025-09-16T18:09:14.064160] [DEBUG] [LOCAL ] DTE: Deleted primary data file. [2025-09-16T18:09:14.064213] [INFO ] [MAIN ] DTE: Starting full primary refresh. [2025-09-16T18:09:14.420942] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-09-16T18:09:14.442929] [INFO ] [REMOTE ] DSX: Full fetch complete (27399 records). [2025-09-16T18:09:14.446208] [INFO ] [REMOTE ] DTE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:14.446350] [DEBUG] [VALIDATOR ] DSX: Validating primary data. [2025-09-16T18:09:14.735954] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-09-16T18:09:14.780012] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-09. [2025-09-16T18:09:14.821347] [DEBUG] [VALIDATOR ] DSX: All expected trading days are present. [2025-09-16T18:09:14.821423] [DEBUG] [VALIDATOR ] DSX: Primary data validation passed. [2025-09-16T18:09:14.821974] [INFO ] [MAIN ] DSX: Fetched primary data is valid. [2025-09-16T18:09:14.822019] [DEBUG] [LOCAL ] DSX: Saving primary data. [2025-09-16T18:09:14.822760] [DEBUG] [LOCAL ] DSX: Trimmed 27399 records to 17834 (keeping 150 days of data) [2025-09-16T18:09:14.925850] [DEBUG] [LOCAL ] DSX: Primary data saved (17834 records). [2025-09-16T18:09:14.925999] [INFO ] [MAIN ] DSX: Full history saved. [2025-09-16T18:09:14.926085] [INFO ] [MAIN ] Update Progress: 656/2369 symbols completed. DSX took 28.1s. Estimated remaining: 3h 0m 42s. Rate: 9.5 symbols/min [2025-09-16T18:09:15.095160] [INFO ] [MAIN ] DTF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:15.095339] [DEBUG] [LOCAL ] DTF: No existing data file found in primary storage. [2025-09-16T18:09:15.095378] [WARN ] [MAIN ] DTF: No existing data found, performing full refresh instead. [2025-09-16T18:09:15.095493] [INFO ] [MAIN ] DTF: Wiping existing primary data before full refresh. [2025-09-16T18:09:15.095531] [INFO ] [LOCAL ] DTF: Starting deletion of primary data. [2025-09-16T18:09:15.095571] [DEBUG] [LOCAL ] DTF: No primary data file found to delete. [2025-09-16T18:09:15.095603] [INFO ] [MAIN ] DTF: Starting full primary refresh. [2025-09-16T18:09:15.445918] [INFO ] [REMOTE ] DTF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:15.574861] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-02. [2025-09-16T18:09:16.085950] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-09-16T18:09:16.374888] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-08. [2025-09-16T18:09:16.699993] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-09-16T18:09:16.706299] [INFO ] [REMOTE ] DT: Full fetch complete (57586 records). [2025-09-16T18:09:16.706402] [DEBUG] [VALIDATOR ] DT: Validating primary data. [2025-09-16T18:09:17.105806] [DEBUG] [VALIDATOR ] DT: All expected trading days are present. [2025-09-16T18:09:17.105906] [DEBUG] [VALIDATOR ] DT: Primary data validation passed. [2025-09-16T18:09:17.106719] [INFO ] [MAIN ] DT: Fetched primary data is valid. [2025-09-16T18:09:17.106770] [DEBUG] [LOCAL ] DT: Saving primary data. [2025-09-16T18:09:17.108036] [DEBUG] [LOCAL ] DT: Trimmed 57586 records to 39430 (keeping 150 days of data) [2025-09-16T18:09:17.199870] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-03. [2025-09-16T18:09:17.203722] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-09-16T18:09:17.352393] [DEBUG] [LOCAL ] DT: Primary data saved (39430 records). [2025-09-16T18:09:17.352554] [INFO ] [MAIN ] DT: Full history saved. [2025-09-16T18:09:17.352776] [INFO ] [MAIN ] Update Progress: 657/2369 symbols completed. DT took 27.9s. Estimated remaining: 3h 0m 25s. Rate: 9.5 symbols/min [2025-09-16T18:09:17.523782] [INFO ] [MAIN ] DTG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:17.524025] [DEBUG] [LOCAL ] DTG: No existing data file found in primary storage. [2025-09-16T18:09:17.524065] [WARN ] [MAIN ] DTG: No existing data found, performing full refresh instead. [2025-09-16T18:09:17.524226] [INFO ] [MAIN ] DTG: Wiping existing primary data before full refresh. [2025-09-16T18:09:17.524279] [INFO ] [LOCAL ] DTG: Starting deletion of primary data. [2025-09-16T18:09:17.524323] [DEBUG] [LOCAL ] DTG: No primary data file found to delete. [2025-09-16T18:09:17.524356] [INFO ] [MAIN ] DTG: Starting full primary refresh. [2025-09-16T18:09:17.772958] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-09-16T18:09:17.883437] [INFO ] [REMOTE ] DTG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:17.974905] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-09. [2025-09-16T18:09:18.774920] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-02. [2025-09-16T18:09:18.801112] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-09-16T18:09:18.802445] [INFO ] [REMOTE ] DTB: Full fetch complete (4990 records). [2025-09-16T18:09:18.802512] [DEBUG] [VALIDATOR ] DTB: Validating primary data. [2025-09-16T18:09:19.163776] [DEBUG] [VALIDATOR ] DTB: All expected trading days are present. [2025-09-16T18:09:19.163873] [DEBUG] [VALIDATOR ] DTB: Primary data validation passed. [2025-09-16T18:09:19.164038] [INFO ] [MAIN ] DTB: Fetched primary data is valid. [2025-09-16T18:09:19.164080] [DEBUG] [LOCAL ] DTB: Saving primary data. [2025-09-16T18:09:19.164484] [DEBUG] [LOCAL ] DTB: Trimmed 4990 records to 3766 (keeping 150 days of data) [2025-09-16T18:09:19.187890] [DEBUG] [LOCAL ] DTB: Primary data saved (3766 records). [2025-09-16T18:09:19.187974] [INFO ] [MAIN ] DTB: Full history saved. [2025-09-16T18:09:19.188076] [INFO ] [MAIN ] Update Progress: 658/2369 symbols completed. DTB took 16.8s. Estimated remaining: 3h 0m 7s. Rate: 9.5 symbols/min [2025-09-16T18:09:19.356074] [INFO ] [MAIN ] DTM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:19.389721] [DEBUG] [LOCAL ] DTM: Loaded 31775 existing records from primary storage. [2025-09-16T18:09:19.389859] [INFO ] [MAIN ] DTM: Loaded 31775 existing records. [2025-09-16T18:09:19.566837] [WARN ] [MAIN ] DTM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:09:19.567083] [INFO ] [MAIN ] DTM: Wiping existing primary data before full refresh. [2025-09-16T18:09:19.567128] [INFO ] [LOCAL ] DTM: Starting deletion of primary data. [2025-09-16T18:09:19.568001] [DEBUG] [LOCAL ] DTM: Deleted primary data file. [2025-09-16T18:09:19.568052] [INFO ] [MAIN ] DTM: Starting full primary refresh. [2025-09-16T18:09:19.580844] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-04. [2025-09-16T18:09:19.930227] [INFO ] [REMOTE ] DTM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:20.110353] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-09-16T18:09:20.387128] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-02. [2025-09-16T18:09:20.407007] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T18:09:21.170252] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-09-16T18:09:21.174834] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-03. [2025-09-16T18:09:21.974937] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-03. [2025-09-16T18:09:22.095090] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-09-16T18:09:22.774956] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-04. [2025-09-16T18:09:23.132707] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-09-16T18:09:23.574918] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-05. [2025-09-16T18:09:24.374909] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-05. [2025-09-16T18:09:24.405821] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T18:09:24.749029] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-09-16T18:09:25.118050] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-03 (3.14 seconds). [2025-09-16T18:09:25.174873] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-06. [2025-09-16T18:09:25.546953] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-09-16T18:09:25.974906] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-06. [2025-09-16T18:09:26.351767] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-09-16T18:09:26.774904] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-02. [2025-09-16T18:09:27.574903] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-04. [2025-09-16T18:09:28.087337] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T18:09:28.374908] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-03. [2025-09-16T18:09:28.942130] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T18:09:29.174938] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-07. [2025-09-16T18:09:29.624882] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-09-16T18:09:29.675036] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T18:09:29.974895] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-04. [2025-09-16T18:09:30.774894] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-05. [2025-09-16T18:09:31.282381] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T18:09:31.574899] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-07. [2025-09-16T18:09:32.084611] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-09-16T18:09:32.098305] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T18:09:32.374890] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-06. [2025-09-16T18:09:33.174924] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-05. [2025-09-16T18:09:33.790442] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T18:09:33.974966] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-07. [2025-09-16T18:09:34.503905] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T18:09:34.774984] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-06. [2025-09-16T18:09:35.327974] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T18:09:35.574898] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-08. [2025-09-16T18:09:36.070557] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T18:09:36.374940] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-08. [2025-09-16T18:09:36.654688] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-08 (0.28 seconds). [2025-09-16T18:09:36.880272] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T18:09:37.174918] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-08. [2025-09-16T18:09:37.553074] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-09-16T18:09:37.974932] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-09. [2025-09-16T18:09:38.774934] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-09. [2025-09-16T18:09:38.810635] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T18:09:38.811827] [INFO ] [REMOTE ] DTG: Full fetch complete (6398 records). [2025-09-16T18:09:38.811895] [DEBUG] [VALIDATOR ] DTG: Validating primary data. [2025-09-16T18:09:38.812321] [FATAL] [VALIDATOR ] DTG: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:09:38.812401] [FATAL] [MAIN ] DTG: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:09:38.812451] [WARN ] [MAIN ] DTG: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:09:38.812510] [INFO ] [LOCAL ] DTG: Starting deletion of primary data. [2025-09-16T18:09:38.812567] [DEBUG] [LOCAL ] DTG: No primary data file found to delete. [2025-09-16T18:09:38.812603] [INFO ] [LOCAL ] DTG: Starting deletion of derivative data. [2025-09-16T18:09:38.812678] [DEBUG] [LOCAL ] DTG: No derivative data file found to delete. [2025-09-16T18:09:38.812748] [INFO ] [MAIN ] Update Progress: 659/2369 symbols completed. DTG took 21.5s. Estimated remaining: 3h 0m 35s. Rate: 9.5 symbols/min [2025-09-16T18:09:38.983515] [INFO ] [MAIN ] DTW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:38.989118] [DEBUG] [LOCAL ] DTW: Loaded 3503 existing records from primary storage. [2025-09-16T18:09:38.989222] [INFO ] [MAIN ] DTW: Loaded 3503 existing records. [2025-09-16T18:09:39.168690] [WARN ] [MAIN ] DTW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:09:39.168930] [INFO ] [MAIN ] DTW: Wiping existing primary data before full refresh. [2025-09-16T18:09:39.169001] [INFO ] [LOCAL ] DTW: Starting deletion of primary data. [2025-09-16T18:09:39.169289] [DEBUG] [LOCAL ] DTW: Deleted primary data file. [2025-09-16T18:09:39.169342] [INFO ] [MAIN ] DTW: Starting full primary refresh. [2025-09-16T18:09:39.528429] [INFO ] [REMOTE ] DTW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:39.574929] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-02. [2025-09-16T18:09:40.084466] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-09-16T18:09:40.268216] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T18:09:40.277122] [INFO ] [REMOTE ] DTE: Full fetch complete (52179 records). [2025-09-16T18:09:40.277307] [DEBUG] [VALIDATOR ] DTE: Validating primary data. [2025-09-16T18:09:40.377734] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-03. [2025-09-16T18:09:40.655852] [DEBUG] [VALIDATOR ] DTE: All expected trading days are present. [2025-09-16T18:09:40.655933] [DEBUG] [VALIDATOR ] DTE: Primary data validation passed. [2025-09-16T18:09:40.656606] [INFO ] [MAIN ] DTE: Fetched primary data is valid. [2025-09-16T18:09:40.656677] [DEBUG] [LOCAL ] DTE: Saving primary data. [2025-09-16T18:09:40.658200] [DEBUG] [LOCAL ] DTE: Trimmed 52179 records to 35490 (keeping 150 days of data) [2025-09-16T18:09:40.878499] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-09-16T18:09:40.907193] [DEBUG] [LOCAL ] DTE: Primary data saved (35490 records). [2025-09-16T18:09:40.907377] [INFO ] [MAIN ] DTE: Full history saved. [2025-09-16T18:09:40.907518] [INFO ] [MAIN ] Update Progress: 660/2369 symbols completed. DTE took 27.2s. Estimated remaining: 3h 0m 18s. Rate: 9.5 symbols/min [2025-09-16T18:09:41.080921] [INFO ] [MAIN ] DUK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:41.125077] [DEBUG] [LOCAL ] DUK: Loaded 39449 existing records from primary storage. [2025-09-16T18:09:41.125230] [INFO ] [MAIN ] DUK: Loaded 39449 existing records. [2025-09-16T18:09:41.180140] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-07. [2025-09-16T18:09:41.317839] [WARN ] [MAIN ] DUK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:09:41.318239] [INFO ] [MAIN ] DUK: Wiping existing primary data before full refresh. [2025-09-16T18:09:41.318294] [INFO ] [LOCAL ] DUK: Starting deletion of primary data. [2025-09-16T18:09:41.319362] [DEBUG] [LOCAL ] DUK: Deleted primary data file. [2025-09-16T18:09:41.319429] [INFO ] [MAIN ] DUK: Starting full primary refresh. [2025-09-16T18:09:41.685158] [INFO ] [REMOTE ] DUK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:41.974916] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-09. [2025-09-16T18:09:42.617789] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-09-16T18:09:42.619056] [INFO ] [REMOTE ] DTF: Full fetch complete (1331 records). [2025-09-16T18:09:42.619154] [DEBUG] [VALIDATOR ] DTF: Validating primary data. [2025-09-16T18:09:42.619659] [FATAL] [VALIDATOR ] DTF: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:09:42.619754] [FATAL] [MAIN ] DTF: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:09:42.619804] [WARN ] [MAIN ] DTF: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:09:42.619856] [INFO ] [LOCAL ] DTF: Starting deletion of primary data. [2025-09-16T18:09:42.619942] [DEBUG] [LOCAL ] DTF: No primary data file found to delete. [2025-09-16T18:09:42.619991] [INFO ] [LOCAL ] DTF: Starting deletion of derivative data. [2025-09-16T18:09:42.620056] [DEBUG] [LOCAL ] DTF: No derivative data file found to delete. [2025-09-16T18:09:42.620131] [INFO ] [MAIN ] Update Progress: 661/2369 symbols completed. DTF took 27.7s. Estimated remaining: 2h 59m 60s. Rate: 9.5 symbols/min [2025-09-16T18:09:42.784841] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-04. [2025-09-16T18:09:42.816715] [INFO ] [MAIN ] DUKB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:42.825504] [DEBUG] [LOCAL ] DUKB: Loaded 4653 existing records from primary storage. [2025-09-16T18:09:42.825692] [INFO ] [MAIN ] DUKB: Loaded 4653 existing records. [2025-09-16T18:09:43.007333] [WARN ] [MAIN ] DUKB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:09:43.007749] [INFO ] [MAIN ] DUKB: Wiping existing primary data before full refresh. [2025-09-16T18:09:43.007852] [INFO ] [LOCAL ] DUKB: Starting deletion of primary data. [2025-09-16T18:09:43.008120] [DEBUG] [LOCAL ] DUKB: Deleted primary data file. [2025-09-16T18:09:43.008166] [INFO ] [MAIN ] DUKB: Starting full primary refresh. [2025-09-16T18:09:43.347236] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-09-16T18:09:43.349580] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-09-16T18:09:43.426113] [INFO ] [REMOTE ] DUKB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:43.574899] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-05. [2025-09-16T18:09:43.975300] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-09-16T18:09:44.374898] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-06. [2025-09-16T18:09:44.786536] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-09-16T18:09:45.174900] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-02. [2025-09-16T18:09:45.974933] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-07. [2025-09-16T18:09:46.438802] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-09-16T18:09:46.539974] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T18:09:46.774908] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-08. [2025-09-16T18:09:47.133206] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-09-16T18:09:47.574926] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-03. [2025-09-16T18:09:48.374900] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-02. [2025-09-16T18:09:49.174896] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-08. [2025-09-16T18:09:49.329192] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-09-16T18:09:49.710375] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-09-16T18:09:49.974907] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-04. [2025-09-16T18:09:50.587372] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-09-16T18:09:50.774907] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-03. [2025-09-16T18:09:51.292555] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-09-16T18:09:51.574872] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-04. [2025-09-16T18:09:52.044510] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-09-16T18:09:52.062777] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-09-16T18:09:52.374905] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-05. [2025-09-16T18:09:53.174921] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-09. [2025-09-16T18:09:53.974908] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-09. [2025-09-16T18:09:54.316401] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-09-16T18:09:54.350197] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T18:09:54.357951] [INFO ] [REMOTE ] DTM: Full fetch complete (48820 records). [2025-09-16T18:09:54.358094] [DEBUG] [VALIDATOR ] DTM: Validating primary data. [2025-09-16T18:09:54.673966] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-09-16T18:09:54.700015] [INFO ] [REMOTE ] DTW: Full fetch complete (5242 records). [2025-09-16T18:09:54.700177] [DEBUG] [VALIDATOR ] DTW: Validating primary data. [2025-09-16T18:09:54.783790] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-06. [2025-09-16T18:09:54.905836] [DEBUG] [VALIDATOR ] DTM: All expected trading days are present. [2025-09-16T18:09:54.940892] [DEBUG] [VALIDATOR ] DTM: Primary data validation passed. [2025-09-16T18:09:54.959243] [INFO ] [MAIN ] DTM: Fetched primary data is valid. [2025-09-16T18:09:54.959591] [DEBUG] [LOCAL ] DTM: Saving primary data. [2025-09-16T18:09:55.060241] [DEBUG] [LOCAL ] DTM: Trimmed 48820 records to 32101 (keeping 150 days of data) [2025-09-16T18:09:55.262248] [DEBUG] [VALIDATOR ] DTW: All expected trading days are present. [2025-09-16T18:09:55.262630] [DEBUG] [VALIDATOR ] DTW: Primary data validation passed. [2025-09-16T18:09:55.263378] [INFO ] [MAIN ] DTW: Fetched primary data is valid. [2025-09-16T18:09:55.264028] [DEBUG] [LOCAL ] DTW: Saving primary data. [2025-09-16T18:09:55.265873] [DEBUG] [LOCAL ] DTW: Trimmed 5242 records to 3569 (keeping 150 days of data) [2025-09-16T18:09:55.294727] [DEBUG] [LOCAL ] DTW: Primary data saved (3569 records). [2025-09-16T18:09:55.295403] [INFO ] [MAIN ] DTW: Full history saved. [2025-09-16T18:09:55.295874] [INFO ] [MAIN ] Update Progress: 662/2369 symbols completed. DTW took 16.5s. Estimated remaining: 3h 0m 10s. Rate: 9.5 symbols/min [2025-09-16T18:09:55.540663] [INFO ] [MAIN ] DV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:55.581581] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-05. [2025-09-16T18:09:55.646573] [DEBUG] [LOCAL ] DV: Loaded 37587 existing records from primary storage. [2025-09-16T18:09:55.647941] [INFO ] [MAIN ] DV: Loaded 37587 existing records. [2025-09-16T18:09:55.752622] [DEBUG] [LOCAL ] DTM: Primary data saved (32101 records). [2025-09-16T18:09:55.752786] [INFO ] [MAIN ] DTM: Full history saved. [2025-09-16T18:09:55.753825] [INFO ] [MAIN ] Update Progress: 663/2369 symbols completed. DTM took 36.6s. Estimated remaining: 2h 59m 48s. Rate: 9.5 symbols/min [2025-09-16T18:09:55.929890] [WARN ] [MAIN ] DV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:09:55.930377] [INFO ] [MAIN ] DV: Wiping existing primary data before full refresh. [2025-09-16T18:09:55.940805] [INFO ] [LOCAL ] DV: Starting deletion of primary data. [2025-09-16T18:09:55.976682] [DEBUG] [LOCAL ] DV: Deleted primary data file. [2025-09-16T18:09:55.999225] [INFO ] [MAIN ] DV: Starting full primary refresh. [2025-09-16T18:09:55.999179] [INFO ] [MAIN ] DVA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:09:56.188124] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-09-16T18:09:56.322080] [DEBUG] [LOCAL ] DVA: Loaded 30220 existing records from primary storage. [2025-09-16T18:09:56.378264] [INFO ] [MAIN ] DVA: Loaded 30220 existing records. [2025-09-16T18:09:56.387182] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-06. [2025-09-16T18:09:56.387130] [INFO ] [REMOTE ] DV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:56.566098] [WARN ] [MAIN ] DVA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:09:56.566409] [INFO ] [MAIN ] DVA: Wiping existing primary data before full refresh. [2025-09-16T18:09:56.566533] [INFO ] [LOCAL ] DVA: Starting deletion of primary data. [2025-09-16T18:09:56.567269] [DEBUG] [LOCAL ] DVA: Deleted primary data file. [2025-09-16T18:09:56.583709] [INFO ] [MAIN ] DVA: Starting full primary refresh. [2025-09-16T18:09:56.696679] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-09-16T18:09:56.957125] [INFO ] [REMOTE ] DVA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:09:57.177086] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-02. [2025-09-16T18:09:57.181159] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-09-16T18:09:57.974924] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-07. [2025-09-16T18:09:58.774876] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-07. [2025-09-16T18:09:58.865897] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T18:09:59.589286] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-02. [2025-09-16T18:09:59.598303] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-09-16T18:09:59.617115] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-09-16T18:10:00.374926] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-08. [2025-09-16T18:10:00.970174] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T18:10:01.174902] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-03. [2025-09-16T18:10:01.455538] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-09-16T18:10:01.974899] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-08. [2025-09-16T18:10:02.774921] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-09. [2025-09-16T18:10:03.056324] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-09-16T18:10:03.322863] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-09-16T18:10:03.324114] [INFO ] [REMOTE ] DUKB: Full fetch complete (6886 records). [2025-09-16T18:10:03.324188] [DEBUG] [VALIDATOR ] DUKB: Validating primary data. [2025-09-16T18:10:03.587827] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-03. [2025-09-16T18:10:03.712900] [DEBUG] [VALIDATOR ] DUKB: All expected trading days are present. [2025-09-16T18:10:03.713179] [DEBUG] [VALIDATOR ] DUKB: Primary data validation passed. [2025-09-16T18:10:03.713373] [INFO ] [MAIN ] DUKB: Fetched primary data is valid. [2025-09-16T18:10:03.713416] [DEBUG] [LOCAL ] DUKB: Saving primary data. [2025-09-16T18:10:03.713851] [DEBUG] [LOCAL ] DUKB: Trimmed 6886 records to 4679 (keeping 150 days of data) [2025-09-16T18:10:03.747262] [DEBUG] [LOCAL ] DUKB: Primary data saved (4679 records). [2025-09-16T18:10:03.747425] [INFO ] [MAIN ] DUKB: Full history saved. [2025-09-16T18:10:03.747560] [INFO ] [MAIN ] Update Progress: 664/2369 symbols completed. DUKB took 21.1s. Estimated remaining: 2h 59m 46s. Rate: 9.5 symbols/min [2025-09-16T18:10:03.937515] [INFO ] [MAIN ] DVN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:10:03.973132] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-09-16T18:10:04.023372] [DEBUG] [LOCAL ] DVN: Loaded 39659 existing records from primary storage. [2025-09-16T18:10:04.023557] [INFO ] [MAIN ] DVN: Loaded 39659 existing records. [2025-09-16T18:10:04.206006] [WARN ] [MAIN ] DVN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:10:04.206272] [INFO ] [MAIN ] DVN: Wiping existing primary data before full refresh. [2025-09-16T18:10:04.206325] [INFO ] [LOCAL ] DVN: Starting deletion of primary data. [2025-09-16T18:10:04.207238] [DEBUG] [LOCAL ] DVN: Deleted primary data file. [2025-09-16T18:10:04.207301] [INFO ] [MAIN ] DVN: Starting full primary refresh. [2025-09-16T18:10:04.379990] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-04. [2025-09-16T18:10:04.598500] [INFO ] [REMOTE ] DVN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:10:05.174917] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-02. [2025-09-16T18:10:05.454652] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-09-16T18:10:05.974957] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-04. [2025-09-16T18:10:06.044292] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T18:10:06.771329] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T18:10:06.780580] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-05. [2025-09-16T18:10:07.574941] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-03. [2025-09-16T18:10:07.661026] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T18:10:08.314282] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T18:10:08.374950] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-05. [2025-09-16T18:10:09.174916] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-06. [2025-09-16T18:10:09.677777] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T18:10:09.752854] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-03 (2.18 seconds). [2025-09-16T18:10:09.974945] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-06. [2025-09-16T18:10:10.311717] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:10:10.774912] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-09. [2025-09-16T18:10:11.574957] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-04. [2025-09-16T18:10:12.374944] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-07. [2025-09-16T18:10:12.722014] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-06 (2.75 seconds). [2025-09-16T18:10:12.748824] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-09-16T18:10:12.755722] [INFO ] [REMOTE ] DUK: Full fetch complete (58155 records). [2025-09-16T18:10:12.755896] [DEBUG] [VALIDATOR ] DUK: Validating primary data. [2025-09-16T18:10:13.139463] [DEBUG] [VALIDATOR ] DUK: All expected trading days are present. [2025-09-16T18:10:13.139542] [DEBUG] [VALIDATOR ] DUK: Primary data validation passed. [2025-09-16T18:10:13.140271] [INFO ] [MAIN ] DUK: Fetched primary data is valid. [2025-09-16T18:10:13.140317] [DEBUG] [LOCAL ] DUK: Saving primary data. [2025-09-16T18:10:13.141469] [DEBUG] [LOCAL ] DUK: Trimmed 58155 records to 39839 (keeping 150 days of data) [2025-09-16T18:10:13.201296] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-07. [2025-09-16T18:10:13.428301] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-09-16T18:10:13.444956] [DEBUG] [LOCAL ] DUK: Primary data saved (39839 records). [2025-09-16T18:10:13.445125] [INFO ] [MAIN ] DUK: Full history saved. [2025-09-16T18:10:13.447248] [INFO ] [MAIN ] Update Progress: 665/2369 symbols completed. DUK took 32.5s. Estimated remaining: 2h 59m 49s. Rate: 9.5 symbols/min [2025-09-16T18:10:13.623015] [INFO ] [MAIN ] DX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:10:13.675992] [DEBUG] [LOCAL ] DX: Loaded 38676 existing records from primary storage. [2025-09-16T18:10:13.676260] [INFO ] [MAIN ] DX: Loaded 38676 existing records. [2025-09-16T18:10:13.868737] [WARN ] [MAIN ] DX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:10:13.868984] [INFO ] [MAIN ] DX: Wiping existing primary data before full refresh. [2025-09-16T18:10:13.869142] [INFO ] [LOCAL ] DX: Starting deletion of primary data. [2025-09-16T18:10:13.878391] [DEBUG] [LOCAL ] DX: Deleted primary data file. [2025-09-16T18:10:13.892614] [INFO ] [MAIN ] DX: Starting full primary refresh. [2025-09-16T18:10:13.969186] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T18:10:13.985905] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-05. [2025-09-16T18:10:14.259235] [INFO ] [REMOTE ] DX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:10:14.774887] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-02. [2025-09-16T18:10:14.960833] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T18:10:15.574987] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-08. [2025-09-16T18:10:16.074907] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-05 (2.09 seconds). [2025-09-16T18:10:16.374902] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-06. [2025-09-16T18:10:16.591244] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-09-16T18:10:16.731579] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T18:10:17.174910] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-08. [2025-09-16T18:10:17.974939] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-09. [2025-09-16T18:10:18.436856] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-09-16T18:10:18.597206] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T18:10:18.774888] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-07. [2025-09-16T18:10:19.364718] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T18:10:19.370609] [INFO ] [REMOTE ] DVA: Full fetch complete (45950 records). [2025-09-16T18:10:19.370794] [DEBUG] [VALIDATOR ] DVA: Validating primary data. [2025-09-16T18:10:19.584207] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-03. [2025-09-16T18:10:19.761047] [DEBUG] [VALIDATOR ] DVA: All expected trading days are present. [2025-09-16T18:10:19.761122] [DEBUG] [VALIDATOR ] DVA: Primary data validation passed. [2025-09-16T18:10:19.761755] [INFO ] [MAIN ] DVA: Fetched primary data is valid. [2025-09-16T18:10:19.761800] [DEBUG] [LOCAL ] DVA: Saving primary data. [2025-09-16T18:10:19.762758] [DEBUG] [LOCAL ] DVA: Trimmed 45950 records to 30522 (keeping 150 days of data) [2025-09-16T18:10:19.944620] [DEBUG] [LOCAL ] DVA: Primary data saved (30522 records). [2025-09-16T18:10:19.944773] [INFO ] [MAIN ] DVA: Full history saved. [2025-09-16T18:10:19.946701] [INFO ] [MAIN ] Update Progress: 666/2369 symbols completed. DVA took 24.2s. Estimated remaining: 2h 59m 43s. Rate: 9.5 symbols/min [2025-09-16T18:10:20.116780] [INFO ] [MAIN ] DXC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:10:20.155470] [DEBUG] [LOCAL ] DXC: Loaded 36743 existing records from primary storage. [2025-09-16T18:10:20.155632] [INFO ] [MAIN ] DXC: Loaded 36743 existing records. [2025-09-16T18:10:20.335758] [WARN ] [MAIN ] DXC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:10:20.336048] [INFO ] [MAIN ] DXC: Wiping existing primary data before full refresh. [2025-09-16T18:10:20.336094] [INFO ] [LOCAL ] DXC: Starting deletion of primary data. [2025-09-16T18:10:20.336868] [DEBUG] [LOCAL ] DXC: Deleted primary data file. [2025-09-16T18:10:20.336922] [INFO ] [MAIN ] DXC: Starting full primary refresh. [2025-09-16T18:10:20.383005] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-09. [2025-09-16T18:10:20.696439] [INFO ] [REMOTE ] DXC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:10:21.042355] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-07 (2.27 seconds). [2025-09-16T18:10:21.174905] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-02. [2025-09-16T18:10:21.539689] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-09-16T18:10:21.605029] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-09-16T18:10:21.611538] [INFO ] [REMOTE ] DV: Full fetch complete (55583 records). [2025-09-16T18:10:21.611752] [DEBUG] [VALIDATOR ] DV: Validating primary data. [2025-09-16T18:10:21.975966] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-04. [2025-09-16T18:10:22.011521] [DEBUG] [VALIDATOR ] DV: All expected trading days are present. [2025-09-16T18:10:22.011614] [DEBUG] [VALIDATOR ] DV: Primary data validation passed. [2025-09-16T18:10:22.012353] [INFO ] [MAIN ] DV: Fetched primary data is valid. [2025-09-16T18:10:22.012402] [DEBUG] [LOCAL ] DV: Saving primary data. [2025-09-16T18:10:22.014583] [DEBUG] [LOCAL ] DV: Trimmed 55583 records to 37966 (keeping 150 days of data) [2025-09-16T18:10:22.245378] [DEBUG] [LOCAL ] DV: Primary data saved (37966 records). [2025-09-16T18:10:22.245536] [INFO ] [MAIN ] DV: Full history saved. [2025-09-16T18:10:22.245625] [INFO ] [MAIN ] Update Progress: 667/2369 symbols completed. DV took 26.9s. Estimated remaining: 2h 59m 26s. Rate: 9.5 symbols/min [2025-09-16T18:10:22.428317] [INFO ] [MAIN ] DXYZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:10:22.455997] [DEBUG] [LOCAL ] DXYZ: Loaded 26469 existing records from primary storage. [2025-09-16T18:10:22.456181] [INFO ] [MAIN ] DXYZ: Loaded 26469 existing records. [2025-09-16T18:10:22.598137] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T18:10:22.685111] [WARN ] [MAIN ] DXYZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:10:22.685338] [INFO ] [MAIN ] DXYZ: Wiping existing primary data before full refresh. [2025-09-16T18:10:22.685385] [INFO ] [LOCAL ] DXYZ: Starting deletion of primary data. [2025-09-16T18:10:22.685982] [DEBUG] [LOCAL ] DXYZ: Deleted primary data file. [2025-09-16T18:10:22.686029] [INFO ] [MAIN ] DXYZ: Starting full primary refresh. [2025-09-16T18:10:22.783019] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-08. [2025-09-16T18:10:23.037696] [INFO ] [REMOTE ] DXYZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:10:23.574902] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-03. [2025-09-16T18:10:23.952156] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-09-16T18:10:24.374921] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-05. [2025-09-16T18:10:24.840717] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-09-16T18:10:25.018190] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T18:10:25.174840] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-04. [2025-09-16T18:10:25.825119] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T18:10:25.974841] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-06. [2025-09-16T18:10:26.254940] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-09-16T18:10:26.774977] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-05. [2025-09-16T18:10:27.574923] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-09. [2025-09-16T18:10:27.877013] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-09-16T18:10:28.248034] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T18:10:28.374872] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-06. [2025-09-16T18:10:29.174901] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-02. [2025-09-16T18:10:29.523261] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-09-16T18:10:29.530112] [INFO ] [REMOTE ] DVN: Full fetch complete (58379 records). [2025-09-16T18:10:29.530264] [DEBUG] [VALIDATOR ] DVN: Validating primary data. [2025-09-16T18:10:29.925814] [DEBUG] [VALIDATOR ] DVN: All expected trading days are present. [2025-09-16T18:10:29.925990] [DEBUG] [VALIDATOR ] DVN: Primary data validation passed. [2025-09-16T18:10:29.926820] [INFO ] [MAIN ] DVN: Fetched primary data is valid. [2025-09-16T18:10:29.926959] [DEBUG] [LOCAL ] DVN: Saving primary data. [2025-09-16T18:10:29.928453] [DEBUG] [LOCAL ] DVN: Trimmed 58379 records to 40049 (keeping 150 days of data) [2025-09-16T18:10:30.008878] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-07. [2025-09-16T18:10:30.051240] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T18:10:30.226132] [DEBUG] [LOCAL ] DVN: Primary data saved (40049 records). [2025-09-16T18:10:30.226265] [INFO ] [MAIN ] DVN: Full history saved. [2025-09-16T18:10:30.226348] [INFO ] [MAIN ] Update Progress: 668/2369 symbols completed. DVN took 26.5s. Estimated remaining: 2h 59m 24s. Rate: 9.5 symbols/min [2025-09-16T18:10:30.401615] [INFO ] [MAIN ] DY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:10:30.423739] [DEBUG] [LOCAL ] DY: Loaded 20943 existing records from primary storage. [2025-09-16T18:10:30.423966] [INFO ] [MAIN ] DY: Loaded 20943 existing records. [2025-09-16T18:10:30.611715] [WARN ] [MAIN ] DY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:10:30.611949] [INFO ] [MAIN ] DY: Wiping existing primary data before full refresh. [2025-09-16T18:10:30.611995] [INFO ] [LOCAL ] DY: Starting deletion of primary data. [2025-09-16T18:10:30.612557] [DEBUG] [LOCAL ] DY: Deleted primary data file. [2025-09-16T18:10:30.612602] [INFO ] [MAIN ] DY: Starting full primary refresh. [2025-09-16T18:10:30.784928] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-07. [2025-09-16T18:10:30.968074] [INFO ] [REMOTE ] DY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:10:31.017802] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-09-16T18:10:31.574902] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-02. [2025-09-16T18:10:31.984771] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-09-16T18:10:32.374927] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-03. [2025-09-16T18:10:32.430289] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T18:10:32.731968] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-09-16T18:10:33.174901] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-03. [2025-09-16T18:10:33.974920] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-08. [2025-09-16T18:10:34.062279] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-09-16T18:10:34.503630] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T18:10:34.774953] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-04. [2025-09-16T18:10:35.574903] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-08. [2025-09-16T18:10:35.741573] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T18:10:36.152322] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-09-16T18:10:36.374935] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-04. [2025-09-16T18:10:37.174893] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-09. [2025-09-16T18:10:37.244910] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:10:37.974917] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-09. [2025-09-16T18:10:38.133833] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T18:10:38.674852] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:10:38.683046] [INFO ] [REMOTE ] DXC: Full fetch complete (53294 records). [2025-09-16T18:10:38.683234] [DEBUG] [VALIDATOR ] DXC: Validating primary data. [2025-09-16T18:10:38.779993] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-05. [2025-09-16T18:10:39.092067] [DEBUG] [VALIDATOR ] DXC: All expected trading days are present. [2025-09-16T18:10:39.092150] [DEBUG] [VALIDATOR ] DXC: Primary data validation passed. [2025-09-16T18:10:39.092897] [INFO ] [MAIN ] DXC: Fetched primary data is valid. [2025-09-16T18:10:39.092944] [DEBUG] [LOCAL ] DXC: Saving primary data. [2025-09-16T18:10:39.094115] [DEBUG] [LOCAL ] DXC: Trimmed 53294 records to 37119 (keeping 150 days of data) [2025-09-16T18:10:39.316709] [DEBUG] [LOCAL ] DXC: Primary data saved (37119 records). [2025-09-16T18:10:39.316870] [INFO ] [MAIN ] DXC: Full history saved. [2025-09-16T18:10:39.317506] [INFO ] [MAIN ] Update Progress: 669/2369 symbols completed. DXC took 19.4s. Estimated remaining: 2h 59m 25s. Rate: 9.5 symbols/min [2025-09-16T18:10:39.487877] [INFO ] [MAIN ] E: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:10:39.512426] [DEBUG] [LOCAL ] E: Loaded 24276 existing records from primary storage. [2025-09-16T18:10:39.512577] [INFO ] [MAIN ] E: Loaded 24276 existing records. [2025-09-16T18:10:39.581234] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-05. [2025-09-16T18:10:39.715732] [WARN ] [MAIN ] E: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:10:39.715979] [INFO ] [MAIN ] E: Wiping existing primary data before full refresh. [2025-09-16T18:10:39.716032] [INFO ] [LOCAL ] E: Starting deletion of primary data. [2025-09-16T18:10:39.716611] [DEBUG] [LOCAL ] E: Deleted primary data file. [2025-09-16T18:10:39.716688] [INFO ] [MAIN ] E: Starting full primary refresh. [2025-09-16T18:10:40.071219] [INFO ] [REMOTE ] E: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:10:40.087232] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-09 (2.11 seconds). [2025-09-16T18:10:40.093339] [INFO ] [REMOTE ] DX: Full fetch complete (57002 records). [2025-09-16T18:10:40.093444] [DEBUG] [VALIDATOR ] DX: Validating primary data. [2025-09-16T18:10:40.380085] [INFO ] [REMOTE ] E: Fetching primary data for 2025-02. [2025-09-16T18:10:40.483055] [DEBUG] [VALIDATOR ] DX: All expected trading days are present. [2025-09-16T18:10:40.483154] [DEBUG] [VALIDATOR ] DX: Primary data validation passed. [2025-09-16T18:10:40.483977] [INFO ] [MAIN ] DX: Fetched primary data is valid. [2025-09-16T18:10:40.484027] [DEBUG] [LOCAL ] DX: Saving primary data. [2025-09-16T18:10:40.485296] [DEBUG] [LOCAL ] DX: Trimmed 57002 records to 39065 (keeping 150 days of data) [2025-09-16T18:10:40.632151] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-09-16T18:10:40.771245] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-09-16T18:10:40.791736] [DEBUG] [LOCAL ] DX: Primary data saved (39065 records). [2025-09-16T18:10:40.791866] [INFO ] [MAIN ] DX: Full history saved. [2025-09-16T18:10:40.791951] [INFO ] [MAIN ] Update Progress: 670/2369 symbols completed. DX took 27.3s. Estimated remaining: 2h 59m 6s. Rate: 9.5 symbols/min [2025-09-16T18:10:40.966458] [INFO ] [MAIN ] EAF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:10:40.997519] [DEBUG] [LOCAL ] EAF: Loaded 32555 existing records from primary storage. [2025-09-16T18:10:40.997690] [INFO ] [MAIN ] EAF: Loaded 32555 existing records. [2025-09-16T18:10:41.176462] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-06. [2025-09-16T18:10:41.179918] [WARN ] [MAIN ] EAF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:10:41.180814] [INFO ] [MAIN ] EAF: Wiping existing primary data before full refresh. [2025-09-16T18:10:41.180865] [INFO ] [LOCAL ] EAF: Starting deletion of primary data. [2025-09-16T18:10:41.181594] [DEBUG] [LOCAL ] EAF: Deleted primary data file. [2025-09-16T18:10:41.181671] [INFO ] [MAIN ] EAF: Starting full primary refresh. [2025-09-16T18:10:41.544093] [INFO ] [REMOTE ] EAF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:10:41.599608] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T18:10:41.974917] [INFO ] [REMOTE ] E: Fetching primary data for 2025-03. [2025-09-16T18:10:42.774904] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-02. [2025-09-16T18:10:43.204844] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-09-16T18:10:43.365278] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T18:10:43.575199] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-07. [2025-09-16T18:10:44.374917] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-06. [2025-09-16T18:10:44.410677] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:10:45.174944] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-03. [2025-09-16T18:10:45.681476] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-09-16T18:10:45.708137] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T18:10:45.974914] [INFO ] [REMOTE ] E: Fetching primary data for 2025-04. [2025-09-16T18:10:46.347221] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T18:10:46.774904] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-07. [2025-09-16T18:10:47.505707] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-09-16T18:10:47.574881] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-04. [2025-09-16T18:10:47.978827] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T18:10:48.374914] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-08. [2025-09-16T18:10:48.990193] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T18:10:49.174919] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-08. [2025-09-16T18:10:49.670477] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-09-16T18:10:49.974950] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-05. [2025-09-16T18:10:50.774901] [INFO ] [REMOTE ] E: Fetching primary data for 2025-05. [2025-09-16T18:10:50.960967] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T18:10:51.535193] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T18:10:51.574863] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-06. [2025-09-16T18:10:52.217175] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T18:10:52.374909] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-09. [2025-09-16T18:10:52.805326] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T18:10:53.174932] [INFO ] [REMOTE ] E: Fetching primary data for 2025-06. [2025-09-16T18:10:53.398787] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T18:10:53.404318] [INFO ] [REMOTE ] DY: Full fetch complete (33429 records). [2025-09-16T18:10:53.404448] [DEBUG] [VALIDATOR ] DY: Validating primary data. [2025-09-16T18:10:53.779854] [DEBUG] [VALIDATOR ] DY: All expected trading days are present. [2025-09-16T18:10:53.779939] [DEBUG] [VALIDATOR ] DY: Primary data validation passed. [2025-09-16T18:10:53.780464] [INFO ] [MAIN ] DY: Fetched primary data is valid. [2025-09-16T18:10:53.780507] [DEBUG] [LOCAL ] DY: Saving primary data. [2025-09-16T18:10:53.781301] [DEBUG] [LOCAL ] DY: Trimmed 33429 records to 21107 (keeping 150 days of data) [2025-09-16T18:10:53.907980] [DEBUG] [LOCAL ] DY: Primary data saved (21107 records). [2025-09-16T18:10:53.908110] [INFO ] [MAIN ] DY: Full history saved. [2025-09-16T18:10:53.908191] [INFO ] [MAIN ] Update Progress: 671/2369 symbols completed. DY took 23.7s. Estimated remaining: 2h 59m 17s. Rate: 9.5 symbols/min [2025-09-16T18:10:53.980061] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-09. [2025-09-16T18:10:54.086477] [INFO ] [MAIN ] EAI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:10:54.090984] [DEBUG] [LOCAL ] EAI: Loaded 2827 existing records from primary storage. [2025-09-16T18:10:54.091063] [INFO ] [MAIN ] EAI: Loaded 2827 existing records. [2025-09-16T18:10:54.266114] [WARN ] [MAIN ] EAI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:10:54.266323] [INFO ] [MAIN ] EAI: Wiping existing primary data before full refresh. [2025-09-16T18:10:54.266389] [INFO ] [LOCAL ] EAI: Starting deletion of primary data. [2025-09-16T18:10:54.266614] [DEBUG] [LOCAL ] EAI: Deleted primary data file. [2025-09-16T18:10:54.266680] [INFO ] [MAIN ] EAI: Starting full primary refresh. [2025-09-16T18:10:54.615514] [INFO ] [REMOTE ] EAI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:10:54.652409] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T18:10:54.774849] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-02. [2025-09-16T18:10:55.197183] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-09-16T18:10:55.327052] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T18:10:55.331368] [INFO ] [REMOTE ] DXYZ: Full fetch complete (40513 records). [2025-09-16T18:10:55.331447] [DEBUG] [VALIDATOR ] DXYZ: Validating primary data. [2025-09-16T18:10:55.580063] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-03. [2025-09-16T18:10:55.707680] [DEBUG] [VALIDATOR ] DXYZ: All expected trading days are present. [2025-09-16T18:10:55.707879] [DEBUG] [VALIDATOR ] DXYZ: Primary data validation passed. [2025-09-16T18:10:55.708513] [INFO ] [MAIN ] DXYZ: Fetched primary data is valid. [2025-09-16T18:10:55.708557] [DEBUG] [LOCAL ] DXYZ: Saving primary data. [2025-09-16T18:10:55.709466] [DEBUG] [LOCAL ] DXYZ: Trimmed 40513 records to 26694 (keeping 150 days of data) [2025-09-16T18:10:55.864707] [DEBUG] [LOCAL ] DXYZ: Primary data saved (26694 records). [2025-09-16T18:10:55.864842] [INFO ] [MAIN ] DXYZ: Full history saved. [2025-09-16T18:10:55.864926] [INFO ] [MAIN ] Update Progress: 672/2369 symbols completed. DXYZ took 33.6s. Estimated remaining: 2h 58m 60s. Rate: 9.5 symbols/min [2025-09-16T18:10:56.034224] [INFO ] [MAIN ] EARN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:10:56.052962] [DEBUG] [LOCAL ] EARN: Loaded 20059 existing records from primary storage. [2025-09-16T18:10:56.053111] [INFO ] [MAIN ] EARN: Loaded 20059 existing records. [2025-09-16T18:10:56.229535] [WARN ] [MAIN ] EARN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:10:56.233825] [INFO ] [MAIN ] EARN: Wiping existing primary data before full refresh. [2025-09-16T18:10:56.234360] [INFO ] [LOCAL ] EARN: Starting deletion of primary data. [2025-09-16T18:10:56.234599] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-09-16T18:10:56.235832] [DEBUG] [LOCAL ] EARN: Deleted primary data file. [2025-09-16T18:10:56.235947] [INFO ] [MAIN ] EARN: Starting full primary refresh. [2025-09-16T18:10:56.379157] [INFO ] [REMOTE ] E: Fetching primary data for 2025-07. [2025-09-16T18:10:56.589411] [INFO ] [REMOTE ] EARN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:10:57.174961] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-02. [2025-09-16T18:10:57.830371] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T18:10:57.974888] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-04. [2025-09-16T18:10:58.370540] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-09-16T18:10:58.598978] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-09-16T18:10:58.774891] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-05. [2025-09-16T18:10:59.402104] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-09-16T18:10:59.574944] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-03. [2025-09-16T18:11:00.374914] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-07. [2025-09-16T18:11:00.775133] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T18:11:01.174902] [INFO ] [REMOTE ] E: Fetching primary data for 2025-08. [2025-09-16T18:11:01.974999] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-06. [2025-09-16T18:11:02.284265] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-09-16T18:11:02.328111] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T18:11:02.774956] [INFO ] [REMOTE ] E: Fetching primary data for 2025-09. [2025-09-16T18:11:02.822512] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-07 (2.45 seconds). [2025-09-16T18:11:03.575011] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-07. [2025-09-16T18:11:04.049332] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-09-16T18:11:04.098408] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T18:11:04.101879] [INFO ] [REMOTE ] E: Full fetch complete (36179 records). [2025-09-16T18:11:04.101959] [DEBUG] [VALIDATOR ] E: Validating primary data. [2025-09-16T18:11:04.102523] [FATAL] [VALIDATOR ] E: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:11:04.102612] [FATAL] [MAIN ] E: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:11:04.102702] [WARN ] [MAIN ] E: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:11:04.102767] [INFO ] [LOCAL ] E: Starting deletion of primary data. [2025-09-16T18:11:04.102853] [DEBUG] [LOCAL ] E: No primary data file found to delete. [2025-09-16T18:11:04.102892] [INFO ] [LOCAL ] E: Starting deletion of derivative data. [2025-09-16T18:11:04.103224] [DEBUG] [LOCAL ] E: Deleted derivative data file. [2025-09-16T18:11:04.103319] [INFO ] [MAIN ] Update Progress: 673/2369 symbols completed. E took 24.8s. Estimated remaining: 2h 58m 58s. Rate: 9.5 symbols/min [2025-09-16T18:11:04.271706] [INFO ] [MAIN ] EAT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:11:04.312037] [DEBUG] [LOCAL ] EAT: Loaded 36616 existing records from primary storage. [2025-09-16T18:11:04.312177] [INFO ] [MAIN ] EAT: Loaded 36616 existing records. [2025-09-16T18:11:04.380104] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-04. [2025-09-16T18:11:04.491123] [WARN ] [MAIN ] EAT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:11:04.491356] [INFO ] [MAIN ] EAT: Wiping existing primary data before full refresh. [2025-09-16T18:11:04.491400] [INFO ] [LOCAL ] EAT: Starting deletion of primary data. [2025-09-16T18:11:04.492203] [DEBUG] [LOCAL ] EAT: Deleted primary data file. [2025-09-16T18:11:04.492252] [INFO ] [MAIN ] EAT: Starting full primary refresh. [2025-09-16T18:11:04.844021] [INFO ] [REMOTE ] EAT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:11:05.174912] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-02. [2025-09-16T18:11:05.857529] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T18:11:05.974880] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-05. [2025-09-16T18:11:06.774863] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-08. [2025-09-16T18:11:06.845034] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T18:11:07.402244] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-09-16T18:11:07.574980] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-06. [2025-09-16T18:11:07.948661] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T18:11:08.374956] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-03. [2025-09-16T18:11:08.490636] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-09-16T18:11:09.174914] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-08. [2025-09-16T18:11:09.641245] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-09-16T18:11:09.776171] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T18:11:09.974873] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-09. [2025-09-16T18:11:10.539620] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T18:11:10.540755] [INFO ] [REMOTE ] EAI: Full fetch complete (4340 records). [2025-09-16T18:11:10.540822] [DEBUG] [VALIDATOR ] EAI: Validating primary data. [2025-09-16T18:11:10.780049] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-04. [2025-09-16T18:11:10.906673] [DEBUG] [VALIDATOR ] EAI: All expected trading days are present. [2025-09-16T18:11:10.906754] [DEBUG] [VALIDATOR ] EAI: Primary data validation passed. [2025-09-16T18:11:10.906903] [INFO ] [MAIN ] EAI: Fetched primary data is valid. [2025-09-16T18:11:10.906958] [DEBUG] [LOCAL ] EAI: Saving primary data. [2025-09-16T18:11:10.907357] [DEBUG] [LOCAL ] EAI: Trimmed 4340 records to 2911 (keeping 150 days of data) [2025-09-16T18:11:10.926995] [DEBUG] [LOCAL ] EAI: Primary data saved (2911 records). [2025-09-16T18:11:10.927073] [INFO ] [MAIN ] EAI: Full history saved. [2025-09-16T18:11:10.927150] [INFO ] [MAIN ] Update Progress: 674/2369 symbols completed. EAI took 17.0s. Estimated remaining: 2h 58m 53s. Rate: 9.5 symbols/min [2025-09-16T18:11:11.098333] [INFO ] [MAIN ] EB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:11:11.122916] [DEBUG] [LOCAL ] EB: Loaded 26972 existing records from primary storage. [2025-09-16T18:11:11.123056] [INFO ] [MAIN ] EB: Loaded 26972 existing records. [2025-09-16T18:11:11.301095] [WARN ] [MAIN ] EB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:11:11.301337] [INFO ] [MAIN ] EB: Wiping existing primary data before full refresh. [2025-09-16T18:11:11.301382] [INFO ] [LOCAL ] EB: Starting deletion of primary data. [2025-09-16T18:11:11.301950] [DEBUG] [LOCAL ] EB: Deleted primary data file. [2025-09-16T18:11:11.302009] [INFO ] [MAIN ] EB: Starting full primary refresh. [2025-09-16T18:11:11.584466] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-07. [2025-09-16T18:11:11.665491] [INFO ] [REMOTE ] EB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:11:12.266390] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T18:11:12.374888] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-05. [2025-09-16T18:11:12.570045] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-09-16T18:11:13.174897] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-09. [2025-09-16T18:11:13.835506] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T18:11:13.974907] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-08. [2025-09-16T18:11:14.523239] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T18:11:14.526582] [INFO ] [REMOTE ] EAF: Full fetch complete (48473 records). [2025-09-16T18:11:14.526721] [DEBUG] [VALIDATOR ] EAF: Validating primary data. [2025-09-16T18:11:14.527356] [FATAL] [VALIDATOR ] EAF: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:11:14.527434] [FATAL] [MAIN ] EAF: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:11:14.527472] [WARN ] [MAIN ] EAF: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:11:14.527520] [INFO ] [LOCAL ] EAF: Starting deletion of primary data. [2025-09-16T18:11:14.527600] [DEBUG] [LOCAL ] EAF: No primary data file found to delete. [2025-09-16T18:11:14.527660] [INFO ] [LOCAL ] EAF: Starting deletion of derivative data. [2025-09-16T18:11:14.527935] [DEBUG] [LOCAL ] EAF: Deleted derivative data file. [2025-09-16T18:11:14.528031] [INFO ] [MAIN ] Update Progress: 675/2369 symbols completed. EAF took 33.7s. Estimated remaining: 2h 58m 40s. Rate: 9.5 symbols/min [2025-09-16T18:11:14.697808] [INFO ] [MAIN ] EBF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:11:14.712347] [DEBUG] [LOCAL ] EBF: Loaded 13862 existing records from primary storage. [2025-09-16T18:11:14.712457] [INFO ] [MAIN ] EBF: Loaded 13862 existing records. [2025-09-16T18:11:14.776836] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-06. [2025-09-16T18:11:14.903988] [WARN ] [MAIN ] EBF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:11:14.919415] [INFO ] [MAIN ] EBF: Wiping existing primary data before full refresh. [2025-09-16T18:11:14.919677] [INFO ] [LOCAL ] EBF: Starting deletion of primary data. [2025-09-16T18:11:14.920841] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-09-16T18:11:14.920881] [DEBUG] [LOCAL ] EBF: Deleted primary data file. [2025-09-16T18:11:14.926388] [INFO ] [MAIN ] EBF: Starting full primary refresh. [2025-09-16T18:11:15.278929] [INFO ] [REMOTE ] EBF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:11:15.574908] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-09. [2025-09-16T18:11:16.228333] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T18:11:16.374847] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-07. [2025-09-16T18:11:16.418721] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T18:11:16.421547] [INFO ] [REMOTE ] EARN: Full fetch complete (34252 records). [2025-09-16T18:11:16.421623] [DEBUG] [VALIDATOR ] EARN: Validating primary data. [2025-09-16T18:11:16.422210] [FATAL] [VALIDATOR ] EARN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:11:16.422304] [FATAL] [MAIN ] EARN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:11:16.422361] [WARN ] [MAIN ] EARN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:11:16.422426] [INFO ] [LOCAL ] EARN: Starting deletion of primary data. [2025-09-16T18:11:16.422491] [DEBUG] [LOCAL ] EARN: No primary data file found to delete. [2025-09-16T18:11:16.422530] [INFO ] [LOCAL ] EARN: Starting deletion of derivative data. [2025-09-16T18:11:16.422830] [DEBUG] [LOCAL ] EARN: Deleted derivative data file. [2025-09-16T18:11:16.422927] [INFO ] [MAIN ] Update Progress: 676/2369 symbols completed. EARN took 20.6s. Estimated remaining: 2h 58m 22s. Rate: 9.5 symbols/min [2025-09-16T18:11:16.593484] [INFO ] [MAIN ] EBR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:11:16.624754] [DEBUG] [LOCAL ] EBR: Loaded 31335 existing records from primary storage. [2025-09-16T18:11:16.624925] [INFO ] [MAIN ] EBR: Loaded 31335 existing records. [2025-09-16T18:11:16.801555] [WARN ] [MAIN ] EBR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:11:16.801819] [INFO ] [MAIN ] EBR: Wiping existing primary data before full refresh. [2025-09-16T18:11:16.801866] [INFO ] [LOCAL ] EBR: Starting deletion of primary data. [2025-09-16T18:11:16.802561] [DEBUG] [LOCAL ] EBR: Deleted primary data file. [2025-09-16T18:11:16.802610] [INFO ] [MAIN ] EBR: Starting full primary refresh. [2025-09-16T18:11:17.188389] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-02. [2025-09-16T18:11:17.193755] [INFO ] [REMOTE ] EBR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:11:17.974940] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-02. [2025-09-16T18:11:18.157683] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-09-16T18:11:18.328376] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-09-16T18:11:18.774912] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-08. [2025-09-16T18:11:18.841379] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T18:11:19.574907] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-03. [2025-09-16T18:11:20.094767] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-09-16T18:11:20.374911] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-03. [2025-09-16T18:11:20.705374] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-09-16T18:11:21.174899] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-04. [2025-09-16T18:11:21.481842] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-09-16T18:11:21.974915] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-09. [2025-09-16T18:11:22.298378] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T18:11:22.774899] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-05. [2025-09-16T18:11:23.411295] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T18:11:23.416828] [INFO ] [REMOTE ] EAT: Full fetch complete (53943 records). [2025-09-16T18:11:23.416928] [DEBUG] [VALIDATOR ] EAT: Validating primary data. [2025-09-16T18:11:23.417585] [FATAL] [VALIDATOR ] EAT: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:11:23.417689] [FATAL] [MAIN ] EAT: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:11:23.417731] [WARN ] [MAIN ] EAT: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:11:23.417772] [INFO ] [LOCAL ] EAT: Starting deletion of primary data. [2025-09-16T18:11:23.417832] [DEBUG] [LOCAL ] EAT: No primary data file found to delete. [2025-09-16T18:11:23.417871] [INFO ] [LOCAL ] EAT: Starting deletion of derivative data. [2025-09-16T18:11:23.418157] [DEBUG] [LOCAL ] EAT: Deleted derivative data file. [2025-09-16T18:11:23.418819] [INFO ] [MAIN ] Update Progress: 677/2369 symbols completed. EAT took 19.3s. Estimated remaining: 2h 58m 18s. Rate: 9.5 symbols/min [2025-09-16T18:11:23.574906] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-04. [2025-09-16T18:11:23.830311] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-09-16T18:11:24.349624] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-09-16T18:11:24.374874] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-05. [2025-09-16T18:11:25.119353] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-09-16T18:11:25.174888] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-06. [2025-09-16T18:11:25.860158] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-09-16T18:11:25.974904] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-06. [2025-09-16T18:11:26.774907] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-02. [2025-09-16T18:11:27.156990] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-09-16T18:11:27.574898] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-07. [2025-09-16T18:11:28.376933] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-07. [2025-09-16T18:11:28.380085] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T18:11:29.095216] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-09-16T18:11:29.174870] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-03. [2025-09-16T18:11:29.236481] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-09-16T18:11:29.974905] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-08. [2025-09-16T18:11:30.539516] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T18:11:30.774890] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-04. [2025-09-16T18:11:31.247242] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-09-16T18:11:31.574895] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-08. [2025-09-16T18:11:32.173743] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T18:11:32.374905] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-09. [2025-09-16T18:11:32.618583] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-09-16T18:11:33.174957] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-09. [2025-09-16T18:11:33.619446] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T18:11:33.623323] [INFO ] [REMOTE ] EB: Full fetch complete (41476 records). [2025-09-16T18:11:33.623418] [DEBUG] [VALIDATOR ] EB: Validating primary data. [2025-09-16T18:11:33.987097] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-05. [2025-09-16T18:11:34.002063] [DEBUG] [VALIDATOR ] EB: All expected trading days are present. [2025-09-16T18:11:34.002137] [DEBUG] [VALIDATOR ] EB: Primary data validation passed. [2025-09-16T18:11:34.002763] [INFO ] [MAIN ] EB: Fetched primary data is valid. [2025-09-16T18:11:34.002886] [DEBUG] [LOCAL ] EB: Saving primary data. [2025-09-16T18:11:34.003988] [DEBUG] [LOCAL ] EB: Trimmed 41476 records to 27190 (keeping 150 days of data) [2025-09-16T18:11:34.164510] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T18:11:34.168318] [DEBUG] [LOCAL ] EB: Primary data saved (27190 records). [2025-09-16T18:11:34.168614] [INFO ] [MAIN ] EB: Full history saved. [2025-09-16T18:11:34.168713] [INFO ] [MAIN ] Update Progress: 678/2369 symbols completed. EB took 23.2s. Estimated remaining: 2h 58m 22s. Rate: 9.5 symbols/min [2025-09-16T18:11:34.168794] [INFO ] [REMOTE ] EBF: Full fetch complete (21435 records). [2025-09-16T18:11:34.168963] [DEBUG] [VALIDATOR ] EBF: Validating primary data. [2025-09-16T18:11:34.550565] [DEBUG] [VALIDATOR ] EBF: All expected trading days are present. [2025-09-16T18:11:34.550664] [DEBUG] [VALIDATOR ] EBF: Primary data validation passed. [2025-09-16T18:11:34.551099] [INFO ] [MAIN ] EBF: Fetched primary data is valid. [2025-09-16T18:11:34.551141] [DEBUG] [LOCAL ] EBF: Saving primary data. [2025-09-16T18:11:34.551858] [DEBUG] [LOCAL ] EBF: Trimmed 21435 records to 13951 (keeping 150 days of data) [2025-09-16T18:11:34.635876] [DEBUG] [LOCAL ] EBF: Primary data saved (13951 records). [2025-09-16T18:11:34.636023] [INFO ] [MAIN ] EBF: Full history saved. [2025-09-16T18:11:34.636110] [INFO ] [MAIN ] Update Progress: 679/2369 symbols completed. EBF took 20.1s. Estimated remaining: 2h 58m 1s. Rate: 9.5 symbols/min [2025-09-16T18:11:35.512929] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T18:11:35.522369] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-06. [2025-09-16T18:11:37.045884] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T18:11:37.054315] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-07. [2025-09-16T18:11:38.677510] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-09-16T18:11:38.690294] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-08. [2025-09-16T18:11:40.322448] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-09-16T18:11:40.332067] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-09. [2025-09-16T18:11:41.873397] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T18:11:41.878576] [INFO ] [REMOTE ] EBR: Full fetch complete (46456 records). [2025-09-16T18:11:41.878686] [DEBUG] [VALIDATOR ] EBR: Validating primary data. [2025-09-16T18:11:42.268205] [DEBUG] [VALIDATOR ] EBR: All expected trading days are present. [2025-09-16T18:11:42.268282] [DEBUG] [VALIDATOR ] EBR: Primary data validation passed. [2025-09-16T18:11:42.268914] [INFO ] [MAIN ] EBR: Fetched primary data is valid. [2025-09-16T18:11:42.268959] [DEBUG] [LOCAL ] EBR: Saving primary data. [2025-09-16T18:11:42.269932] [DEBUG] [LOCAL ] EBR: Trimmed 46456 records to 31640 (keeping 150 days of data) [2025-09-16T18:11:42.450492] [DEBUG] [LOCAL ] EBR: Primary data saved (31640 records). [2025-09-16T18:11:42.450634] [INFO ] [MAIN ] EBR: Full history saved. [2025-09-16T18:11:42.450737] [INFO ] [MAIN ] Update Progress: 680/2369 symbols completed. EBR took 26.0s. Estimated remaining: 2h 57m 59s. Rate: 9.5 symbols/min [2025-09-16T18:11:43.147598] [INFO ] [MAIN ] ECC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:11:43.207763] [INFO ] [MAIN ] EBS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:11:43.180219] [INFO ] [MAIN ] ECAT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:11:43.241874] [INFO ] [MAIN ] EC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:11:43.301617] [DEBUG] [LOCAL ] ECAT: Loaded 21685 existing records from primary storage. [2025-09-16T18:11:43.302319] [INFO ] [MAIN ] ECAT: Loaded 21685 existing records. [2025-09-16T18:11:43.369951] [DEBUG] [LOCAL ] ECC: Loaded 31558 existing records from primary storage. [2025-09-16T18:11:43.395934] [INFO ] [MAIN ] ECC: Loaded 31558 existing records. [2025-09-16T18:11:43.443969] [DEBUG] [LOCAL ] EC: Loaded 36092 existing records from primary storage. [2025-09-16T18:11:43.460349] [DEBUG] [LOCAL ] EBS: Loaded 34806 existing records from primary storage. [2025-09-16T18:11:43.500102] [INFO ] [MAIN ] EC: Loaded 36092 existing records. [2025-09-16T18:11:43.519823] [INFO ] [MAIN ] EBS: Loaded 34806 existing records. [2025-09-16T18:11:43.778454] [WARN ] [MAIN ] ECAT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:11:43.926290] [WARN ] [MAIN ] ECC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:11:43.957485] [INFO ] [MAIN ] ECAT: Wiping existing primary data before full refresh. [2025-09-16T18:11:43.977195] [INFO ] [LOCAL ] ECAT: Starting deletion of primary data. [2025-09-16T18:11:43.993233] [INFO ] [MAIN ] ECC: Wiping existing primary data before full refresh. [2025-09-16T18:11:44.014104] [INFO ] [LOCAL ] ECC: Starting deletion of primary data. [2025-09-16T18:11:44.023349] [WARN ] [MAIN ] EBS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:11:44.055227] [DEBUG] [LOCAL ] ECAT: Deleted primary data file. [2025-09-16T18:11:44.054188] [INFO ] [MAIN ] EBS: Wiping existing primary data before full refresh. [2025-09-16T18:11:44.055435] [INFO ] [LOCAL ] EBS: Starting deletion of primary data. [2025-09-16T18:11:44.047270] [DEBUG] [LOCAL ] ECC: Deleted primary data file. [2025-09-16T18:11:44.055382] [INFO ] [MAIN ] ECAT: Starting full primary refresh. [2025-09-16T18:11:44.074231] [INFO ] [MAIN ] ECC: Starting full primary refresh. [2025-09-16T18:11:44.085692] [DEBUG] [LOCAL ] EBS: Deleted primary data file. [2025-09-16T18:11:44.085631] [WARN ] [MAIN ] EC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:11:44.125336] [INFO ] [MAIN ] EBS: Starting full primary refresh. [2025-09-16T18:11:44.154076] [INFO ] [MAIN ] EC: Wiping existing primary data before full refresh. [2025-09-16T18:11:44.162901] [INFO ] [LOCAL ] EC: Starting deletion of primary data. [2025-09-16T18:11:44.256437] [DEBUG] [LOCAL ] EC: Deleted primary data file. [2025-09-16T18:11:44.256554] [INFO ] [MAIN ] EC: Starting full primary refresh. [2025-09-16T18:11:45.323842] [INFO ] [REMOTE ] EBS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:11:45.338121] [INFO ] [REMOTE ] ECAT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:11:45.343605] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-02. [2025-09-16T18:11:45.372033] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-02. [2025-09-16T18:11:45.446953] [INFO ] [REMOTE ] ECC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:11:45.477250] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-02. [2025-09-16T18:11:45.508501] [INFO ] [REMOTE ] EC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:11:45.509124] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-02. [2025-09-16T18:11:46.907587] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T18:11:46.916296] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-03. [2025-09-16T18:11:47.036500] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T18:11:47.043287] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-03. [2025-09-16T18:11:47.164149] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T18:11:47.171622] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-03. [2025-09-16T18:11:47.289900] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T18:11:47.299415] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-03. [2025-09-16T18:11:48.112541] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T18:11:48.122787] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-04. [2025-09-16T18:11:48.408858] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T18:11:48.415759] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-04. [2025-09-16T18:11:48.799788] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T18:11:48.809261] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-04. [2025-09-16T18:11:48.929003] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T18:11:48.939496] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-04. [2025-09-16T18:11:49.796137] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T18:11:49.805850] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-05. [2025-09-16T18:11:49.829309] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-09-16T18:11:49.835385] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-05. [2025-09-16T18:11:50.158884] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T18:11:50.168625] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-05. [2025-09-16T18:11:50.440423] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T18:11:50.451276] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-05. [2025-09-16T18:11:50.985306] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T18:11:50.991422] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-06. [2025-09-16T18:11:51.151853] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T18:11:51.574912] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-06. [2025-09-16T18:11:51.759746] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T18:11:52.258850] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-09-16T18:11:52.374901] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-07. [2025-09-16T18:11:52.459094] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-05 (2.01 seconds). [2025-09-16T18:11:53.097282] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T18:11:53.174927] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-07. [2025-09-16T18:11:53.683330] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T18:11:53.974910] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-06. [2025-09-16T18:11:54.549584] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T18:11:54.774872] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-08. [2025-09-16T18:11:55.574868] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-08. [2025-09-16T18:11:55.576128] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-09-16T18:11:56.231010] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-09-16T18:11:56.374849] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-09. [2025-09-16T18:11:56.481947] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-09-16T18:11:57.174919] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-06. [2025-09-16T18:11:57.974920] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-09. [2025-09-16T18:11:58.054721] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T18:11:58.060103] [INFO ] [REMOTE ] EBS: Full fetch complete (51089 records). [2025-09-16T18:11:58.060243] [DEBUG] [VALIDATOR ] EBS: Validating primary data. [2025-09-16T18:11:58.435668] [DEBUG] [VALIDATOR ] EBS: All expected trading days are present. [2025-09-16T18:11:58.435746] [DEBUG] [VALIDATOR ] EBS: Primary data validation passed. [2025-09-16T18:11:58.436355] [INFO ] [MAIN ] EBS: Fetched primary data is valid. [2025-09-16T18:11:58.436419] [DEBUG] [LOCAL ] EBS: Saving primary data. [2025-09-16T18:11:58.437424] [DEBUG] [LOCAL ] EBS: Trimmed 51089 records to 35107 (keeping 150 days of data) [2025-09-16T18:11:58.644366] [DEBUG] [LOCAL ] EBS: Primary data saved (35107 records). [2025-09-16T18:11:58.644564] [INFO ] [MAIN ] EBS: Full history saved. [2025-09-16T18:11:58.646124] [INFO ] [MAIN ] Update Progress: 681/2369 symbols completed. EBS took 16.2s. Estimated remaining: 2h 58m 17s. Rate: 9.5 symbols/min [2025-09-16T18:11:58.776394] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-07. [2025-09-16T18:11:58.824713] [INFO ] [MAIN ] ECCC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:11:58.840064] [DEBUG] [LOCAL ] ECCC: No existing data file found in primary storage. [2025-09-16T18:11:58.850754] [WARN ] [MAIN ] ECCC: No existing data found, performing full refresh instead. [2025-09-16T18:11:58.851184] [INFO ] [MAIN ] ECCC: Wiping existing primary data before full refresh. [2025-09-16T18:11:58.851757] [INFO ] [LOCAL ] ECCC: Starting deletion of primary data. [2025-09-16T18:11:58.853154] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T18:11:58.863210] [DEBUG] [LOCAL ] ECCC: No primary data file found to delete. [2025-09-16T18:11:58.863613] [INFO ] [MAIN ] ECCC: Starting full primary refresh. [2025-09-16T18:11:59.214534] [INFO ] [REMOTE ] ECCC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:11:59.400819] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-09-16T18:11:59.405038] [INFO ] [REMOTE ] ECAT: Full fetch complete (32389 records). [2025-09-16T18:11:59.405115] [DEBUG] [VALIDATOR ] ECAT: Validating primary data. [2025-09-16T18:11:59.581799] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-02. [2025-09-16T18:11:59.773105] [DEBUG] [VALIDATOR ] ECAT: All expected trading days are present. [2025-09-16T18:11:59.773178] [DEBUG] [VALIDATOR ] ECAT: Primary data validation passed. [2025-09-16T18:11:59.773684] [INFO ] [MAIN ] ECAT: Fetched primary data is valid. [2025-09-16T18:11:59.773727] [DEBUG] [LOCAL ] ECAT: Saving primary data. [2025-09-16T18:11:59.774435] [DEBUG] [LOCAL ] ECAT: Trimmed 32389 records to 21933 (keeping 150 days of data) [2025-09-16T18:11:59.914252] [DEBUG] [LOCAL ] ECAT: Primary data saved (21933 records). [2025-09-16T18:11:59.914391] [INFO ] [MAIN ] ECAT: Full history saved. [2025-09-16T18:11:59.914500] [INFO ] [MAIN ] Update Progress: 682/2369 symbols completed. ECAT took 17.5s. Estimated remaining: 2h 57m 58s. Rate: 9.5 symbols/min [2025-09-16T18:12:00.085191] [INFO ] [MAIN ] ECCU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:00.089443] [DEBUG] [LOCAL ] ECCU: Loaded 1672 existing records from primary storage. [2025-09-16T18:12:00.089527] [INFO ] [MAIN ] ECCU: Loaded 1672 existing records. [2025-09-16T18:12:00.139407] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-09-16T18:12:00.268038] [WARN ] [MAIN ] ECCU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:12:00.268251] [INFO ] [MAIN ] ECCU: Wiping existing primary data before full refresh. [2025-09-16T18:12:00.268318] [INFO ] [LOCAL ] ECCU: Starting deletion of primary data. [2025-09-16T18:12:00.268546] [DEBUG] [LOCAL ] ECCU: Deleted primary data file. [2025-09-16T18:12:00.268610] [INFO ] [MAIN ] ECCU: Starting full primary refresh. [2025-09-16T18:12:00.379975] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-07. [2025-09-16T18:12:00.628845] [INFO ] [REMOTE ] ECCU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:00.657291] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T18:12:01.174941] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-02. [2025-09-16T18:12:01.833321] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-09-16T18:12:01.974879] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-03. [2025-09-16T18:12:02.227948] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T18:12:02.314683] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-09-16T18:12:02.774906] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-03. [2025-09-16T18:12:03.338007] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-09-16T18:12:03.574939] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-04. [2025-09-16T18:12:04.021777] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-09-16T18:12:04.374897] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-08. [2025-09-16T18:12:05.174896] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-05. [2025-09-16T18:12:05.694400] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-09-16T18:12:05.974881] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-06. [2025-09-16T18:12:06.042508] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:12:06.447161] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-09-16T18:12:06.774901] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-09. [2025-09-16T18:12:07.574906] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-08. [2025-09-16T18:12:08.374908] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-04. [2025-09-16T18:12:08.518909] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-09-16T18:12:08.524183] [INFO ] [REMOTE ] ECC: Full fetch complete (46703 records). [2025-09-16T18:12:08.524273] [DEBUG] [VALIDATOR ] ECC: Validating primary data. [2025-09-16T18:12:08.689038] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-09-16T18:12:08.914955] [DEBUG] [VALIDATOR ] ECC: All expected trading days are present. [2025-09-16T18:12:08.915050] [DEBUG] [VALIDATOR ] ECC: Primary data validation passed. [2025-09-16T18:12:08.915771] [INFO ] [MAIN ] ECC: Fetched primary data is valid. [2025-09-16T18:12:08.915821] [DEBUG] [LOCAL ] ECC: Saving primary data. [2025-09-16T18:12:08.917038] [DEBUG] [LOCAL ] ECC: Trimmed 46703 records to 31843 (keeping 150 days of data) [2025-09-16T18:12:09.111039] [DEBUG] [LOCAL ] ECC: Primary data saved (31843 records). [2025-09-16T18:12:09.111172] [INFO ] [MAIN ] ECC: Full history saved. [2025-09-16T18:12:09.111298] [INFO ] [MAIN ] Update Progress: 683/2369 symbols completed. ECC took 26.7s. Estimated remaining: 2h 57m 59s. Rate: 9.5 symbols/min [2025-09-16T18:12:09.180125] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-07. [2025-09-16T18:12:09.287071] [INFO ] [MAIN ] ECCW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:09.287268] [DEBUG] [LOCAL ] ECCW: No existing data file found in primary storage. [2025-09-16T18:12:09.287310] [WARN ] [MAIN ] ECCW: No existing data found, performing full refresh instead. [2025-09-16T18:12:09.287609] [INFO ] [MAIN ] ECCW: Wiping existing primary data before full refresh. [2025-09-16T18:12:09.287680] [INFO ] [LOCAL ] ECCW: Starting deletion of primary data. [2025-09-16T18:12:09.287967] [DEBUG] [LOCAL ] ECCW: No primary data file found to delete. [2025-09-16T18:12:09.288040] [INFO ] [MAIN ] ECCW: Starting full primary refresh. [2025-09-16T18:12:09.681891] [INFO ] [REMOTE ] ECCW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:09.685158] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-09-16T18:12:09.686473] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-09-16T18:12:09.974916] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-09. [2025-09-16T18:12:10.774915] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-05. [2025-09-16T18:12:11.574940] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-08. [2025-09-16T18:12:11.678986] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-09-16T18:12:11.994159] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-09-16T18:12:12.098703] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-09 (2.12 seconds). [2025-09-16T18:12:12.105016] [INFO ] [REMOTE ] EC: Full fetch complete (53962 records). [2025-09-16T18:12:12.105171] [DEBUG] [VALIDATOR ] EC: Validating primary data. [2025-09-16T18:12:12.380021] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-09. [2025-09-16T18:12:12.497938] [DEBUG] [VALIDATOR ] EC: All expected trading days are present. [2025-09-16T18:12:12.498072] [DEBUG] [VALIDATOR ] EC: Primary data validation passed. [2025-09-16T18:12:12.499318] [INFO ] [MAIN ] EC: Fetched primary data is valid. [2025-09-16T18:12:12.499423] [DEBUG] [LOCAL ] EC: Saving primary data. [2025-09-16T18:12:12.501216] [DEBUG] [LOCAL ] EC: Trimmed 53962 records to 36426 (keeping 150 days of data) [2025-09-16T18:12:12.763814] [DEBUG] [LOCAL ] EC: Primary data saved (36426 records). [2025-09-16T18:12:12.763996] [INFO ] [MAIN ] EC: Full history saved. [2025-09-16T18:12:12.764102] [INFO ] [MAIN ] Update Progress: 684/2369 symbols completed. EC took 30.3s. Estimated remaining: 2h 57m 46s. Rate: 9.5 symbols/min [2025-09-16T18:12:12.935524] [INFO ] [MAIN ] ECCX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:12.935704] [DEBUG] [LOCAL ] ECCX: No existing data file found in primary storage. [2025-09-16T18:12:12.935743] [WARN ] [MAIN ] ECCX: No existing data found, performing full refresh instead. [2025-09-16T18:12:12.935915] [INFO ] [MAIN ] ECCX: Wiping existing primary data before full refresh. [2025-09-16T18:12:12.935969] [INFO ] [LOCAL ] ECCX: Starting deletion of primary data. [2025-09-16T18:12:12.936015] [DEBUG] [LOCAL ] ECCX: No primary data file found to delete. [2025-09-16T18:12:12.936050] [INFO ] [MAIN ] ECCX: Starting full primary refresh. [2025-09-16T18:12:13.115517] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-09-16T18:12:13.164087] [INFO ] [REMOTE ] ECCU: Full fetch complete (3176 records). [2025-09-16T18:12:13.193812] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-02. [2025-09-16T18:12:13.194142] [DEBUG] [VALIDATOR ] ECCU: Validating primary data. [2025-09-16T18:12:13.394814] [INFO ] [REMOTE ] ECCX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:13.674196] [DEBUG] [VALIDATOR ] ECCU: All expected trading days are present. [2025-09-16T18:12:13.676622] [DEBUG] [VALIDATOR ] ECCU: Primary data validation passed. [2025-09-16T18:12:13.676775] [INFO ] [MAIN ] ECCU: Fetched primary data is valid. [2025-09-16T18:12:13.676916] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-09-16T18:12:13.676823] [DEBUG] [LOCAL ] ECCU: Saving primary data. [2025-09-16T18:12:13.678404] [DEBUG] [LOCAL ] ECCU: Trimmed 3176 records to 1691 (keeping 150 days of data) [2025-09-16T18:12:13.690182] [DEBUG] [LOCAL ] ECCU: Primary data saved (1691 records). [2025-09-16T18:12:13.690320] [INFO ] [MAIN ] ECCU: Full history saved. [2025-09-16T18:12:13.690404] [INFO ] [MAIN ] Update Progress: 685/2369 symbols completed. ECCU took 13.8s. Estimated remaining: 2h 57m 26s. Rate: 9.5 symbols/min [2025-09-16T18:12:13.885484] [INFO ] [MAIN ] ECG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:13.908887] [DEBUG] [LOCAL ] ECG: Loaded 24130 existing records from primary storage. [2025-09-16T18:12:13.909088] [INFO ] [MAIN ] ECG: Loaded 24130 existing records. [2025-09-16T18:12:13.980040] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-06. [2025-09-16T18:12:14.087733] [WARN ] [MAIN ] ECG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:12:14.088047] [INFO ] [MAIN ] ECG: Wiping existing primary data before full refresh. [2025-09-16T18:12:14.088094] [INFO ] [LOCAL ] ECG: Starting deletion of primary data. [2025-09-16T18:12:14.088785] [DEBUG] [LOCAL ] ECG: Deleted primary data file. [2025-09-16T18:12:14.088836] [INFO ] [MAIN ] ECG: Starting full primary refresh. [2025-09-16T18:12:14.362798] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-09-16T18:12:14.441289] [INFO ] [REMOTE ] ECG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:14.774947] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-02. [2025-09-16T18:12:15.164066] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-09-16T18:12:15.574909] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-03. [2025-09-16T18:12:15.869143] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-09-16T18:12:16.374960] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-04. [2025-09-16T18:12:16.701292] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-09-16T18:12:17.174943] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-03. [2025-09-16T18:12:17.444882] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-03 (0.27 seconds). [2025-09-16T18:12:17.974915] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-07. [2025-09-16T18:12:18.235228] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-07 (0.26 seconds). [2025-09-16T18:12:18.774911] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-05. [2025-09-16T18:12:19.030460] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-05 (0.26 seconds). [2025-09-16T18:12:19.574911] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-08. [2025-09-16T18:12:20.028086] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-09-16T18:12:20.374910] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-04. [2025-09-16T18:12:20.819420] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-09-16T18:12:21.174895] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-05. [2025-09-16T18:12:21.458845] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-09-16T18:12:21.974906] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-06. [2025-09-16T18:12:22.282832] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-09-16T18:12:22.774903] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-07. [2025-09-16T18:12:23.072241] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-07 (0.30 seconds). [2025-09-16T18:12:23.574911] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-08. [2025-09-16T18:12:24.003420] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-09-16T18:12:24.374921] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-09. [2025-09-16T18:12:24.771854] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-09-16T18:12:24.772675] [INFO ] [REMOTE ] ECCX: Full fetch complete (1160 records). [2025-09-16T18:12:24.772743] [DEBUG] [VALIDATOR ] ECCX: Validating primary data. [2025-09-16T18:12:24.970609] [WARN ] [VALIDATOR ] ECCX: Validation failed - insufficient historical data. Have 146 unique days, need 150 days [2025-09-16T18:12:24.970795] [FATAL] [MAIN ] ECCX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:12:24.970846] [WARN ] [MAIN ] ECCX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:12:24.970895] [INFO ] [LOCAL ] ECCX: Starting deletion of primary data. [2025-09-16T18:12:24.970973] [DEBUG] [LOCAL ] ECCX: No primary data file found to delete. [2025-09-16T18:12:24.971013] [INFO ] [LOCAL ] ECCX: Starting deletion of derivative data. [2025-09-16T18:12:24.971067] [DEBUG] [LOCAL ] ECCX: No derivative data file found to delete. [2025-09-16T18:12:24.971128] [INFO ] [MAIN ] Update Progress: 686/2369 symbols completed. ECCX took 12.2s. Estimated remaining: 2h 57m 32s. Rate: 9.5 symbols/min [2025-09-16T18:12:25.144004] [INFO ] [MAIN ] ECL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:25.180449] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-02. [2025-09-16T18:12:25.183036] [DEBUG] [LOCAL ] ECL: Loaded 33964 existing records from primary storage. [2025-09-16T18:12:25.183247] [INFO ] [MAIN ] ECL: Loaded 33964 existing records. [2025-09-16T18:12:25.368532] [WARN ] [MAIN ] ECL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:12:25.368776] [INFO ] [MAIN ] ECL: Wiping existing primary data before full refresh. [2025-09-16T18:12:25.368834] [INFO ] [LOCAL ] ECL: Starting deletion of primary data. [2025-09-16T18:12:25.369755] [DEBUG] [LOCAL ] ECL: Deleted primary data file. [2025-09-16T18:12:25.369811] [INFO ] [MAIN ] ECL: Starting full primary refresh. [2025-09-16T18:12:25.729390] [INFO ] [REMOTE ] ECL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:25.974889] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-02. [2025-09-16T18:12:26.514084] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-09-16T18:12:26.774900] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-06. [2025-09-16T18:12:27.451248] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T18:12:27.526081] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-09-16T18:12:27.574860] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-07. [2025-09-16T18:12:28.176933] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-09-16T18:12:28.374900] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-03. [2025-09-16T18:12:29.174894] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-09. [2025-09-16T18:12:29.974932] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-03. [2025-09-16T18:12:30.152088] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T18:12:30.272496] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-09-16T18:12:30.273221] [INFO ] [REMOTE ] ECCC: Full fetch complete (970 records). [2025-09-16T18:12:30.273292] [DEBUG] [VALIDATOR ] ECCC: Validating primary data. [2025-09-16T18:12:30.453690] [WARN ] [VALIDATOR ] ECCC: Validation failed - insufficient historical data. Have 146 unique days, need 150 days [2025-09-16T18:12:30.453909] [FATAL] [MAIN ] ECCC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:12:30.453964] [WARN ] [MAIN ] ECCC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:12:30.454008] [INFO ] [LOCAL ] ECCC: Starting deletion of primary data. [2025-09-16T18:12:30.454095] [DEBUG] [LOCAL ] ECCC: No primary data file found to delete. [2025-09-16T18:12:30.454132] [INFO ] [LOCAL ] ECCC: Starting deletion of derivative data. [2025-09-16T18:12:30.454173] [DEBUG] [LOCAL ] ECCC: No derivative data file found to delete. [2025-09-16T18:12:30.454229] [INFO ] [MAIN ] Update Progress: 687/2369 symbols completed. ECCC took 31.8s. Estimated remaining: 2h 57m 24s. Rate: 9.5 symbols/min [2025-09-16T18:12:30.626255] [INFO ] [MAIN ] ECO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:30.649573] [DEBUG] [LOCAL ] ECO: Loaded 13055 existing records from primary storage. [2025-09-16T18:12:30.649761] [INFO ] [MAIN ] ECO: Loaded 13055 existing records. [2025-09-16T18:12:30.780063] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-08. [2025-09-16T18:12:30.832839] [WARN ] [MAIN ] ECO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:12:30.833089] [INFO ] [MAIN ] ECO: Wiping existing primary data before full refresh. [2025-09-16T18:12:30.833139] [INFO ] [LOCAL ] ECO: Starting deletion of primary data. [2025-09-16T18:12:30.833556] [DEBUG] [LOCAL ] ECO: Deleted primary data file. [2025-09-16T18:12:30.833604] [INFO ] [MAIN ] ECO: Starting full primary refresh. [2025-09-16T18:12:31.207831] [INFO ] [REMOTE ] ECO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:31.211365] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-09-16T18:12:31.240736] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T18:12:31.574923] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-04. [2025-09-16T18:12:32.374954] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-02. [2025-09-16T18:12:32.666996] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T18:12:33.174932] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-04. [2025-09-16T18:12:33.238514] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T18:12:33.974917] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-03. [2025-09-16T18:12:34.703137] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-09-16T18:12:34.774870] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-05. [2025-09-16T18:12:35.436406] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T18:12:35.574895] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-04. [2025-09-16T18:12:36.248075] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T18:12:36.374903] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-06. [2025-09-16T18:12:36.483511] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-09-16T18:12:37.174935] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-05. [2025-09-16T18:12:37.627337] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T18:12:37.974905] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-07. [2025-09-16T18:12:38.492747] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T18:12:38.774906] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-05. [2025-09-16T18:12:39.401184] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-09-16T18:12:39.574906] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-09. [2025-09-16T18:12:39.787612] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T18:12:40.137002] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T18:12:40.137891] [INFO ] [REMOTE ] ECCW: Full fetch complete (1187 records). [2025-09-16T18:12:40.137967] [DEBUG] [VALIDATOR ] ECCW: Validating primary data. [2025-09-16T18:12:40.313443] [WARN ] [VALIDATOR ] ECCW: Validation failed - insufficient historical data. Have 144 unique days, need 150 days [2025-09-16T18:12:40.313659] [FATAL] [MAIN ] ECCW: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:12:40.313709] [WARN ] [MAIN ] ECCW: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:12:40.313759] [INFO ] [LOCAL ] ECCW: Starting deletion of primary data. [2025-09-16T18:12:40.313833] [DEBUG] [LOCAL ] ECCW: No primary data file found to delete. [2025-09-16T18:12:40.313910] [INFO ] [LOCAL ] ECCW: Starting deletion of derivative data. [2025-09-16T18:12:40.313958] [DEBUG] [LOCAL ] ECCW: No derivative data file found to delete. [2025-09-16T18:12:40.314016] [INFO ] [MAIN ] Update Progress: 688/2369 symbols completed. ECCW took 31.2s. Estimated remaining: 2h 57m 26s. Rate: 9.5 symbols/min [2025-09-16T18:12:40.379981] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-06. [2025-09-16T18:12:40.484598] [INFO ] [MAIN ] ECVT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:40.514617] [DEBUG] [LOCAL ] ECVT: Loaded 31664 existing records from primary storage. [2025-09-16T18:12:40.514777] [INFO ] [MAIN ] ECVT: Loaded 31664 existing records. [2025-09-16T18:12:40.688716] [WARN ] [MAIN ] ECVT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:12:40.688953] [INFO ] [MAIN ] ECVT: Wiping existing primary data before full refresh. [2025-09-16T18:12:40.688997] [INFO ] [LOCAL ] ECVT: Starting deletion of primary data. [2025-09-16T18:12:40.689726] [DEBUG] [LOCAL ] ECVT: Deleted primary data file. [2025-09-16T18:12:40.689778] [INFO ] [MAIN ] ECVT: Starting full primary refresh. [2025-09-16T18:12:41.038692] [INFO ] [REMOTE ] ECVT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:41.174864] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-08. [2025-09-16T18:12:41.494690] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-09-16T18:12:41.974930] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-06. [2025-09-16T18:12:42.774911] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-07. [2025-09-16T18:12:42.860765] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T18:12:43.112532] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:12:43.574920] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-09. [2025-09-16T18:12:43.934631] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-09-16T18:12:44.374920] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-08. [2025-09-16T18:12:44.864510] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T18:12:44.870250] [INFO ] [REMOTE ] ECL: Full fetch complete (50904 records). [2025-09-16T18:12:44.870379] [DEBUG] [VALIDATOR ] ECL: Validating primary data. [2025-09-16T18:12:45.179986] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-02. [2025-09-16T18:12:45.258921] [DEBUG] [VALIDATOR ] ECL: All expected trading days are present. [2025-09-16T18:12:45.258993] [DEBUG] [VALIDATOR ] ECL: Primary data validation passed. [2025-09-16T18:12:45.259662] [INFO ] [MAIN ] ECL: Fetched primary data is valid. [2025-09-16T18:12:45.259712] [DEBUG] [LOCAL ] ECL: Saving primary data. [2025-09-16T18:12:45.260773] [DEBUG] [LOCAL ] ECL: Trimmed 50904 records to 34316 (keeping 150 days of data) [2025-09-16T18:12:45.474789] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-09-16T18:12:45.487986] [DEBUG] [LOCAL ] ECL: Primary data saved (34316 records). [2025-09-16T18:12:45.488097] [INFO ] [MAIN ] ECL: Full history saved. [2025-09-16T18:12:45.488170] [INFO ] [MAIN ] Update Progress: 689/2369 symbols completed. ECL took 20.5s. Estimated remaining: 2h 57m 17s. Rate: 9.5 symbols/min [2025-09-16T18:12:45.660118] [INFO ] [MAIN ] ED: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:45.698948] [DEBUG] [LOCAL ] ED: Loaded 38957 existing records from primary storage. [2025-09-16T18:12:45.699082] [INFO ] [MAIN ] ED: Loaded 38957 existing records. [2025-09-16T18:12:45.875511] [WARN ] [MAIN ] ED: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:12:45.875782] [INFO ] [MAIN ] ED: Wiping existing primary data before full refresh. [2025-09-16T18:12:45.875828] [INFO ] [LOCAL ] ED: Starting deletion of primary data. [2025-09-16T18:12:45.876770] [DEBUG] [LOCAL ] ED: Deleted primary data file. [2025-09-16T18:12:45.876819] [INFO ] [MAIN ] ED: Starting full primary refresh. [2025-09-16T18:12:45.976546] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-07. [2025-09-16T18:12:46.227981] [INFO ] [REMOTE ] ED: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:46.654343] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T18:12:46.774853] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-03. [2025-09-16T18:12:46.926703] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-09-16T18:12:47.574968] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-09. [2025-09-16T18:12:48.110113] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-09-16T18:12:48.374877] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-04. [2025-09-16T18:12:48.438082] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T18:12:48.440636] [INFO ] [REMOTE ] ECO: Full fetch complete (20789 records). [2025-09-16T18:12:48.440742] [DEBUG] [VALIDATOR ] ECO: Validating primary data. [2025-09-16T18:12:48.805056] [DEBUG] [VALIDATOR ] ECO: All expected trading days are present. [2025-09-16T18:12:48.805135] [DEBUG] [VALIDATOR ] ECO: Primary data validation passed. [2025-09-16T18:12:48.805550] [INFO ] [MAIN ] ECO: Fetched primary data is valid. [2025-09-16T18:12:48.805591] [DEBUG] [LOCAL ] ECO: Saving primary data. [2025-09-16T18:12:48.806179] [DEBUG] [LOCAL ] ECO: Trimmed 20789 records to 13314 (keeping 150 days of data) [2025-09-16T18:12:48.886596] [DEBUG] [LOCAL ] ECO: Primary data saved (13314 records). [2025-09-16T18:12:48.886753] [INFO ] [MAIN ] ECO: Full history saved. [2025-09-16T18:12:48.886837] [INFO ] [MAIN ] Update Progress: 690/2369 symbols completed. ECO took 18.4s. Estimated remaining: 2h 57m 3s. Rate: 9.5 symbols/min [2025-09-16T18:12:49.057713] [INFO ] [MAIN ] EDD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:49.077990] [DEBUG] [LOCAL ] EDD: Loaded 19939 existing records from primary storage. [2025-09-16T18:12:49.078123] [INFO ] [MAIN ] EDD: Loaded 19939 existing records. [2025-09-16T18:12:49.177988] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-02. [2025-09-16T18:12:49.259516] [WARN ] [MAIN ] EDD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:12:49.259791] [INFO ] [MAIN ] EDD: Wiping existing primary data before full refresh. [2025-09-16T18:12:49.259844] [INFO ] [LOCAL ] EDD: Starting deletion of primary data. [2025-09-16T18:12:49.260318] [DEBUG] [LOCAL ] EDD: Deleted primary data file. [2025-09-16T18:12:49.260380] [INFO ] [MAIN ] EDD: Starting full primary refresh. [2025-09-16T18:12:49.626781] [INFO ] [REMOTE ] EDD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:49.836235] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T18:12:49.974926] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-02. [2025-09-16T18:12:50.652490] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T18:12:50.774915] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-05. [2025-09-16T18:12:50.870033] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-09-16T18:12:51.574908] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-08. [2025-09-16T18:12:52.203251] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-09-16T18:12:52.374930] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-03. [2025-09-16T18:12:53.026840] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T18:12:53.174872] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-09. [2025-09-16T18:12:53.531666] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T18:12:53.974908] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-06. [2025-09-16T18:12:54.136593] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T18:12:54.140382] [INFO ] [REMOTE ] ECG: Full fetch complete (38853 records). [2025-09-16T18:12:54.140485] [DEBUG] [VALIDATOR ] ECG: Validating primary data. [2025-09-16T18:12:54.516919] [DEBUG] [VALIDATOR ] ECG: All expected trading days are present. [2025-09-16T18:12:54.517073] [DEBUG] [VALIDATOR ] ECG: Primary data validation passed. [2025-09-16T18:12:54.517724] [INFO ] [MAIN ] ECG: Fetched primary data is valid. [2025-09-16T18:12:54.517776] [DEBUG] [LOCAL ] ECG: Saving primary data. [2025-09-16T18:12:54.518915] [DEBUG] [LOCAL ] ECG: Trimmed 38853 records to 24286 (keeping 150 days of data) [2025-09-16T18:12:54.663218] [DEBUG] [LOCAL ] ECG: Primary data saved (24286 records). [2025-09-16T18:12:54.663348] [INFO ] [MAIN ] ECG: Full history saved. [2025-09-16T18:12:54.663460] [INFO ] [MAIN ] Update Progress: 691/2369 symbols completed. ECG took 41.0s. Estimated remaining: 2h 56m 56s. Rate: 9.5 symbols/min [2025-09-16T18:12:54.785070] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-04. [2025-09-16T18:12:54.839045] [INFO ] [MAIN ] EDF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:12:54.850318] [DEBUG] [LOCAL ] EDF: Loaded 9497 existing records from primary storage. [2025-09-16T18:12:54.850554] [INFO ] [MAIN ] EDF: Loaded 9497 existing records. [2025-09-16T18:12:55.026108] [WARN ] [MAIN ] EDF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:12:55.026356] [INFO ] [MAIN ] EDF: Wiping existing primary data before full refresh. [2025-09-16T18:12:55.026401] [INFO ] [LOCAL ] EDF: Starting deletion of primary data. [2025-09-16T18:12:55.026767] [DEBUG] [LOCAL ] EDF: Deleted primary data file. [2025-09-16T18:12:55.026820] [INFO ] [MAIN ] EDF: Starting full primary refresh. [2025-09-16T18:12:55.391847] [INFO ] [REMOTE ] EDF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:12:55.415524] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T18:12:55.574853] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-07. [2025-09-16T18:12:56.045230] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-09-16T18:12:56.374948] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-02. [2025-09-16T18:12:57.053562] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T18:12:57.174868] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-08. [2025-09-16T18:12:57.492128] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-09-16T18:12:57.974953] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-03. [2025-09-16T18:12:58.411070] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T18:12:58.698606] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-09-16T18:12:58.774924] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-05. [2025-09-16T18:12:59.574902] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-03. [2025-09-16T18:12:59.601570] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T18:13:00.374902] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-06. [2025-09-16T18:13:01.174927] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-04. [2025-09-16T18:13:01.646459] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-09-16T18:13:01.788798] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T18:13:01.974903] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-09. [2025-09-16T18:13:02.197030] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-09-16T18:13:02.774903] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-05. [2025-09-16T18:13:03.574911] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-04. [2025-09-16T18:13:03.651894] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T18:13:03.657352] [INFO ] [REMOTE ] ECVT: Full fetch complete (46333 records). [2025-09-16T18:13:03.657473] [DEBUG] [VALIDATOR ] ECVT: Validating primary data. [2025-09-16T18:13:03.859848] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-09-16T18:13:04.077223] [DEBUG] [VALIDATOR ] ECVT: All expected trading days are present. [2025-09-16T18:13:04.077317] [DEBUG] [VALIDATOR ] ECVT: Primary data validation passed. [2025-09-16T18:13:04.078023] [INFO ] [MAIN ] ECVT: Fetched primary data is valid. [2025-09-16T18:13:04.078075] [DEBUG] [LOCAL ] ECVT: Saving primary data. [2025-09-16T18:13:04.079580] [DEBUG] [LOCAL ] ECVT: Trimmed 46333 records to 32031 (keeping 150 days of data) [2025-09-16T18:13:04.269673] [DEBUG] [LOCAL ] ECVT: Primary data saved (32031 records). [2025-09-16T18:13:04.269839] [INFO ] [MAIN ] ECVT: Full history saved. [2025-09-16T18:13:04.269953] [INFO ] [MAIN ] Update Progress: 692/2369 symbols completed. ECVT took 24.0s. Estimated remaining: 2h 56m 57s. Rate: 9.5 symbols/min [2025-09-16T18:13:04.379084] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-06. [2025-09-16T18:13:04.455866] [INFO ] [MAIN ] EDN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:13:04.470210] [DEBUG] [LOCAL ] EDN: Loaded 12886 existing records from primary storage. [2025-09-16T18:13:04.470343] [INFO ] [MAIN ] EDN: Loaded 12886 existing records. [2025-09-16T18:13:04.648027] [WARN ] [MAIN ] EDN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:13:04.648304] [INFO ] [MAIN ] EDN: Wiping existing primary data before full refresh. [2025-09-16T18:13:04.648349] [INFO ] [LOCAL ] EDN: Starting deletion of primary data. [2025-09-16T18:13:04.648890] [DEBUG] [LOCAL ] EDN: Deleted primary data file. [2025-09-16T18:13:04.648940] [INFO ] [MAIN ] EDN: Starting full primary refresh. [2025-09-16T18:13:05.003166] [INFO ] [REMOTE ] EDN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:13:05.174921] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-02. [2025-09-16T18:13:05.290883] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-09-16T18:13:05.292675] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T18:13:05.974918] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-07. [2025-09-16T18:13:06.399360] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T18:13:06.774956] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-07. [2025-09-16T18:13:07.327957] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T18:13:07.574871] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-08. [2025-09-16T18:13:07.894745] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T18:13:08.374915] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-03. [2025-09-16T18:13:08.766357] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-09-16T18:13:09.174945] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-05. [2025-09-16T18:13:09.382730] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-09-16T18:13:09.974927] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-04. [2025-09-16T18:13:10.563003] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T18:13:10.774907] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-06. [2025-09-16T18:13:11.036010] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-09-16T18:13:11.574940] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-05. [2025-09-16T18:13:12.380016] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-09. [2025-09-16T18:13:12.393111] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T18:13:12.754241] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-09-16T18:13:13.174917] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-07. [2025-09-16T18:13:13.642299] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T18:13:13.646180] [INFO ] [REMOTE ] EDD: Full fetch complete (29787 records). [2025-09-16T18:13:13.646262] [DEBUG] [VALIDATOR ] EDD: Validating primary data. [2025-09-16T18:13:13.980551] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-08. [2025-09-16T18:13:14.024540] [DEBUG] [VALIDATOR ] EDD: All expected trading days are present. [2025-09-16T18:13:14.024690] [DEBUG] [VALIDATOR ] EDD: Primary data validation passed. [2025-09-16T18:13:14.025190] [INFO ] [MAIN ] EDD: Fetched primary data is valid. [2025-09-16T18:13:14.025244] [DEBUG] [LOCAL ] EDD: Saving primary data. [2025-09-16T18:13:14.026205] [DEBUG] [LOCAL ] EDD: Trimmed 29787 records to 20209 (keeping 150 days of data) [2025-09-16T18:13:14.141465] [DEBUG] [LOCAL ] EDD: Primary data saved (20209 records). [2025-09-16T18:13:14.141592] [INFO ] [MAIN ] EDD: Full history saved. [2025-09-16T18:13:14.141692] [INFO ] [MAIN ] Update Progress: 693/2369 symbols completed. EDD took 25.3s. Estimated remaining: 2h 56m 60s. Rate: 9.5 symbols/min [2025-09-16T18:13:14.326055] [INFO ] [MAIN ] EDU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:13:14.361836] [DEBUG] [LOCAL ] EDU: Loaded 32550 existing records from primary storage. [2025-09-16T18:13:14.362018] [INFO ] [MAIN ] EDU: Loaded 32550 existing records. [2025-09-16T18:13:14.540138] [WARN ] [MAIN ] EDU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:13:14.540424] [INFO ] [MAIN ] EDU: Wiping existing primary data before full refresh. [2025-09-16T18:13:14.540473] [INFO ] [LOCAL ] EDU: Starting deletion of primary data. [2025-09-16T18:13:14.541255] [DEBUG] [LOCAL ] EDU: Deleted primary data file. [2025-09-16T18:13:14.541311] [INFO ] [MAIN ] EDU: Starting full primary refresh. [2025-09-16T18:13:14.783414] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-06. [2025-09-16T18:13:14.899930] [INFO ] [REMOTE ] EDU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:13:14.907871] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T18:13:15.101590] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-09-16T18:13:15.574894] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-09. [2025-09-16T18:13:15.793829] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T18:13:16.374897] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-02. [2025-09-16T18:13:16.407304] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-09-16T18:13:16.409201] [INFO ] [REMOTE ] EDF: Full fetch complete (15175 records). [2025-09-16T18:13:16.409288] [DEBUG] [VALIDATOR ] EDF: Validating primary data. [2025-09-16T18:13:16.770019] [DEBUG] [VALIDATOR ] EDF: All expected trading days are present. [2025-09-16T18:13:16.770097] [DEBUG] [VALIDATOR ] EDF: Primary data validation passed. [2025-09-16T18:13:16.770442] [INFO ] [MAIN ] EDF: Fetched primary data is valid. [2025-09-16T18:13:16.770491] [DEBUG] [LOCAL ] EDF: Saving primary data. [2025-09-16T18:13:16.771016] [DEBUG] [LOCAL ] EDF: Trimmed 15175 records to 9544 (keeping 150 days of data) [2025-09-16T18:13:16.827538] [DEBUG] [LOCAL ] EDF: Primary data saved (9544 records). [2025-09-16T18:13:16.827696] [INFO ] [MAIN ] EDF: Full history saved. [2025-09-16T18:13:16.827785] [INFO ] [MAIN ] Update Progress: 694/2369 symbols completed. EDF took 22.2s. Estimated remaining: 2h 56m 44s. Rate: 9.5 symbols/min [2025-09-16T18:13:16.996339] [INFO ] [MAIN ] EE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:13:17.018690] [DEBUG] [LOCAL ] EE: Loaded 23948 existing records from primary storage. [2025-09-16T18:13:17.018830] [INFO ] [MAIN ] EE: Loaded 23948 existing records. [2025-09-16T18:13:17.177860] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-07. [2025-09-16T18:13:17.196618] [WARN ] [MAIN ] EE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:13:17.197760] [INFO ] [MAIN ] EE: Wiping existing primary data before full refresh. [2025-09-16T18:13:17.197818] [INFO ] [LOCAL ] EE: Starting deletion of primary data. [2025-09-16T18:13:17.198342] [DEBUG] [LOCAL ] EE: Deleted primary data file. [2025-09-16T18:13:17.198430] [INFO ] [MAIN ] EE: Starting full primary refresh. [2025-09-16T18:13:17.573540] [INFO ] [REMOTE ] EE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:13:17.974909] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-08. [2025-09-16T18:13:18.117312] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T18:13:18.358435] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-09-16T18:13:18.774909] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-03. [2025-09-16T18:13:19.574899] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-08. [2025-09-16T18:13:19.763257] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T18:13:20.374882] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-09. [2025-09-16T18:13:20.643999] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T18:13:20.677931] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-09-16T18:13:21.174929] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-02. [2025-09-16T18:13:21.975270] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-09. [2025-09-16T18:13:21.979540] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T18:13:21.986516] [INFO ] [REMOTE ] ED: Full fetch complete (57427 records). [2025-09-16T18:13:21.986630] [DEBUG] [VALIDATOR ] ED: Validating primary data. [2025-09-16T18:13:22.371852] [DEBUG] [VALIDATOR ] ED: All expected trading days are present. [2025-09-16T18:13:22.371947] [DEBUG] [VALIDATOR ] ED: Primary data validation passed. [2025-09-16T18:13:22.372842] [INFO ] [MAIN ] ED: Fetched primary data is valid. [2025-09-16T18:13:22.372891] [DEBUG] [LOCAL ] ED: Saving primary data. [2025-09-16T18:13:22.374276] [DEBUG] [LOCAL ] ED: Trimmed 57427 records to 39343 (keeping 150 days of data) [2025-09-16T18:13:22.617581] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T18:13:22.647324] [DEBUG] [LOCAL ] ED: Primary data saved (39343 records). [2025-09-16T18:13:22.647448] [INFO ] [MAIN ] ED: Full history saved. [2025-09-16T18:13:22.647545] [INFO ] [MAIN ] Update Progress: 695/2369 symbols completed. ED took 37.2s. Estimated remaining: 2h 56m 37s. Rate: 9.5 symbols/min [2025-09-16T18:13:22.779984] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-04. [2025-09-16T18:13:22.832952] [INFO ] [MAIN ] EEA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:13:22.836897] [DEBUG] [LOCAL ] EEA: Loaded 1980 existing records from primary storage. [2025-09-16T18:13:22.837053] [INFO ] [MAIN ] EEA: Loaded 1980 existing records. [2025-09-16T18:13:23.026503] [WARN ] [MAIN ] EEA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:13:23.026778] [INFO ] [MAIN ] EEA: Wiping existing primary data before full refresh. [2025-09-16T18:13:23.026827] [INFO ] [LOCAL ] EEA: Starting deletion of primary data. [2025-09-16T18:13:23.027077] [DEBUG] [LOCAL ] EEA: Deleted primary data file. [2025-09-16T18:13:23.027123] [INFO ] [MAIN ] EEA: Starting full primary refresh. [2025-09-16T18:13:23.401483] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-09-16T18:13:23.406683] [INFO ] [REMOTE ] EEA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:13:23.410713] [INFO ] [REMOTE ] EDN: Full fetch complete (19673 records). [2025-09-16T18:13:23.410807] [DEBUG] [VALIDATOR ] EDN: Validating primary data. [2025-09-16T18:13:23.584509] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-02. [2025-09-16T18:13:23.793595] [DEBUG] [VALIDATOR ] EDN: All expected trading days are present. [2025-09-16T18:13:23.793700] [DEBUG] [VALIDATOR ] EDN: Primary data validation passed. [2025-09-16T18:13:23.794182] [INFO ] [MAIN ] EDN: Fetched primary data is valid. [2025-09-16T18:13:23.794228] [DEBUG] [LOCAL ] EDN: Saving primary data. [2025-09-16T18:13:23.794942] [DEBUG] [LOCAL ] EDN: Trimmed 19673 records to 13047 (keeping 150 days of data) [2025-09-16T18:13:23.871137] [DEBUG] [LOCAL ] EDN: Primary data saved (13047 records). [2025-09-16T18:13:23.871294] [INFO ] [MAIN ] EDN: Full history saved. [2025-09-16T18:13:23.871387] [INFO ] [MAIN ] Update Progress: 696/2369 symbols completed. EDN took 19.6s. Estimated remaining: 2h 56m 18s. Rate: 9.5 symbols/min [2025-09-16T18:13:24.039946] [INFO ] [MAIN ] EEX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:13:24.049158] [DEBUG] [LOCAL ] EEX: Loaded 7231 existing records from primary storage. [2025-09-16T18:13:24.049277] [INFO ] [MAIN ] EEX: Loaded 7231 existing records. [2025-09-16T18:13:24.238460] [WARN ] [MAIN ] EEX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:13:24.241834] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-09-16T18:13:24.242147] [INFO ] [MAIN ] EEX: Wiping existing primary data before full refresh. [2025-09-16T18:13:24.243535] [INFO ] [LOCAL ] EEX: Starting deletion of primary data. [2025-09-16T18:13:24.248010] [DEBUG] [LOCAL ] EEX: Deleted primary data file. [2025-09-16T18:13:24.248226] [INFO ] [MAIN ] EEX: Starting full primary refresh. [2025-09-16T18:13:24.380073] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-03. [2025-09-16T18:13:24.705024] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-09-16T18:13:24.866900] [INFO ] [REMOTE ] EEX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:13:25.006067] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-09-16T18:13:25.174934] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-05. [2025-09-16T18:13:25.974904] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-02. [2025-09-16T18:13:26.718344] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T18:13:26.770663] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-09-16T18:13:26.774831] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-03. [2025-09-16T18:13:27.502357] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-09-16T18:13:27.574857] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-04. [2025-09-16T18:13:28.374940] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-03. [2025-09-16T18:13:28.598140] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-09-16T18:13:29.174903] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-05. [2025-09-16T18:13:29.494281] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-09-16T18:13:29.974859] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-04. [2025-09-16T18:13:30.030017] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T18:13:30.774897] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-06. [2025-09-16T18:13:31.508463] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-09-16T18:13:31.574909] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-05. [2025-09-16T18:13:31.669746] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T18:13:32.374906] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-07. [2025-09-16T18:13:33.013761] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-09-16T18:13:33.126162] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-09-16T18:13:33.174862] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-06. [2025-09-16T18:13:33.974928] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-04. [2025-09-16T18:13:34.478116] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-09-16T18:13:34.498954] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T18:13:34.774940] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-08. [2025-09-16T18:13:35.191358] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-09-16T18:13:35.574905] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-09. [2025-09-16T18:13:36.374924] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-05. [2025-09-16T18:13:36.424003] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T18:13:36.425550] [INFO ] [REMOTE ] EEX: Full fetch complete (13652 records). [2025-09-16T18:13:36.425631] [DEBUG] [VALIDATOR ] EEX: Validating primary data. [2025-09-16T18:13:36.795633] [DEBUG] [VALIDATOR ] EEX: All expected trading days are present. [2025-09-16T18:13:36.795741] [DEBUG] [VALIDATOR ] EEX: Primary data validation passed. [2025-09-16T18:13:36.796062] [INFO ] [MAIN ] EEX: Fetched primary data is valid. [2025-09-16T18:13:36.796106] [DEBUG] [LOCAL ] EEX: Saving primary data. [2025-09-16T18:13:36.796696] [DEBUG] [LOCAL ] EEX: Trimmed 13652 records to 7285 (keeping 150 days of data) [2025-09-16T18:13:36.838746] [DEBUG] [LOCAL ] EEX: Primary data saved (7285 records). [2025-09-16T18:13:36.838878] [INFO ] [MAIN ] EEX: Full history saved. [2025-09-16T18:13:36.838962] [INFO ] [MAIN ] Update Progress: 697/2369 symbols completed. EEX took 13.0s. Estimated remaining: 2h 56m 28s. Rate: 9.5 symbols/min [2025-09-16T18:13:36.947268] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-09-16T18:13:37.018875] [INFO ] [MAIN ] EFC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:13:37.054975] [DEBUG] [LOCAL ] EFC: Loaded 33691 existing records from primary storage. [2025-09-16T18:13:37.055150] [INFO ] [MAIN ] EFC: Loaded 33691 existing records. [2025-09-16T18:13:37.179989] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-06. [2025-09-16T18:13:37.237143] [WARN ] [MAIN ] EFC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:13:37.237383] [INFO ] [MAIN ] EFC: Wiping existing primary data before full refresh. [2025-09-16T18:13:37.237431] [INFO ] [LOCAL ] EFC: Starting deletion of primary data. [2025-09-16T18:13:37.238215] [DEBUG] [LOCAL ] EFC: Deleted primary data file. [2025-09-16T18:13:37.238266] [INFO ] [MAIN ] EFC: Starting full primary refresh. [2025-09-16T18:13:37.599622] [INFO ] [REMOTE ] EFC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:13:37.974904] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-02. [2025-09-16T18:13:38.774869] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-07. [2025-09-16T18:13:38.810480] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T18:13:39.574971] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-06. [2025-09-16T18:13:39.617405] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:13:40.095125] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-09-16T18:13:40.289106] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T18:13:40.374950] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-07. [2025-09-16T18:13:41.174905] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-07. [2025-09-16T18:13:41.658852] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-09-16T18:13:41.974900] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-08. [2025-09-16T18:13:42.103338] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T18:13:42.429478] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-09-16T18:13:42.774916] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-03. [2025-09-16T18:13:43.574904] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-08. [2025-09-16T18:13:44.374942] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-08. [2025-09-16T18:13:44.448118] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T18:13:45.159484] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T18:13:45.174841] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-09. [2025-09-16T18:13:45.672956] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-09-16T18:13:45.974906] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-09. [2025-09-16T18:13:46.613228] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T18:13:46.618694] [INFO ] [REMOTE ] EDU: Full fetch complete (49863 records). [2025-09-16T18:13:46.618787] [DEBUG] [VALIDATOR ] EDU: Validating primary data. [2025-09-16T18:13:46.796580] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-04. [2025-09-16T18:13:47.003779] [DEBUG] [VALIDATOR ] EDU: All expected trading days are present. [2025-09-16T18:13:47.003855] [DEBUG] [VALIDATOR ] EDU: Primary data validation passed. [2025-09-16T18:13:47.004509] [INFO ] [MAIN ] EDU: Fetched primary data is valid. [2025-09-16T18:13:47.004551] [DEBUG] [LOCAL ] EDU: Saving primary data. [2025-09-16T18:13:47.005743] [DEBUG] [LOCAL ] EDU: Trimmed 49863 records to 32909 (keeping 150 days of data) [2025-09-16T18:13:47.196333] [DEBUG] [LOCAL ] EDU: Primary data saved (32909 records). [2025-09-16T18:13:47.196474] [INFO ] [MAIN ] EDU: Full history saved. [2025-09-16T18:13:47.197188] [INFO ] [MAIN ] Update Progress: 698/2369 symbols completed. EDU took 33.1s. Estimated remaining: 2h 56m 31s. Rate: 9.5 symbols/min [2025-09-16T18:13:47.367532] [INFO ] [MAIN ] EFR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:13:47.375879] [DEBUG] [LOCAL ] EFR: No existing data file found in primary storage. [2025-09-16T18:13:47.376719] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T18:13:47.376383] [WARN ] [MAIN ] EFR: No existing data found, performing full refresh instead. [2025-09-16T18:13:47.380085] [INFO ] [MAIN ] EFR: Wiping existing primary data before full refresh. [2025-09-16T18:13:47.380416] [INFO ] [LOCAL ] EFR: Starting deletion of primary data. [2025-09-16T18:13:47.380540] [DEBUG] [LOCAL ] EFR: No primary data file found to delete. [2025-09-16T18:13:47.380576] [INFO ] [MAIN ] EFR: Starting full primary refresh. [2025-09-16T18:13:47.424507] [INFO ] [REMOTE ] EE: Full fetch complete (34474 records). [2025-09-16T18:13:47.456626] [DEBUG] [VALIDATOR ] EE: Validating primary data. [2025-09-16T18:13:47.578820] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-09. [2025-09-16T18:13:48.039456] [INFO ] [REMOTE ] EFR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:13:48.121975] [DEBUG] [VALIDATOR ] EE: All expected trading days are present. [2025-09-16T18:13:48.122079] [DEBUG] [VALIDATOR ] EE: Primary data validation passed. [2025-09-16T18:13:48.122603] [INFO ] [MAIN ] EE: Fetched primary data is valid. [2025-09-16T18:13:48.122660] [DEBUG] [LOCAL ] EE: Saving primary data. [2025-09-16T18:13:48.123518] [DEBUG] [LOCAL ] EE: Trimmed 34474 records to 24150 (keeping 150 days of data) [2025-09-16T18:13:48.289407] [DEBUG] [LOCAL ] EE: Primary data saved (24150 records). [2025-09-16T18:13:48.289540] [INFO ] [MAIN ] EE: Full history saved. [2025-09-16T18:13:48.289666] [INFO ] [MAIN ] Update Progress: 699/2369 symbols completed. EE took 31.5s. Estimated remaining: 2h 56m 12s. Rate: 9.5 symbols/min [2025-09-16T18:13:48.382949] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-02. [2025-09-16T18:13:48.372335] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T18:13:48.391731] [INFO ] [REMOTE ] EEA: Full fetch complete (3047 records). [2025-09-16T18:13:48.395910] [DEBUG] [VALIDATOR ] EEA: Validating primary data. [2025-09-16T18:13:48.576220] [INFO ] [MAIN ] EFT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:13:48.718560] [DEBUG] [LOCAL ] EFT: Loaded 9090 existing records from primary storage. [2025-09-16T18:13:48.718819] [INFO ] [MAIN ] EFT: Loaded 9090 existing records. [2025-09-16T18:13:49.015739] [DEBUG] [VALIDATOR ] EEA: All expected trading days are present. [2025-09-16T18:13:49.049860] [WARN ] [MAIN ] EFT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:13:49.050193] [DEBUG] [VALIDATOR ] EEA: Primary data validation passed. [2025-09-16T18:13:49.050511] [INFO ] [MAIN ] EEA: Fetched primary data is valid. [2025-09-16T18:13:49.050743] [DEBUG] [LOCAL ] EEA: Saving primary data. [2025-09-16T18:13:49.051101] [INFO ] [MAIN ] EFT: Wiping existing primary data before full refresh. [2025-09-16T18:13:49.051239] [INFO ] [LOCAL ] EFT: Starting deletion of primary data. [2025-09-16T18:13:49.051744] [DEBUG] [LOCAL ] EEA: Trimmed 3047 records to 1988 (keeping 150 days of data) [2025-09-16T18:13:49.052139] [DEBUG] [LOCAL ] EFT: Deleted primary data file. [2025-09-16T18:13:49.052271] [INFO ] [MAIN ] EFT: Starting full primary refresh. [2025-09-16T18:13:49.108441] [DEBUG] [LOCAL ] EEA: Primary data saved (1988 records). [2025-09-16T18:13:49.159119] [INFO ] [MAIN ] EEA: Full history saved. [2025-09-16T18:13:49.193635] [INFO ] [MAIN ] Update Progress: 700/2369 symbols completed. EEA took 26.5s. Estimated remaining: 2h 55m 53s. Rate: 9.5 symbols/min [2025-09-16T18:13:49.199814] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-04 (2.40 seconds). [2025-09-16T18:13:49.313166] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-05. [2025-09-16T18:13:49.579571] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-09-16T18:13:49.644009] [INFO ] [MAIN ] EFX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:13:49.723040] [INFO ] [REMOTE ] EFT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:13:49.752778] [DEBUG] [LOCAL ] EFX: Loaded 32620 existing records from primary storage. [2025-09-16T18:13:49.752935] [INFO ] [MAIN ] EFX: Loaded 32620 existing records. [2025-09-16T18:13:49.940315] [WARN ] [MAIN ] EFX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:13:49.940594] [INFO ] [MAIN ] EFX: Wiping existing primary data before full refresh. [2025-09-16T18:13:49.940663] [INFO ] [LOCAL ] EFX: Starting deletion of primary data. [2025-09-16T18:13:49.941540] [DEBUG] [LOCAL ] EFX: Deleted primary data file. [2025-09-16T18:13:49.941609] [INFO ] [MAIN ] EFX: Starting full primary refresh. [2025-09-16T18:13:49.984596] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-03. [2025-09-16T18:13:50.305337] [INFO ] [REMOTE ] EFX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:13:50.774933] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-02. [2025-09-16T18:13:50.880743] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-09-16T18:13:51.131813] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-09-16T18:13:51.574903] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-04. [2025-09-16T18:13:52.230425] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-09-16T18:13:52.374951] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-02. [2025-09-16T18:13:52.604377] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T18:13:52.905076] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-09-16T18:13:53.174902] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-03. [2025-09-16T18:13:53.932265] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-09-16T18:13:53.974931] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-05. [2025-09-16T18:13:54.774899] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-03. [2025-09-16T18:13:54.810051] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T18:13:55.574936] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-04. [2025-09-16T18:13:56.374874] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-06. [2025-09-16T18:13:56.439089] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-09-16T18:13:56.517141] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-09-16T18:13:57.009810] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-09-16T18:13:57.174873] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-05. [2025-09-16T18:13:57.974881] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-07. [2025-09-16T18:13:58.161152] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T18:13:58.774905] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-04. [2025-09-16T18:13:58.940211] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T18:13:59.574911] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-06. [2025-09-16T18:14:00.374939] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-08. [2025-09-16T18:14:01.037531] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-04 (2.26 seconds). [2025-09-16T18:14:01.174941] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-05. [2025-09-16T18:14:01.555033] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-09-16T18:14:01.602961] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-09-16T18:14:01.974899] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-09. [2025-09-16T18:14:02.774867] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-07. [2025-09-16T18:14:02.968133] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T18:14:02.970490] [INFO ] [REMOTE ] EFR: Full fetch complete (14752 records). [2025-09-16T18:14:02.970569] [DEBUG] [VALIDATOR ] EFR: Validating primary data. [2025-09-16T18:14:03.343776] [DEBUG] [VALIDATOR ] EFR: All expected trading days are present. [2025-09-16T18:14:03.343993] [DEBUG] [VALIDATOR ] EFR: Primary data validation passed. [2025-09-16T18:14:03.344400] [INFO ] [MAIN ] EFR: Fetched primary data is valid. [2025-09-16T18:14:03.344466] [DEBUG] [LOCAL ] EFR: Saving primary data. [2025-09-16T18:14:03.345299] [DEBUG] [LOCAL ] EFR: Trimmed 14752 records to 10364 (keeping 150 days of data) [2025-09-16T18:14:03.422689] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-05 (2.25 seconds). [2025-09-16T18:14:03.448087] [DEBUG] [LOCAL ] EFR: Primary data saved (10364 records). [2025-09-16T18:14:03.448206] [INFO ] [MAIN ] EFR: Full history saved. [2025-09-16T18:14:03.448293] [INFO ] [MAIN ] Update Progress: 701/2369 symbols completed. EFR took 16.3s. Estimated remaining: 2h 56m 6s. Rate: 9.5 symbols/min [2025-09-16T18:14:03.590001] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-06. [2025-09-16T18:14:03.620549] [INFO ] [MAIN ] EFXT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:14:03.645411] [DEBUG] [LOCAL ] EFXT: Loaded 20982 existing records from primary storage. [2025-09-16T18:14:03.645543] [INFO ] [MAIN ] EFXT: Loaded 20982 existing records. [2025-09-16T18:14:03.822382] [WARN ] [MAIN ] EFXT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:14:03.822636] [INFO ] [MAIN ] EFXT: Wiping existing primary data before full refresh. [2025-09-16T18:14:03.822704] [INFO ] [LOCAL ] EFXT: Starting deletion of primary data. [2025-09-16T18:14:03.823287] [DEBUG] [LOCAL ] EFXT: Deleted primary data file. [2025-09-16T18:14:03.823344] [INFO ] [MAIN ] EFXT: Starting full primary refresh. [2025-09-16T18:14:04.179411] [INFO ] [REMOTE ] EFXT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:14:04.374957] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-06. [2025-09-16T18:14:04.540156] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-09-16T18:14:05.147304] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T18:14:05.174883] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-07. [2025-09-16T18:14:05.337284] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-09-16T18:14:05.974923] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-02. [2025-09-16T18:14:06.774906] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-08. [2025-09-16T18:14:06.909102] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T18:14:07.316925] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-09-16T18:14:07.574915] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-07. [2025-09-16T18:14:08.283790] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T18:14:08.374850] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-09. [2025-09-16T18:14:08.472951] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-09-16T18:14:09.174916] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-08. [2025-09-16T18:14:09.843510] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-09-16T18:14:09.974889] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-09. [2025-09-16T18:14:10.256635] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-09-16T18:14:10.263517] [INFO ] [REMOTE ] EFC: Full fetch complete (50177 records). [2025-09-16T18:14:10.263722] [DEBUG] [VALIDATOR ] EFC: Validating primary data. [2025-09-16T18:14:10.655009] [DEBUG] [VALIDATOR ] EFC: All expected trading days are present. [2025-09-16T18:14:10.655107] [DEBUG] [VALIDATOR ] EFC: Primary data validation passed. [2025-09-16T18:14:10.655824] [INFO ] [MAIN ] EFC: Fetched primary data is valid. [2025-09-16T18:14:10.655876] [DEBUG] [LOCAL ] EFC: Saving primary data. [2025-09-16T18:14:10.657138] [DEBUG] [LOCAL ] EFC: Trimmed 50177 records to 34040 (keeping 150 days of data) [2025-09-16T18:14:10.775916] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-08. [2025-09-16T18:14:10.867618] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T18:14:10.870467] [INFO ] [REMOTE ] EFT: Full fetch complete (13155 records). [2025-09-16T18:14:10.871371] [DEBUG] [VALIDATOR ] EFT: Validating primary data. [2025-09-16T18:14:11.257988] [DEBUG] [VALIDATOR ] EFT: All expected trading days are present. [2025-09-16T18:14:11.258618] [DEBUG] [VALIDATOR ] EFT: Primary data validation passed. [2025-09-16T18:14:11.260128] [INFO ] [MAIN ] EFT: Fetched primary data is valid. [2025-09-16T18:14:11.260741] [DEBUG] [LOCAL ] EFT: Saving primary data. [2025-09-16T18:14:11.263612] [DEBUG] [LOCAL ] EFC: Primary data saved (34040 records). [2025-09-16T18:14:11.264115] [INFO ] [MAIN ] EFC: Full history saved. [2025-09-16T18:14:11.264391] [INFO ] [MAIN ] Update Progress: 702/2369 symbols completed. EFC took 34.4s. Estimated remaining: 2h 56m 3s. Rate: 9.5 symbols/min [2025-09-16T18:14:11.264498] [DEBUG] [LOCAL ] EFT: Trimmed 13155 records to 9155 (keeping 150 days of data) [2025-09-16T18:14:11.481729] [INFO ] [MAIN ] EG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:14:11.508197] [DEBUG] [LOCAL ] EFT: Primary data saved (9155 records). [2025-09-16T18:14:11.509221] [INFO ] [MAIN ] EFT: Full history saved. [2025-09-16T18:14:11.509543] [DEBUG] [LOCAL ] EG: Loaded 20850 existing records from primary storage. [2025-09-16T18:14:11.509690] [INFO ] [MAIN ] Update Progress: 703/2369 symbols completed. EFT took 23.2s. Estimated remaining: 2h 55m 42s. Rate: 9.5 symbols/min [2025-09-16T18:14:11.509750] [INFO ] [MAIN ] EG: Loaded 20850 existing records. [2025-09-16T18:14:11.594262] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-03. [2025-09-16T18:14:11.839716] [WARN ] [MAIN ] EG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:14:11.864183] [INFO ] [MAIN ] EGO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:14:11.864385] [DEBUG] [LOCAL ] EGO: No existing data file found in primary storage. [2025-09-16T18:14:11.864503] [WARN ] [MAIN ] EGO: No existing data found, performing full refresh instead. [2025-09-16T18:14:11.864440] [INFO ] [MAIN ] EG: Wiping existing primary data before full refresh. [2025-09-16T18:14:11.864606] [INFO ] [LOCAL ] EG: Starting deletion of primary data. [2025-09-16T18:14:11.864950] [INFO ] [MAIN ] EGO: Wiping existing primary data before full refresh. [2025-09-16T18:14:11.865132] [INFO ] [LOCAL ] EGO: Starting deletion of primary data. [2025-09-16T18:14:11.865192] [DEBUG] [LOCAL ] EGO: No primary data file found to delete. [2025-09-16T18:14:11.865229] [INFO ] [MAIN ] EGO: Starting full primary refresh. [2025-09-16T18:14:11.878579] [DEBUG] [LOCAL ] EG: Deleted primary data file. [2025-09-16T18:14:11.932060] [INFO ] [MAIN ] EG: Starting full primary refresh. [2025-09-16T18:14:12.541952] [INFO ] [REMOTE ] EG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:14:12.592783] [INFO ] [REMOTE ] EGO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:14:12.593008] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-02. [2025-09-16T18:14:12.622678] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-09-16T18:14:13.152250] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T18:14:13.174849] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-04. [2025-09-16T18:14:13.748526] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-09-16T18:14:13.974902] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-03. [2025-09-16T18:14:14.101363] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T18:14:14.774906] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-09. [2025-09-16T18:14:15.400463] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-09-16T18:14:15.574897] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-04. [2025-09-16T18:14:16.374887] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-02. [2025-09-16T18:14:16.660618] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T18:14:16.746985] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-09-16T18:14:16.752494] [INFO ] [REMOTE ] EFX: Full fetch complete (48998 records). [2025-09-16T18:14:16.752603] [DEBUG] [VALIDATOR ] EFX: Validating primary data. [2025-09-16T18:14:17.125088] [DEBUG] [VALIDATOR ] EFX: All expected trading days are present. [2025-09-16T18:14:17.125169] [DEBUG] [VALIDATOR ] EFX: Primary data validation passed. [2025-09-16T18:14:17.125892] [INFO ] [MAIN ] EFX: Fetched primary data is valid. [2025-09-16T18:14:17.125955] [DEBUG] [LOCAL ] EFX: Saving primary data. [2025-09-16T18:14:17.127103] [DEBUG] [LOCAL ] EFX: Trimmed 48998 records to 32959 (keeping 150 days of data) [2025-09-16T18:14:17.190362] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-05. [2025-09-16T18:14:17.344907] [DEBUG] [LOCAL ] EFX: Primary data saved (32959 records). [2025-09-16T18:14:17.345054] [INFO ] [MAIN ] EFX: Full history saved. [2025-09-16T18:14:17.345155] [INFO ] [MAIN ] Update Progress: 704/2369 symbols completed. EFX took 28.1s. Estimated remaining: 2h 55m 35s. Rate: 9.5 symbols/min [2025-09-16T18:14:17.521258] [INFO ] [MAIN ] EGP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:14:17.544834] [DEBUG] [LOCAL ] EGP: Loaded 22063 existing records from primary storage. [2025-09-16T18:14:17.545004] [INFO ] [MAIN ] EGP: Loaded 22063 existing records. [2025-09-16T18:14:17.741510] [WARN ] [MAIN ] EGP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:14:17.741833] [INFO ] [MAIN ] EGP: Wiping existing primary data before full refresh. [2025-09-16T18:14:17.742074] [INFO ] [LOCAL ] EGP: Starting deletion of primary data. [2025-09-16T18:14:17.742871] [DEBUG] [LOCAL ] EGP: Deleted primary data file. [2025-09-16T18:14:17.743128] [INFO ] [MAIN ] EGP: Starting full primary refresh. [2025-09-16T18:14:17.979386] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-05. [2025-09-16T18:14:18.316917] [INFO ] [REMOTE ] EGP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:14:18.354287] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-09-16T18:14:18.410318] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-02 (2.04 seconds). [2025-09-16T18:14:18.774941] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-03. [2025-09-16T18:14:18.973338] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T18:14:19.574930] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-06. [2025-09-16T18:14:20.354978] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-09-16T18:14:20.374868] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-04. [2025-09-16T18:14:20.734196] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-09-16T18:14:21.174927] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-07. [2025-09-16T18:14:21.974972] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-02. [2025-09-16T18:14:22.148576] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T18:14:22.210954] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-09-16T18:14:22.774914] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-08. [2025-09-16T18:14:23.311945] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-09-16T18:14:23.574881] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-05. [2025-09-16T18:14:23.719168] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-09-16T18:14:24.374896] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-03. [2025-09-16T18:14:24.995354] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-09-16T18:14:25.174914] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-06. [2025-09-16T18:14:25.915268] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-09-16T18:14:25.974885] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-04. [2025-09-16T18:14:26.097671] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-09-16T18:14:26.774913] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-06. [2025-09-16T18:14:27.263001] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-09-16T18:14:27.574902] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-07. [2025-09-16T18:14:28.374935] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-05. [2025-09-16T18:14:28.514498] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T18:14:28.920581] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T18:14:29.174908] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-07. [2025-09-16T18:14:29.364927] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T18:14:29.974937] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-09. [2025-09-16T18:14:30.752061] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-09-16T18:14:30.774841] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-08. [2025-09-16T18:14:31.429447] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-09-16T18:14:31.433356] [INFO ] [REMOTE ] EFXT: Full fetch complete (32416 records). [2025-09-16T18:14:31.433435] [DEBUG] [VALIDATOR ] EFXT: Validating primary data. [2025-09-16T18:14:31.582140] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-06. [2025-09-16T18:14:31.814350] [DEBUG] [VALIDATOR ] EFXT: All expected trading days are present. [2025-09-16T18:14:31.814431] [DEBUG] [VALIDATOR ] EFXT: Primary data validation passed. [2025-09-16T18:14:31.814965] [INFO ] [MAIN ] EFXT: Fetched primary data is valid. [2025-09-16T18:14:31.815010] [DEBUG] [LOCAL ] EFXT: Saving primary data. [2025-09-16T18:14:31.815824] [DEBUG] [LOCAL ] EFXT: Trimmed 32416 records to 21228 (keeping 150 days of data) [2025-09-16T18:14:31.945520] [DEBUG] [LOCAL ] EFXT: Primary data saved (21228 records). [2025-09-16T18:14:31.945660] [INFO ] [MAIN ] EFXT: Full history saved. [2025-09-16T18:14:31.945758] [INFO ] [MAIN ] Update Progress: 705/2369 symbols completed. EFXT took 28.5s. Estimated remaining: 2h 55m 48s. Rate: 9.5 symbols/min [2025-09-16T18:14:32.119683] [INFO ] [MAIN ] EGY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:14:32.153370] [DEBUG] [LOCAL ] EGY: Loaded 30410 existing records from primary storage. [2025-09-16T18:14:32.153588] [INFO ] [MAIN ] EGY: Loaded 30410 existing records. [2025-09-16T18:14:32.362463] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T18:14:32.375992] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-08. [2025-09-16T18:14:32.376255] [WARN ] [MAIN ] EGY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:14:32.377396] [INFO ] [MAIN ] EGY: Wiping existing primary data before full refresh. [2025-09-16T18:14:32.377541] [INFO ] [LOCAL ] EGY: Starting deletion of primary data. [2025-09-16T18:14:32.378234] [DEBUG] [LOCAL ] EGY: Deleted primary data file. [2025-09-16T18:14:32.378290] [INFO ] [MAIN ] EGY: Starting full primary refresh. [2025-09-16T18:14:32.732876] [INFO ] [REMOTE ] EGY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:14:32.982083] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-09-16T18:14:33.174921] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-02. [2025-09-16T18:14:33.523937] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T18:14:33.974904] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-09. [2025-09-16T18:14:34.714357] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-09-16T18:14:34.774871] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-09. [2025-09-16T18:14:35.350980] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T18:14:35.357267] [INFO ] [REMOTE ] EGO: Full fetch complete (55836 records). [2025-09-16T18:14:35.357392] [DEBUG] [VALIDATOR ] EGO: Validating primary data. [2025-09-16T18:14:35.575959] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-07. [2025-09-16T18:14:35.772773] [DEBUG] [VALIDATOR ] EGO: All expected trading days are present. [2025-09-16T18:14:35.772856] [DEBUG] [VALIDATOR ] EGO: Primary data validation passed. [2025-09-16T18:14:35.773538] [INFO ] [MAIN ] EGO: Fetched primary data is valid. [2025-09-16T18:14:35.773582] [DEBUG] [LOCAL ] EGO: Saving primary data. [2025-09-16T18:14:35.774818] [DEBUG] [LOCAL ] EGO: Trimmed 55836 records to 38230 (keeping 150 days of data) [2025-09-16T18:14:35.995150] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-09-16T18:14:35.999460] [INFO ] [REMOTE ] EG: Full fetch complete (32025 records). [2025-09-16T18:14:36.000086] [DEBUG] [VALIDATOR ] EG: Validating primary data. [2025-09-16T18:14:36.376227] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-03. [2025-09-16T18:14:36.390742] [DEBUG] [VALIDATOR ] EG: All expected trading days are present. [2025-09-16T18:14:36.391949] [DEBUG] [VALIDATOR ] EG: Primary data validation passed. [2025-09-16T18:14:36.393216] [INFO ] [MAIN ] EG: Fetched primary data is valid. [2025-09-16T18:14:36.394044] [DEBUG] [LOCAL ] EG: Saving primary data. [2025-09-16T18:14:36.398063] [DEBUG] [LOCAL ] EGO: Primary data saved (38230 records). [2025-09-16T18:14:36.398237] [INFO ] [MAIN ] EGO: Full history saved. [2025-09-16T18:14:36.398369] [INFO ] [MAIN ] Update Progress: 706/2369 symbols completed. EGO took 24.9s. Estimated remaining: 2h 55m 37s. Rate: 9.5 symbols/min [2025-09-16T18:14:36.430167] [DEBUG] [LOCAL ] EG: Trimmed 32025 records to 21087 (keeping 150 days of data) [2025-09-16T18:14:36.662885] [INFO ] [MAIN ] EHAB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:14:36.699081] [DEBUG] [LOCAL ] EHAB: Loaded 22563 existing records from primary storage. [2025-09-16T18:14:36.702709] [INFO ] [MAIN ] EHAB: Loaded 22563 existing records. [2025-09-16T18:14:36.863383] [DEBUG] [LOCAL ] EG: Primary data saved (21087 records). [2025-09-16T18:14:36.897118] [INFO ] [MAIN ] EG: Full history saved. [2025-09-16T18:14:36.901530] [WARN ] [MAIN ] EHAB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:14:36.901719] [INFO ] [MAIN ] Update Progress: 707/2369 symbols completed. EG took 25.6s. Estimated remaining: 2h 55m 17s. Rate: 9.5 symbols/min [2025-09-16T18:14:36.902085] [INFO ] [MAIN ] EHAB: Wiping existing primary data before full refresh. [2025-09-16T18:14:36.944064] [INFO ] [LOCAL ] EHAB: Starting deletion of primary data. [2025-09-16T18:14:36.991458] [DEBUG] [LOCAL ] EHAB: Deleted primary data file. [2025-09-16T18:14:37.001555] [INFO ] [MAIN ] EHAB: Starting full primary refresh. [2025-09-16T18:14:37.192038] [INFO ] [MAIN ] EHC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:14:37.453922] [INFO ] [REMOTE ] EHAB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:14:37.463343] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-02. [2025-09-16T18:14:37.464891] [DEBUG] [LOCAL ] EHC: Loaded 30763 existing records from primary storage. [2025-09-16T18:14:37.465152] [INFO ] [MAIN ] EHC: Loaded 30763 existing records. [2025-09-16T18:14:37.656242] [WARN ] [MAIN ] EHC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:14:37.656610] [INFO ] [MAIN ] EHC: Wiping existing primary data before full refresh. [2025-09-16T18:14:37.656740] [INFO ] [LOCAL ] EHC: Starting deletion of primary data. [2025-09-16T18:14:37.664395] [DEBUG] [LOCAL ] EHC: Deleted primary data file. [2025-09-16T18:14:37.676669] [INFO ] [MAIN ] EHC: Starting full primary refresh. [2025-09-16T18:14:37.740596] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-09-16T18:14:37.976950] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-08. [2025-09-16T18:14:38.069418] [INFO ] [REMOTE ] EHC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:14:38.097037] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T18:14:38.379201] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-09-16T18:14:38.774938] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-03. [2025-09-16T18:14:39.308039] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-09-16T18:14:39.554063] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-09-16T18:14:39.574867] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-04. [2025-09-16T18:14:40.374908] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-02. [2025-09-16T18:14:40.822334] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-09-16T18:14:41.174912] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-05. [2025-09-16T18:14:41.932099] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T18:14:41.974935] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-09. [2025-09-16T18:14:42.174364] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T18:14:42.774947] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-04. [2025-09-16T18:14:42.942150] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T18:14:42.946302] [INFO ] [REMOTE ] EGP: Full fetch complete (34484 records). [2025-09-16T18:14:42.946380] [DEBUG] [VALIDATOR ] EGP: Validating primary data. [2025-09-16T18:14:43.309689] [DEBUG] [VALIDATOR ] EGP: All expected trading days are present. [2025-09-16T18:14:43.309770] [DEBUG] [VALIDATOR ] EGP: Primary data validation passed. [2025-09-16T18:14:43.310291] [INFO ] [MAIN ] EGP: Fetched primary data is valid. [2025-09-16T18:14:43.310331] [DEBUG] [LOCAL ] EGP: Saving primary data. [2025-09-16T18:14:43.311123] [DEBUG] [LOCAL ] EGP: Trimmed 34484 records to 22312 (keeping 150 days of data) [2025-09-16T18:14:43.448454] [DEBUG] [LOCAL ] EGP: Primary data saved (22312 records). [2025-09-16T18:14:43.448593] [INFO ] [MAIN ] EGP: Full history saved. [2025-09-16T18:14:43.448697] [INFO ] [MAIN ] Update Progress: 708/2369 symbols completed. EGP took 26.1s. Estimated remaining: 2h 55m 11s. Rate: 9.5 symbols/min [2025-09-16T18:14:43.588049] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-06. [2025-09-16T18:14:43.619817] [INFO ] [MAIN ] EHI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:14:43.629263] [DEBUG] [LOCAL ] EHI: Loaded 7828 existing records from primary storage. [2025-09-16T18:14:43.629456] [INFO ] [MAIN ] EHI: Loaded 7828 existing records. [2025-09-16T18:14:43.805746] [WARN ] [MAIN ] EHI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:14:43.806001] [INFO ] [MAIN ] EHI: Wiping existing primary data before full refresh. [2025-09-16T18:14:43.806048] [INFO ] [LOCAL ] EHI: Starting deletion of primary data. [2025-09-16T18:14:43.806390] [DEBUG] [LOCAL ] EHI: Deleted primary data file. [2025-09-16T18:14:43.806437] [INFO ] [MAIN ] EHI: Starting full primary refresh. [2025-09-16T18:14:44.158107] [INFO ] [REMOTE ] EHI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:14:44.374899] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-03. [2025-09-16T18:14:44.469232] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T18:14:44.534880] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-09-16T18:14:45.174916] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-05. [2025-09-16T18:14:45.958163] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-09-16T18:14:45.974837] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-04. [2025-09-16T18:14:46.774900] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-02. [2025-09-16T18:14:46.982084] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T18:14:47.306867] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T18:14:47.514323] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-09-16T18:14:47.574912] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-05. [2025-09-16T18:14:48.374897] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-06. [2025-09-16T18:14:49.174908] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-03. [2025-09-16T18:14:49.267046] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-09-16T18:14:49.875420] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T18:14:49.925634] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-09-16T18:14:49.974955] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-06. [2025-09-16T18:14:50.774900] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-04. [2025-09-16T18:14:51.253360] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T18:14:51.574900] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-07. [2025-09-16T18:14:51.606578] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-09-16T18:14:52.374910] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-05. [2025-09-16T18:14:53.022757] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-09-16T18:14:53.174868] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-07. [2025-09-16T18:14:53.277710] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T18:14:53.974913] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-08. [2025-09-16T18:14:54.403389] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-09-16T18:14:54.774914] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-06. [2025-09-16T18:14:55.444469] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-09-16T18:14:55.574898] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-09. [2025-09-16T18:14:55.642866] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-09-16T18:14:56.374905] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-07. [2025-09-16T18:14:56.793063] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-09-16T18:14:56.798254] [INFO ] [REMOTE ] EHC: Full fetch complete (45086 records). [2025-09-16T18:14:56.798835] [DEBUG] [VALIDATOR ] EHC: Validating primary data. [2025-09-16T18:14:57.152071] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-09-16T18:14:57.176395] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-08. [2025-09-16T18:14:57.188482] [DEBUG] [VALIDATOR ] EHC: All expected trading days are present. [2025-09-16T18:14:57.188568] [DEBUG] [VALIDATOR ] EHC: Primary data validation passed. [2025-09-16T18:14:57.189261] [INFO ] [MAIN ] EHC: Fetched primary data is valid. [2025-09-16T18:14:57.189389] [DEBUG] [LOCAL ] EHC: Saving primary data. [2025-09-16T18:14:57.190447] [DEBUG] [LOCAL ] EHC: Trimmed 45086 records to 30997 (keeping 150 days of data) [2025-09-16T18:14:57.388736] [DEBUG] [LOCAL ] EHC: Primary data saved (30997 records). [2025-09-16T18:14:57.388966] [INFO ] [MAIN ] EHC: Full history saved. [2025-09-16T18:14:57.389059] [INFO ] [MAIN ] Update Progress: 709/2369 symbols completed. EHC took 20.5s. Estimated remaining: 2h 55m 23s. Rate: 9.5 symbols/min [2025-09-16T18:14:57.558703] [INFO ] [MAIN ] EIC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:14:57.576129] [DEBUG] [LOCAL ] EIC: Loaded 17683 existing records from primary storage. [2025-09-16T18:14:57.576258] [INFO ] [MAIN ] EIC: Loaded 17683 existing records. [2025-09-16T18:14:57.752533] [WARN ] [MAIN ] EIC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:14:57.752854] [INFO ] [MAIN ] EIC: Wiping existing primary data before full refresh. [2025-09-16T18:14:57.752899] [INFO ] [LOCAL ] EIC: Starting deletion of primary data. [2025-09-16T18:14:57.753459] [DEBUG] [LOCAL ] EIC: Deleted primary data file. [2025-09-16T18:14:57.753507] [INFO ] [MAIN ] EIC: Starting full primary refresh. [2025-09-16T18:14:57.986269] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-08. [2025-09-16T18:14:58.111234] [INFO ] [REMOTE ] EIC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:14:58.674105] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T18:14:58.691541] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-09-16T18:14:58.774864] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-09. [2025-09-16T18:14:59.574909] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-02. [2025-09-16T18:14:59.664518] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T18:14:59.666350] [INFO ] [REMOTE ] EHI: Full fetch complete (11362 records). [2025-09-16T18:14:59.666424] [DEBUG] [VALIDATOR ] EHI: Validating primary data. [2025-09-16T18:15:00.040746] [DEBUG] [VALIDATOR ] EHI: All expected trading days are present. [2025-09-16T18:15:00.040827] [DEBUG] [VALIDATOR ] EHI: Primary data validation passed. [2025-09-16T18:15:00.041082] [INFO ] [MAIN ] EHI: Fetched primary data is valid. [2025-09-16T18:15:00.041121] [DEBUG] [LOCAL ] EHI: Saving primary data. [2025-09-16T18:15:00.041584] [DEBUG] [LOCAL ] EHI: Trimmed 11362 records to 7902 (keeping 150 days of data) [2025-09-16T18:15:00.087399] [DEBUG] [LOCAL ] EHI: Primary data saved (7902 records). [2025-09-16T18:15:00.087523] [INFO ] [MAIN ] EHI: Full history saved. [2025-09-16T18:15:00.087605] [INFO ] [MAIN ] Update Progress: 710/2369 symbols completed. EHI took 16.6s. Estimated remaining: 2h 55m 8s. Rate: 9.5 symbols/min [2025-09-16T18:15:00.267232] [INFO ] [MAIN ] EICA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:00.267583] [DEBUG] [LOCAL ] EICA: No existing data file found in primary storage. [2025-09-16T18:15:00.267626] [WARN ] [MAIN ] EICA: No existing data found, performing full refresh instead. [2025-09-16T18:15:00.267795] [INFO ] [MAIN ] EICA: Wiping existing primary data before full refresh. [2025-09-16T18:15:00.267878] [INFO ] [LOCAL ] EICA: Starting deletion of primary data. [2025-09-16T18:15:00.267925] [DEBUG] [LOCAL ] EICA: No primary data file found to delete. [2025-09-16T18:15:00.267961] [INFO ] [MAIN ] EICA: Starting full primary refresh. [2025-09-16T18:15:00.380011] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-07. [2025-09-16T18:15:00.630052] [INFO ] [REMOTE ] EICA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:00.791622] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T18:15:01.174950] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-02. [2025-09-16T18:15:01.641180] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-09-16T18:15:01.974914] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-09. [2025-09-16T18:15:02.732012] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-09-16T18:15:02.774843] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-08. [2025-09-16T18:15:03.480777] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-09-16T18:15:03.485623] [INFO ] [REMOTE ] EGY: Full fetch complete (45923 records). [2025-09-16T18:15:03.485733] [DEBUG] [VALIDATOR ] EGY: Validating primary data. [2025-09-16T18:15:03.583275] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-03. [2025-09-16T18:15:03.860530] [DEBUG] [VALIDATOR ] EGY: All expected trading days are present. [2025-09-16T18:15:03.860682] [DEBUG] [VALIDATOR ] EGY: Primary data validation passed. [2025-09-16T18:15:03.861330] [INFO ] [MAIN ] EGY: Fetched primary data is valid. [2025-09-16T18:15:03.861456] [DEBUG] [LOCAL ] EGY: Saving primary data. [2025-09-16T18:15:03.862681] [DEBUG] [LOCAL ] EGY: Trimmed 45923 records to 30724 (keeping 150 days of data) [2025-09-16T18:15:03.881295] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-09-16T18:15:03.979115] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-09-16T18:15:04.072440] [DEBUG] [LOCAL ] EGY: Primary data saved (30724 records). [2025-09-16T18:15:04.072570] [INFO ] [MAIN ] EGY: Full history saved. [2025-09-16T18:15:04.073443] [INFO ] [MAIN ] Update Progress: 711/2369 symbols completed. EGY took 32.1s. Estimated remaining: 2h 54m 56s. Rate: 9.5 symbols/min [2025-09-16T18:15:04.242278] [INFO ] [MAIN ] EICB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:04.242427] [DEBUG] [LOCAL ] EICB: No existing data file found in primary storage. [2025-09-16T18:15:04.242465] [WARN ] [MAIN ] EICB: No existing data found, performing full refresh instead. [2025-09-16T18:15:04.242578] [INFO ] [MAIN ] EICB: Wiping existing primary data before full refresh. [2025-09-16T18:15:04.242617] [INFO ] [LOCAL ] EICB: Starting deletion of primary data. [2025-09-16T18:15:04.242680] [DEBUG] [LOCAL ] EICB: No primary data file found to delete. [2025-09-16T18:15:04.242716] [INFO ] [MAIN ] EICB: Starting full primary refresh. [2025-09-16T18:15:04.382612] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-03. [2025-09-16T18:15:04.595382] [INFO ] [REMOTE ] EICB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:05.174906] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-09. [2025-09-16T18:15:05.863549] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T18:15:05.946878] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-09-16T18:15:05.950067] [INFO ] [REMOTE ] EHAB: Full fetch complete (32217 records). [2025-09-16T18:15:05.950142] [DEBUG] [VALIDATOR ] EHAB: Validating primary data. [2025-09-16T18:15:05.979972] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-04. [2025-09-16T18:15:06.324803] [DEBUG] [VALIDATOR ] EHAB: All expected trading days are present. [2025-09-16T18:15:06.325011] [DEBUG] [VALIDATOR ] EHAB: Primary data validation passed. [2025-09-16T18:15:06.325552] [INFO ] [MAIN ] EHAB: Fetched primary data is valid. [2025-09-16T18:15:06.325595] [DEBUG] [LOCAL ] EHAB: Saving primary data. [2025-09-16T18:15:06.326359] [DEBUG] [LOCAL ] EHAB: Trimmed 32217 records to 22697 (keeping 150 days of data) [2025-09-16T18:15:06.457297] [DEBUG] [LOCAL ] EHAB: Primary data saved (22697 records). [2025-09-16T18:15:06.457464] [INFO ] [MAIN ] EHAB: Full history saved. [2025-09-16T18:15:06.457585] [INFO ] [MAIN ] Update Progress: 712/2369 symbols completed. EHAB took 30.1s. Estimated remaining: 2h 54m 40s. Rate: 9.5 symbols/min [2025-09-16T18:15:06.650186] [INFO ] [MAIN ] EICC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:06.650412] [DEBUG] [LOCAL ] EICC: No existing data file found in primary storage. [2025-09-16T18:15:06.650453] [WARN ] [MAIN ] EICC: No existing data found, performing full refresh instead. [2025-09-16T18:15:06.650606] [INFO ] [MAIN ] EICC: Wiping existing primary data before full refresh. [2025-09-16T18:15:06.650665] [INFO ] [LOCAL ] EICC: Starting deletion of primary data. [2025-09-16T18:15:06.650711] [DEBUG] [LOCAL ] EICC: No primary data file found to delete. [2025-09-16T18:15:06.650746] [INFO ] [MAIN ] EICC: Starting full primary refresh. [2025-09-16T18:15:06.780002] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-04. [2025-09-16T18:15:07.018587] [INFO ] [REMOTE ] EICC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:07.181624] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-09-16T18:15:07.309871] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T18:15:07.574917] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-02. [2025-09-16T18:15:08.139021] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-09-16T18:15:08.374914] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-05. [2025-09-16T18:15:09.174857] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-02. [2025-09-16T18:15:09.576830] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T18:15:09.667286] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-09-16T18:15:09.974908] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-03. [2025-09-16T18:15:10.652395] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-09-16T18:15:10.774912] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-06. [2025-09-16T18:15:11.574928] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-05. [2025-09-16T18:15:11.767896] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-09-16T18:15:11.944756] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-09-16T18:15:12.374913] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-04. [2025-09-16T18:15:13.097877] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-09-16T18:15:13.174893] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-03. [2025-09-16T18:15:13.605470] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-09-16T18:15:13.974920] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-06. [2025-09-16T18:15:14.254547] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-06 (0.28 seconds). [2025-09-16T18:15:14.774911] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-07. [2025-09-16T18:15:15.121013] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-09-16T18:15:15.574898] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-04. [2025-09-16T18:15:16.044613] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-09-16T18:15:16.374904] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-05. [2025-09-16T18:15:16.790404] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T18:15:17.174895] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-08. [2025-09-16T18:15:17.508976] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-09-16T18:15:17.974919] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-05. [2025-09-16T18:15:18.573538] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-09-16T18:15:18.774901] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-06. [2025-09-16T18:15:19.351659] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-09-16T18:15:19.574904] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-09. [2025-09-16T18:15:19.982714] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-09-16T18:15:19.983432] [INFO ] [REMOTE ] EICA: Full fetch complete (872 records). [2025-09-16T18:15:19.983526] [DEBUG] [VALIDATOR ] EICA: Validating primary data. [2025-09-16T18:15:19.983928] [FATAL] [VALIDATOR ] EICA: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:15:19.984002] [FATAL] [MAIN ] EICA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:15:19.984047] [WARN ] [MAIN ] EICA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:15:19.984092] [INFO ] [LOCAL ] EICA: Starting deletion of primary data. [2025-09-16T18:15:19.984162] [DEBUG] [LOCAL ] EICA: No primary data file found to delete. [2025-09-16T18:15:19.984201] [INFO ] [LOCAL ] EICA: Starting deletion of derivative data. [2025-09-16T18:15:19.984259] [DEBUG] [LOCAL ] EICA: No derivative data file found to delete. [2025-09-16T18:15:19.984315] [INFO ] [MAIN ] Update Progress: 713/2369 symbols completed. EICA took 19.9s. Estimated remaining: 2h 54m 51s. Rate: 9.5 symbols/min [2025-09-16T18:15:20.157035] [INFO ] [MAIN ] EIG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:20.172788] [DEBUG] [LOCAL ] EIG: Loaded 15429 existing records from primary storage. [2025-09-16T18:15:20.172934] [INFO ] [MAIN ] EIG: Loaded 15429 existing records. [2025-09-16T18:15:20.359958] [WARN ] [MAIN ] EIG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:15:20.360247] [INFO ] [MAIN ] EIG: Wiping existing primary data before full refresh. [2025-09-16T18:15:20.360308] [INFO ] [LOCAL ] EIG: Starting deletion of primary data. [2025-09-16T18:15:20.360898] [DEBUG] [LOCAL ] EIG: Deleted primary data file. [2025-09-16T18:15:20.360953] [INFO ] [MAIN ] EIG: Starting full primary refresh. [2025-09-16T18:15:20.381759] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-07. [2025-09-16T18:15:20.724726] [INFO ] [REMOTE ] EIG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:20.974293] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-09-16T18:15:21.174913] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-07. [2025-09-16T18:15:21.974942] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-06. [2025-09-16T18:15:22.306400] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-09-16T18:15:22.569790] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T18:15:22.774960] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-08. [2025-09-16T18:15:23.574877] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-02. [2025-09-16T18:15:23.957177] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T18:15:24.374940] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-07. [2025-09-16T18:15:24.399160] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T18:15:25.148831] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T18:15:25.174876] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-08. [2025-09-16T18:15:25.767403] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-09-16T18:15:25.974947] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-09. [2025-09-16T18:15:26.587210] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-09-16T18:15:26.588623] [INFO ] [REMOTE ] EICC: Full fetch complete (6244 records). [2025-09-16T18:15:26.588725] [DEBUG] [VALIDATOR ] EICC: Validating primary data. [2025-09-16T18:15:26.589130] [FATAL] [VALIDATOR ] EICC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:15:26.589207] [FATAL] [MAIN ] EICC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:15:26.589252] [WARN ] [MAIN ] EICC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:15:26.589313] [INFO ] [LOCAL ] EICC: Starting deletion of primary data. [2025-09-16T18:15:26.589384] [DEBUG] [LOCAL ] EICC: No primary data file found to delete. [2025-09-16T18:15:26.589435] [INFO ] [LOCAL ] EICC: Starting deletion of derivative data. [2025-09-16T18:15:26.589517] [DEBUG] [LOCAL ] EICC: No derivative data file found to delete. [2025-09-16T18:15:26.589601] [INFO ] [MAIN ] Update Progress: 714/2369 symbols completed. EICC took 20.1s. Estimated remaining: 2h 54m 45s. Rate: 9.5 symbols/min [2025-09-16T18:15:26.761371] [INFO ] [MAIN ] EIX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:26.761560] [DEBUG] [LOCAL ] EIX: No existing data file found in primary storage. [2025-09-16T18:15:26.761613] [WARN ] [MAIN ] EIX: No existing data found, performing full refresh instead. [2025-09-16T18:15:26.761793] [INFO ] [MAIN ] EIX: Wiping existing primary data before full refresh. [2025-09-16T18:15:26.761856] [INFO ] [LOCAL ] EIX: Starting deletion of primary data. [2025-09-16T18:15:26.761915] [DEBUG] [LOCAL ] EIX: No primary data file found to delete. [2025-09-16T18:15:26.761962] [INFO ] [MAIN ] EIX: Starting full primary refresh. [2025-09-16T18:15:26.782207] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-03. [2025-09-16T18:15:27.115000] [INFO ] [REMOTE ] EIX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:27.574921] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-02. [2025-09-16T18:15:27.915857] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-09-16T18:15:28.374900] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-04. [2025-09-16T18:15:29.174883] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-09. [2025-09-16T18:15:29.244472] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T18:15:29.468001] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T18:15:29.974947] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-03. [2025-09-16T18:15:30.274550] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-09-16T18:15:30.282605] [INFO ] [REMOTE ] EIC: Full fetch complete (26909 records). [2025-09-16T18:15:30.282845] [DEBUG] [VALIDATOR ] EIC: Validating primary data. [2025-09-16T18:15:30.783439] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-08. [2025-09-16T18:15:30.923210] [DEBUG] [VALIDATOR ] EIC: All expected trading days are present. [2025-09-16T18:15:30.923334] [DEBUG] [VALIDATOR ] EIC: Primary data validation passed. [2025-09-16T18:15:30.924199] [INFO ] [MAIN ] EIC: Fetched primary data is valid. [2025-09-16T18:15:30.924290] [DEBUG] [LOCAL ] EIC: Saving primary data. [2025-09-16T18:15:30.925411] [DEBUG] [LOCAL ] EIC: Trimmed 26909 records to 17796 (keeping 150 days of data) [2025-09-16T18:15:31.042516] [DEBUG] [LOCAL ] EIC: Primary data saved (17796 records). [2025-09-16T18:15:31.042732] [INFO ] [MAIN ] EIC: Full history saved. [2025-09-16T18:15:31.042880] [INFO ] [MAIN ] Update Progress: 715/2369 symbols completed. EIC took 33.7s. Estimated remaining: 2h 54m 34s. Rate: 9.5 symbols/min [2025-09-16T18:15:31.243467] [INFO ] [MAIN ] EL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:31.282774] [DEBUG] [LOCAL ] EL: Loaded 39517 existing records from primary storage. [2025-09-16T18:15:31.282924] [INFO ] [MAIN ] EL: Loaded 39517 existing records. [2025-09-16T18:15:31.386219] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T18:15:31.481576] [WARN ] [MAIN ] EL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:15:31.481925] [INFO ] [MAIN ] EL: Wiping existing primary data before full refresh. [2025-09-16T18:15:31.481996] [INFO ] [LOCAL ] EL: Starting deletion of primary data. [2025-09-16T18:15:31.483212] [DEBUG] [LOCAL ] EL: Deleted primary data file. [2025-09-16T18:15:31.483287] [INFO ] [MAIN ] EL: Starting full primary refresh. [2025-09-16T18:15:31.580018] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-09. [2025-09-16T18:15:31.840980] [INFO ] [REMOTE ] EL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:32.009629] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-09-16T18:15:32.073716] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-09-16T18:15:32.074554] [INFO ] [REMOTE ] EICB: Full fetch complete (1970 records). [2025-09-16T18:15:32.074623] [DEBUG] [VALIDATOR ] EICB: Validating primary data. [2025-09-16T18:15:32.075006] [FATAL] [VALIDATOR ] EICB: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:15:32.075080] [FATAL] [MAIN ] EICB: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:15:32.075116] [WARN ] [MAIN ] EICB: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:15:32.075156] [INFO ] [LOCAL ] EICB: Starting deletion of primary data. [2025-09-16T18:15:32.075215] [DEBUG] [LOCAL ] EICB: No primary data file found to delete. [2025-09-16T18:15:32.075250] [INFO ] [LOCAL ] EICB: Starting deletion of derivative data. [2025-09-16T18:15:32.075289] [DEBUG] [LOCAL ] EICB: No derivative data file found to delete. [2025-09-16T18:15:32.075343] [INFO ] [MAIN ] Update Progress: 716/2369 symbols completed. EICB took 28.0s. Estimated remaining: 2h 54m 16s. Rate: 9.5 symbols/min [2025-09-16T18:15:32.247087] [INFO ] [MAIN ] ELAN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:32.286781] [DEBUG] [LOCAL ] ELAN: Loaded 39316 existing records from primary storage. [2025-09-16T18:15:32.286922] [INFO ] [MAIN ] ELAN: Loaded 39316 existing records. [2025-09-16T18:15:32.380052] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-02. [2025-09-16T18:15:32.471603] [WARN ] [MAIN ] ELAN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:15:32.471830] [INFO ] [MAIN ] ELAN: Wiping existing primary data before full refresh. [2025-09-16T18:15:32.471877] [INFO ] [LOCAL ] ELAN: Starting deletion of primary data. [2025-09-16T18:15:32.472782] [DEBUG] [LOCAL ] ELAN: Deleted primary data file. [2025-09-16T18:15:32.472853] [INFO ] [MAIN ] ELAN: Starting full primary refresh. [2025-09-16T18:15:32.847701] [INFO ] [REMOTE ] ELAN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:33.174912] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-02. [2025-09-16T18:15:33.974947] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-04. [2025-09-16T18:15:34.340710] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-09-16T18:15:34.774893] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-03. [2025-09-16T18:15:34.857205] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T18:15:35.491665] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T18:15:35.574912] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-05. [2025-09-16T18:15:36.374898] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-05. [2025-09-16T18:15:36.789014] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-09-16T18:15:37.077396] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-09-16T18:15:37.174867] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-06. [2025-09-16T18:15:37.974865] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-04. [2025-09-16T18:15:38.075979] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T18:15:38.096985] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-09-16T18:15:38.774909] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-07. [2025-09-16T18:15:39.574923] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-06. [2025-09-16T18:15:39.865003] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T18:15:40.086576] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-04 (2.11 seconds). [2025-09-16T18:15:40.374905] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-05. [2025-09-16T18:15:41.174937] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-08. [2025-09-16T18:15:41.382229] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-09-16T18:15:41.974875] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-07. [2025-09-16T18:15:42.204337] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-09-16T18:15:42.367465] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-09-16T18:15:42.774929] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-03. [2025-09-16T18:15:43.574932] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-06. [2025-09-16T18:15:43.993179] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-09-16T18:15:44.374940] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-08. [2025-09-16T18:15:44.480692] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T18:15:45.174904] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-04. [2025-09-16T18:15:45.591895] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-06 (2.02 seconds). [2025-09-16T18:15:45.974904] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-09. [2025-09-16T18:15:46.395838] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-09-16T18:15:46.774923] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-09. [2025-09-16T18:15:46.825335] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T18:15:46.828401] [INFO ] [REMOTE ] EIG: Full fetch complete (21708 records). [2025-09-16T18:15:46.828534] [DEBUG] [VALIDATOR ] EIG: Validating primary data. [2025-09-16T18:15:47.237171] [DEBUG] [VALIDATOR ] EIG: All expected trading days are present. [2025-09-16T18:15:47.237296] [DEBUG] [VALIDATOR ] EIG: Primary data validation passed. [2025-09-16T18:15:47.237911] [INFO ] [MAIN ] EIG: Fetched primary data is valid. [2025-09-16T18:15:47.238006] [DEBUG] [LOCAL ] EIG: Saving primary data. [2025-09-16T18:15:47.239140] [DEBUG] [LOCAL ] EIG: Trimmed 21708 records to 15610 (keeping 150 days of data) [2025-09-16T18:15:47.351445] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-09-16T18:15:47.377260] [DEBUG] [LOCAL ] EIG: Primary data saved (15610 records). [2025-09-16T18:15:47.377394] [INFO ] [MAIN ] EIG: Full history saved. [2025-09-16T18:15:47.377495] [INFO ] [MAIN ] Update Progress: 717/2369 symbols completed. EIG took 27.4s. Estimated remaining: 2h 54m 30s. Rate: 9.5 symbols/min [2025-09-16T18:15:47.574917] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-07. [2025-09-16T18:15:48.374936] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-05. [2025-09-16T18:15:48.569038] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-09-16T18:15:48.575540] [INFO ] [REMOTE ] EIX: Full fetch complete (58036 records). [2025-09-16T18:15:48.575725] [DEBUG] [VALIDATOR ] EIX: Validating primary data. [2025-09-16T18:15:48.966771] [DEBUG] [VALIDATOR ] EIX: All expected trading days are present. [2025-09-16T18:15:48.966873] [DEBUG] [VALIDATOR ] EIX: Primary data validation passed. [2025-09-16T18:15:48.967585] [INFO ] [MAIN ] EIX: Fetched primary data is valid. [2025-09-16T18:15:48.967628] [DEBUG] [LOCAL ] EIX: Saving primary data. [2025-09-16T18:15:48.968947] [DEBUG] [LOCAL ] EIX: Trimmed 58036 records to 39795 (keeping 150 days of data) [2025-09-16T18:15:49.210397] [DEBUG] [LOCAL ] EIX: Primary data saved (39795 records). [2025-09-16T18:15:49.210576] [INFO ] [MAIN ] EIX: Full history saved. [2025-09-16T18:15:49.210747] [INFO ] [MAIN ] Update Progress: 718/2369 symbols completed. EIX took 22.6s. Estimated remaining: 2h 54m 14s. Rate: 9.5 symbols/min [2025-09-16T18:15:49.719038] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-09-16T18:15:49.731007] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-08. [2025-09-16T18:15:49.889872] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-09-16T18:15:49.974948] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-06. [2025-09-16T18:15:51.233156] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T18:15:51.244590] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-07. [2025-09-16T18:15:51.843140] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-09-16T18:15:51.854693] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-09. [2025-09-16T18:15:52.804828] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-09-16T18:15:52.816788] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-08. [2025-09-16T18:15:53.857240] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-09-16T18:15:53.864924] [INFO ] [REMOTE ] EL: Full fetch complete (58143 records). [2025-09-16T18:15:53.865158] [DEBUG] [VALIDATOR ] EL: Validating primary data. [2025-09-16T18:15:54.331615] [DEBUG] [VALIDATOR ] EL: All expected trading days are present. [2025-09-16T18:15:54.331751] [DEBUG] [VALIDATOR ] EL: Primary data validation passed. [2025-09-16T18:15:54.332467] [INFO ] [MAIN ] EL: Fetched primary data is valid. [2025-09-16T18:15:54.332512] [DEBUG] [LOCAL ] EL: Saving primary data. [2025-09-16T18:15:54.333879] [DEBUG] [LOCAL ] EL: Trimmed 58143 records to 39892 (keeping 150 days of data) [2025-09-16T18:15:54.583479] [DEBUG] [LOCAL ] EL: Primary data saved (39892 records). [2025-09-16T18:15:54.583654] [INFO ] [MAIN ] EL: Full history saved. [2025-09-16T18:15:54.583746] [INFO ] [MAIN ] Update Progress: 719/2369 symbols completed. EL took 23.5s. Estimated remaining: 2h 54m 5s. Rate: 9.5 symbols/min [2025-09-16T18:15:54.633340] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T18:15:54.644776] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-09. [2025-09-16T18:15:56.411679] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-09-16T18:15:56.418294] [INFO ] [REMOTE ] ELAN: Full fetch complete (57769 records). [2025-09-16T18:15:56.418445] [DEBUG] [VALIDATOR ] ELAN: Validating primary data. [2025-09-16T18:15:56.804069] [DEBUG] [VALIDATOR ] ELAN: All expected trading days are present. [2025-09-16T18:15:56.804170] [DEBUG] [VALIDATOR ] ELAN: Primary data validation passed. [2025-09-16T18:15:56.805105] [INFO ] [MAIN ] ELAN: Fetched primary data is valid. [2025-09-16T18:15:56.805174] [DEBUG] [LOCAL ] ELAN: Saving primary data. [2025-09-16T18:15:56.806594] [DEBUG] [LOCAL ] ELAN: Trimmed 57769 records to 39704 (keeping 150 days of data) [2025-09-16T18:15:57.068274] [DEBUG] [LOCAL ] ELAN: Primary data saved (39704 records). [2025-09-16T18:15:57.068438] [INFO ] [MAIN ] ELAN: Full history saved. [2025-09-16T18:15:57.068564] [INFO ] [MAIN ] Update Progress: 720/2369 symbols completed. ELAN took 25.0s. Estimated remaining: 2h 53m 50s. Rate: 9.5 symbols/min [2025-09-16T18:15:57.699165] [INFO ] [MAIN ] ELP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:57.809787] [INFO ] [MAIN ] ELC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:57.812731] [INFO ] [MAIN ] ELME: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:57.813566] [DEBUG] [LOCAL ] ELC: No existing data file found in primary storage. [2025-09-16T18:15:57.814407] [DEBUG] [LOCAL ] ELME: No existing data file found in primary storage. [2025-09-16T18:15:57.832716] [INFO ] [MAIN ] ELF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:15:57.814831] [WARN ] [MAIN ] ELC: No existing data found, performing full refresh instead. [2025-09-16T18:15:57.815345] [WARN ] [MAIN ] ELME: No existing data found, performing full refresh instead. [2025-09-16T18:15:57.833152] [INFO ] [MAIN ] ELC: Wiping existing primary data before full refresh. [2025-09-16T18:15:57.833683] [INFO ] [MAIN ] ELME: Wiping existing primary data before full refresh. [2025-09-16T18:15:57.833967] [INFO ] [LOCAL ] ELME: Starting deletion of primary data. [2025-09-16T18:15:57.833785] [INFO ] [LOCAL ] ELC: Starting deletion of primary data. [2025-09-16T18:15:57.838117] [DEBUG] [LOCAL ] ELC: No primary data file found to delete. [2025-09-16T18:15:57.838433] [DEBUG] [LOCAL ] ELME: No primary data file found to delete. [2025-09-16T18:15:57.838362] [INFO ] [MAIN ] ELC: Starting full primary refresh. [2025-09-16T18:15:57.838999] [INFO ] [MAIN ] ELME: Starting full primary refresh. [2025-09-16T18:15:58.119139] [DEBUG] [LOCAL ] ELP: Loaded 19225 existing records from primary storage. [2025-09-16T18:15:58.140582] [INFO ] [MAIN ] ELP: Loaded 19225 existing records. [2025-09-16T18:15:58.381106] [DEBUG] [LOCAL ] ELF: Loaded 38501 existing records from primary storage. [2025-09-16T18:15:58.531241] [INFO ] [MAIN ] ELF: Loaded 38501 existing records. [2025-09-16T18:15:58.854724] [WARN ] [MAIN ] ELP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:15:58.984512] [INFO ] [REMOTE ] ELME: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:59.024253] [INFO ] [MAIN ] ELP: Wiping existing primary data before full refresh. [2025-09-16T18:15:59.061479] [INFO ] [REMOTE ] ELC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:59.081927] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-02. [2025-09-16T18:15:59.066738] [INFO ] [LOCAL ] ELP: Starting deletion of primary data. [2025-09-16T18:15:59.091508] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-02. [2025-09-16T18:15:59.104000] [DEBUG] [LOCAL ] ELP: Deleted primary data file. [2025-09-16T18:15:59.104247] [INFO ] [MAIN ] ELP: Starting full primary refresh. [2025-09-16T18:15:59.130944] [WARN ] [MAIN ] ELF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:15:59.200601] [INFO ] [MAIN ] ELF: Wiping existing primary data before full refresh. [2025-09-16T18:15:59.232074] [INFO ] [LOCAL ] ELF: Starting deletion of primary data. [2025-09-16T18:15:59.240712] [DEBUG] [LOCAL ] ELF: Deleted primary data file. [2025-09-16T18:15:59.261412] [INFO ] [MAIN ] ELF: Starting full primary refresh. [2025-09-16T18:15:59.681886] [INFO ] [REMOTE ] ELP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:59.695234] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-02. [2025-09-16T18:15:59.806548] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-09-16T18:15:59.807987] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-03. [2025-09-16T18:15:59.889671] [INFO ] [REMOTE ] ELF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:15:59.890055] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-02. [2025-09-16T18:16:00.564060] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-09-16T18:16:00.565589] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-04. [2025-09-16T18:16:00.699330] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T18:16:00.707262] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-03. [2025-09-16T18:16:01.143039] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-09-16T18:16:01.144731] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-05. [2025-09-16T18:16:01.229875] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T18:16:01.235476] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-03. [2025-09-16T18:16:01.902832] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-09-16T18:16:01.904707] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-06. [2025-09-16T18:16:01.949670] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-09-16T18:16:01.974913] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-04. [2025-09-16T18:16:02.090584] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-02 (2.20 seconds). [2025-09-16T18:16:02.459506] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-09-16T18:16:02.501473] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T18:16:02.774906] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-04. [2025-09-16T18:16:03.152705] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T18:16:03.574909] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-05. [2025-09-16T18:16:04.257990] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T18:16:04.374922] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-05. [2025-09-16T18:16:05.174902] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-07. [2025-09-16T18:16:05.672897] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T18:16:05.872693] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-09-16T18:16:05.915059] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-05 (2.34 seconds). [2025-09-16T18:16:05.974942] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-06. [2025-09-16T18:16:06.774906] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-03. [2025-09-16T18:16:07.193239] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-09-16T18:16:07.574961] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-08. [2025-09-16T18:16:08.256819] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-09-16T18:16:08.374897] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-09. [2025-09-16T18:16:08.734558] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-09-16T18:16:08.997167] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-09-16T18:16:08.998336] [INFO ] [REMOTE ] ELC: Full fetch complete (3778 records). [2025-09-16T18:16:08.998431] [DEBUG] [VALIDATOR ] ELC: Validating primary data. [2025-09-16T18:16:09.176689] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-04. [2025-09-16T18:16:09.363722] [DEBUG] [VALIDATOR ] ELC: All expected trading days are present. [2025-09-16T18:16:09.363803] [DEBUG] [VALIDATOR ] ELC: Primary data validation passed. [2025-09-16T18:16:09.363971] [INFO ] [MAIN ] ELC: Fetched primary data is valid. [2025-09-16T18:16:09.364037] [DEBUG] [LOCAL ] ELC: Saving primary data. [2025-09-16T18:16:09.364413] [DEBUG] [LOCAL ] ELC: Trimmed 3778 records to 2672 (keeping 150 days of data) [2025-09-16T18:16:09.382445] [DEBUG] [LOCAL ] ELC: Primary data saved (2672 records). [2025-09-16T18:16:09.382519] [INFO ] [MAIN ] ELC: Full history saved. [2025-09-16T18:16:09.382608] [INFO ] [MAIN ] Update Progress: 721/2369 symbols completed. ELC took 12.3s. Estimated remaining: 2h 53m 57s. Rate: 9.5 symbols/min [2025-09-16T18:16:09.562016] [INFO ] [MAIN ] ELPC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:16:09.562194] [DEBUG] [LOCAL ] ELPC: No existing data file found in primary storage. [2025-09-16T18:16:09.562235] [WARN ] [MAIN ] ELPC: No existing data found, performing full refresh instead. [2025-09-16T18:16:09.562358] [INFO ] [MAIN ] ELPC: Wiping existing primary data before full refresh. [2025-09-16T18:16:09.562401] [INFO ] [LOCAL ] ELPC: Starting deletion of primary data. [2025-09-16T18:16:09.562445] [DEBUG] [LOCAL ] ELPC: No primary data file found to delete. [2025-09-16T18:16:09.562493] [INFO ] [MAIN ] ELPC: Starting full primary refresh. [2025-09-16T18:16:09.914452] [INFO ] [REMOTE ] ELPC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:16:09.974864] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-07. [2025-09-16T18:16:10.774907] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-02. [2025-09-16T18:16:11.167023] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-09-16T18:16:11.229545] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-09-16T18:16:11.230843] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T18:16:11.574905] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-08. [2025-09-16T18:16:12.374903] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-05. [2025-09-16T18:16:13.174944] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-06. [2025-09-16T18:16:13.246042] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:16:13.974904] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-09. [2025-09-16T18:16:14.343490] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-09-16T18:16:14.605636] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T18:16:14.774981] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-06. [2025-09-16T18:16:15.430452] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-09-16T18:16:15.436339] [INFO ] [REMOTE ] ELME: Full fetch complete (42849 records). [2025-09-16T18:16:15.436497] [DEBUG] [VALIDATOR ] ELME: Validating primary data. [2025-09-16T18:16:15.592142] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-07. [2025-09-16T18:16:15.850875] [DEBUG] [VALIDATOR ] ELME: All expected trading days are present. [2025-09-16T18:16:15.850990] [DEBUG] [VALIDATOR ] ELME: Primary data validation passed. [2025-09-16T18:16:15.851665] [INFO ] [MAIN ] ELME: Fetched primary data is valid. [2025-09-16T18:16:15.851735] [DEBUG] [LOCAL ] ELME: Saving primary data. [2025-09-16T18:16:15.853242] [DEBUG] [LOCAL ] ELME: Trimmed 42849 records to 29238 (keeping 150 days of data) [2025-09-16T18:16:16.042489] [DEBUG] [LOCAL ] ELME: Primary data saved (29238 records). [2025-09-16T18:16:16.042657] [INFO ] [MAIN ] ELME: Full history saved. [2025-09-16T18:16:16.042754] [INFO ] [MAIN ] Update Progress: 722/2369 symbols completed. ELME took 19.0s. Estimated remaining: 2h 53m 52s. Rate: 9.5 symbols/min [2025-09-16T18:16:16.210318] [INFO ] [MAIN ] ELS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:16:16.210483] [DEBUG] [LOCAL ] ELS: No existing data file found in primary storage. [2025-09-16T18:16:16.210542] [WARN ] [MAIN ] ELS: No existing data found, performing full refresh instead. [2025-09-16T18:16:16.210685] [INFO ] [MAIN ] ELS: Wiping existing primary data before full refresh. [2025-09-16T18:16:16.210768] [INFO ] [LOCAL ] ELS: Starting deletion of primary data. [2025-09-16T18:16:16.210812] [DEBUG] [LOCAL ] ELS: No primary data file found to delete. [2025-09-16T18:16:16.210845] [INFO ] [MAIN ] ELS: Starting full primary refresh. [2025-09-16T18:16:16.387292] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-03. [2025-09-16T18:16:16.592787] [INFO ] [REMOTE ] ELS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:16:16.681576] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-09-16T18:16:16.820747] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-09-16T18:16:16.952275] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-09-16T18:16:17.174910] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-07. [2025-09-16T18:16:17.974909] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-02. [2025-09-16T18:16:18.774885] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-04. [2025-09-16T18:16:19.072223] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-09-16T18:16:19.175190] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-09-16T18:16:19.303096] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-09-16T18:16:19.574863] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-03. [2025-09-16T18:16:20.374910] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-08. [2025-09-16T18:16:21.174867] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-08. [2025-09-16T18:16:21.210064] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T18:16:21.974908] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-04. [2025-09-16T18:16:22.389681] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-09-16T18:16:22.442787] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-09-16T18:16:22.774911] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-09. [2025-09-16T18:16:23.574963] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-09. [2025-09-16T18:16:23.695662] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T18:16:24.374908] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-05. [2025-09-16T18:16:24.517558] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-09-16T18:16:24.523595] [INFO ] [REMOTE ] ELF: Full fetch complete (56999 records). [2025-09-16T18:16:24.523711] [DEBUG] [VALIDATOR ] ELF: Validating primary data. [2025-09-16T18:16:24.904234] [DEBUG] [VALIDATOR ] ELF: All expected trading days are present. [2025-09-16T18:16:24.904312] [DEBUG] [VALIDATOR ] ELF: Primary data validation passed. [2025-09-16T18:16:24.905070] [INFO ] [MAIN ] ELF: Fetched primary data is valid. [2025-09-16T18:16:24.905117] [DEBUG] [LOCAL ] ELF: Saving primary data. [2025-09-16T18:16:24.906256] [DEBUG] [LOCAL ] ELF: Trimmed 56999 records to 38878 (keeping 150 days of data) [2025-09-16T18:16:24.979899] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T18:16:25.009243] [INFO ] [REMOTE ] ELP: Full fetch complete (28043 records). [2025-09-16T18:16:25.010133] [DEBUG] [VALIDATOR ] ELP: Validating primary data. [2025-09-16T18:16:25.015802] [FATAL] [VALIDATOR ] ELP: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:16:25.016602] [FATAL] [MAIN ] ELP: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:16:25.017726] [WARN ] [MAIN ] ELP: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:16:25.018092] [INFO ] [LOCAL ] ELP: Starting deletion of primary data. [2025-09-16T18:16:25.018704] [DEBUG] [LOCAL ] ELP: No primary data file found to delete. [2025-09-16T18:16:25.019302] [INFO ] [LOCAL ] ELP: Starting deletion of derivative data. [2025-09-16T18:16:25.020627] [DEBUG] [LOCAL ] ELP: Deleted derivative data file. [2025-09-16T18:16:25.021330] [INFO ] [MAIN ] Update Progress: 723/2369 symbols completed. ELP took 28.0s. Estimated remaining: 2h 53m 51s. Rate: 9.5 symbols/min [2025-09-16T18:16:25.177596] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-05. [2025-09-16T18:16:25.201714] [INFO ] [MAIN ] ELV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:16:25.203199] [DEBUG] [LOCAL ] ELV: No existing data file found in primary storage. [2025-09-16T18:16:25.203950] [WARN ] [MAIN ] ELV: No existing data found, performing full refresh instead. [2025-09-16T18:16:25.204567] [INFO ] [MAIN ] ELV: Wiping existing primary data before full refresh. [2025-09-16T18:16:25.205211] [INFO ] [LOCAL ] ELV: Starting deletion of primary data. [2025-09-16T18:16:25.205829] [DEBUG] [LOCAL ] ELV: No primary data file found to delete. [2025-09-16T18:16:25.206605] [INFO ] [MAIN ] ELV: Starting full primary refresh. [2025-09-16T18:16:25.645686] [INFO ] [REMOTE ] ELV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:16:25.707699] [DEBUG] [LOCAL ] ELF: Primary data saved (38878 records). [2025-09-16T18:16:25.707871] [INFO ] [MAIN ] ELF: Full history saved. [2025-09-16T18:16:25.707973] [INFO ] [MAIN ] Update Progress: 724/2369 symbols completed. ELF took 28.6s. Estimated remaining: 2h 53m 32s. Rate: 9.5 symbols/min [2025-09-16T18:16:25.879972] [INFO ] [MAIN ] EMA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:16:25.880168] [DEBUG] [LOCAL ] EMA: No existing data file found in primary storage. [2025-09-16T18:16:25.880207] [WARN ] [MAIN ] EMA: No existing data found, performing full refresh instead. [2025-09-16T18:16:25.880335] [INFO ] [MAIN ] EMA: Wiping existing primary data before full refresh. [2025-09-16T18:16:25.880374] [INFO ] [LOCAL ] EMA: Starting deletion of primary data. [2025-09-16T18:16:25.880414] [DEBUG] [LOCAL ] EMA: No primary data file found to delete. [2025-09-16T18:16:25.880446] [INFO ] [MAIN ] EMA: Starting full primary refresh. [2025-09-16T18:16:25.963462] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-09-16T18:16:25.975185] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-02. [2025-09-16T18:16:26.242498] [INFO ] [REMOTE ] EMA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:16:26.269497] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-09-16T18:16:26.774897] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-06. [2025-09-16T18:16:27.142751] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-09-16T18:16:27.574896] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-07. [2025-09-16T18:16:27.866945] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-09-16T18:16:27.991898] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-09-16T18:16:28.374902] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-03. [2025-09-16T18:16:29.174906] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-06. [2025-09-16T18:16:29.974914] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-02. [2025-09-16T18:16:30.135835] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T18:16:30.160130] [DEBUG] [REMOTE ] EMA: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:16:30.160478] [WARN ] [REMOTE ] EMA: No data returned for month 2025-02 [2025-09-16T18:16:30.774918] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-03. [2025-09-16T18:16:30.966117] [DEBUG] [REMOTE ] EMA: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:16:30.966418] [WARN ] [REMOTE ] EMA: No data returned for month 2025-03 [2025-09-16T18:16:31.310102] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-06 (2.14 seconds). [2025-09-16T18:16:31.574916] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-04. [2025-09-16T18:16:31.792995] [DEBUG] [REMOTE ] EMA: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:16:31.793300] [WARN ] [REMOTE ] EMA: No data returned for month 2025-04 [2025-09-16T18:16:32.374898] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-07. [2025-09-16T18:16:33.174919] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-04. [2025-09-16T18:16:33.974949] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-08. [2025-09-16T18:16:34.245290] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T18:16:34.289658] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-08 (0.31 seconds). [2025-09-16T18:16:34.774877] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-08. [2025-09-16T18:16:34.900195] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T18:16:35.574924] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-09. [2025-09-16T18:16:35.932790] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-09 (0.36 seconds). [2025-09-16T18:16:35.933477] [INFO ] [REMOTE ] ELPC: Full fetch complete (963 records). [2025-09-16T18:16:35.933546] [DEBUG] [VALIDATOR ] ELPC: Validating primary data. [2025-09-16T18:16:35.934001] [FATAL] [VALIDATOR ] ELPC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:16:35.934096] [FATAL] [MAIN ] ELPC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:16:35.934153] [WARN ] [MAIN ] ELPC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:16:35.934221] [INFO ] [LOCAL ] ELPC: Starting deletion of primary data. [2025-09-16T18:16:35.934285] [DEBUG] [LOCAL ] ELPC: No primary data file found to delete. [2025-09-16T18:16:35.934321] [INFO ] [LOCAL ] ELPC: Starting deletion of derivative data. [2025-09-16T18:16:35.934499] [DEBUG] [LOCAL ] ELPC: No derivative data file found to delete. [2025-09-16T18:16:35.934576] [INFO ] [MAIN ] Update Progress: 725/2369 symbols completed. ELPC took 26.6s. Estimated remaining: 2h 53m 35s. Rate: 9.5 symbols/min [2025-09-16T18:16:36.110003] [INFO ] [MAIN ] EMD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:16:36.128782] [DEBUG] [LOCAL ] EMD: Loaded 14899 existing records from primary storage. [2025-09-16T18:16:36.128917] [INFO ] [MAIN ] EMD: Loaded 14899 existing records. [2025-09-16T18:16:36.307382] [WARN ] [MAIN ] EMD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:16:36.307699] [INFO ] [MAIN ] EMD: Wiping existing primary data before full refresh. [2025-09-16T18:16:36.307762] [INFO ] [LOCAL ] EMD: Starting deletion of primary data. [2025-09-16T18:16:36.308287] [DEBUG] [LOCAL ] EMD: Deleted primary data file. [2025-09-16T18:16:36.308335] [INFO ] [MAIN ] EMD: Starting full primary refresh. [2025-09-16T18:16:36.383007] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-05. [2025-09-16T18:16:36.672295] [INFO ] [REMOTE ] EMD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:16:36.707082] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T18:16:37.174914] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-09. [2025-09-16T18:16:37.974909] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-05. [2025-09-16T18:16:38.387087] [INFO ] [REMOTE ] EMA: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-09-16T18:16:38.774911] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-06. [2025-09-16T18:16:38.818604] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-05 (2.44 seconds). [2025-09-16T18:16:38.849304] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-09-16T18:16:38.855854] [INFO ] [REMOTE ] ELS: Full fetch complete (53312 records). [2025-09-16T18:16:38.855986] [DEBUG] [VALIDATOR ] ELS: Validating primary data. [2025-09-16T18:16:39.244475] [DEBUG] [VALIDATOR ] ELS: All expected trading days are present. [2025-09-16T18:16:39.244557] [DEBUG] [VALIDATOR ] ELS: Primary data validation passed. [2025-09-16T18:16:39.245272] [INFO ] [MAIN ] ELS: Fetched primary data is valid. [2025-09-16T18:16:39.245319] [DEBUG] [LOCAL ] ELS: Saving primary data. [2025-09-16T18:16:39.246452] [DEBUG] [LOCAL ] ELS: Trimmed 53312 records to 36880 (keeping 150 days of data) [2025-09-16T18:16:39.469981] [DEBUG] [LOCAL ] ELS: Primary data saved (36880 records). [2025-09-16T18:16:39.470137] [INFO ] [MAIN ] ELS: Full history saved. [2025-09-16T18:16:39.470215] [INFO ] [MAIN ] Update Progress: 726/2369 symbols completed. ELS took 23.4s. Estimated remaining: 2h 53m 22s. Rate: 9.5 symbols/min [2025-09-16T18:16:39.580132] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-06. [2025-09-16T18:16:39.645899] [INFO ] [MAIN ] EME: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:16:39.656691] [INFO ] [REMOTE ] EMA: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-09-16T18:16:39.678736] [DEBUG] [LOCAL ] EME: Loaded 22200 existing records from primary storage. [2025-09-16T18:16:39.678874] [INFO ] [MAIN ] EME: Loaded 22200 existing records. [2025-09-16T18:16:39.856381] [WARN ] [MAIN ] EME: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:16:39.856703] [INFO ] [MAIN ] EME: Wiping existing primary data before full refresh. [2025-09-16T18:16:39.856755] [INFO ] [LOCAL ] EME: Starting deletion of primary data. [2025-09-16T18:16:39.857410] [DEBUG] [LOCAL ] EME: Deleted primary data file. [2025-09-16T18:16:39.857473] [INFO ] [MAIN ] EME: Starting full primary refresh. [2025-09-16T18:16:40.208484] [INFO ] [REMOTE ] EME: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:16:40.374916] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-07. [2025-09-16T18:16:41.174910] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-02. [2025-09-16T18:16:41.199763] [INFO ] [REMOTE ] EMA: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-09-16T18:16:41.366055] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-09-16T18:16:41.974915] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-07. [2025-09-16T18:16:42.157999] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-09-16T18:16:42.774906] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-03. [2025-09-16T18:16:43.574878] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-02. [2025-09-16T18:16:44.029377] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-09-16T18:16:44.115361] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-09-16T18:16:44.374908] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-08. [2025-09-16T18:16:45.166451] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T18:16:45.174840] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-03. [2025-09-16T18:16:45.375792] [INFO ] [REMOTE ] EMA: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-09-16T18:16:45.974940] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-08. [2025-09-16T18:16:46.568132] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T18:16:46.774967] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-04. [2025-09-16T18:16:47.574875] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-09. [2025-09-16T18:16:47.683369] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-09-16T18:16:47.925505] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T18:16:48.374919] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-09. [2025-09-16T18:16:48.499659] [INFO ] [REMOTE ] EMA: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T18:16:48.501927] [INFO ] [REMOTE ] EMA: Full fetch complete (7287 records). [2025-09-16T18:16:48.502005] [DEBUG] [VALIDATOR ] EMA: Validating primary data. [2025-09-16T18:16:48.696157] [WARN ] [VALIDATOR ] EMA: Validation failed - insufficient historical data. Have 77 unique days, need 150 days [2025-09-16T18:16:48.696438] [FATAL] [MAIN ] EMA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:16:48.696517] [WARN ] [MAIN ] EMA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:16:48.696584] [INFO ] [LOCAL ] EMA: Starting deletion of primary data. [2025-09-16T18:16:48.696686] [DEBUG] [LOCAL ] EMA: No primary data file found to delete. [2025-09-16T18:16:48.696744] [INFO ] [LOCAL ] EMA: Starting deletion of derivative data. [2025-09-16T18:16:48.696804] [DEBUG] [LOCAL ] EMA: No derivative data file found to delete. [2025-09-16T18:16:48.696886] [INFO ] [MAIN ] Update Progress: 727/2369 symbols completed. EMA took 23.0s. Estimated remaining: 2h 53m 22s. Rate: 9.5 symbols/min [2025-09-16T18:16:48.865917] [INFO ] [MAIN ] EMF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:16:48.869615] [DEBUG] [LOCAL ] EMF: Loaded 2155 existing records from primary storage. [2025-09-16T18:16:48.869722] [INFO ] [MAIN ] EMF: Loaded 2155 existing records. [2025-09-16T18:16:49.046800] [WARN ] [MAIN ] EMF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:16:49.047094] [INFO ] [MAIN ] EMF: Wiping existing primary data before full refresh. [2025-09-16T18:16:49.047196] [INFO ] [LOCAL ] EMF: Starting deletion of primary data. [2025-09-16T18:16:49.047482] [DEBUG] [LOCAL ] EMF: Deleted primary data file. [2025-09-16T18:16:49.047543] [INFO ] [MAIN ] EMF: Starting full primary refresh. [2025-09-16T18:16:49.190207] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-04. [2025-09-16T18:16:49.406250] [INFO ] [REMOTE ] EMF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:16:49.974916] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-05. [2025-09-16T18:16:50.317402] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-09-16T18:16:50.323958] [INFO ] [REMOTE ] ELV: Full fetch complete (54870 records). [2025-09-16T18:16:50.324077] [DEBUG] [VALIDATOR ] ELV: Validating primary data. [2025-09-16T18:16:50.735565] [DEBUG] [VALIDATOR ] ELV: All expected trading days are present. [2025-09-16T18:16:50.735673] [DEBUG] [VALIDATOR ] ELV: Primary data validation passed. [2025-09-16T18:16:50.736495] [INFO ] [MAIN ] ELV: Fetched primary data is valid. [2025-09-16T18:16:50.736565] [DEBUG] [LOCAL ] ELV: Saving primary data. [2025-09-16T18:16:50.737858] [DEBUG] [LOCAL ] ELV: Trimmed 54870 records to 37831 (keeping 150 days of data) [2025-09-16T18:16:50.806203] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-02. [2025-09-16T18:16:50.997565] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T18:16:51.035378] [DEBUG] [LOCAL ] ELV: Primary data saved (37831 records). [2025-09-16T18:16:51.035543] [INFO ] [MAIN ] ELV: Full history saved. [2025-09-16T18:16:51.035622] [INFO ] [MAIN ] Update Progress: 728/2369 symbols completed. ELV took 26.0s. Estimated remaining: 2h 53m 7s. Rate: 9.5 symbols/min [2025-09-16T18:16:51.221707] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T18:16:51.223464] [INFO ] [MAIN ] EMN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:16:51.267052] [DEBUG] [LOCAL ] EMN: Loaded 37455 existing records from primary storage. [2025-09-16T18:16:51.267212] [INFO ] [MAIN ] EMN: Loaded 37455 existing records. [2025-09-16T18:16:51.452922] [WARN ] [MAIN ] EMN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:16:51.453151] [INFO ] [MAIN ] EMN: Wiping existing primary data before full refresh. [2025-09-16T18:16:51.453197] [INFO ] [LOCAL ] EMN: Starting deletion of primary data. [2025-09-16T18:16:51.454222] [DEBUG] [LOCAL ] EMN: Deleted primary data file. [2025-09-16T18:16:51.454280] [INFO ] [MAIN ] EMN: Starting full primary refresh. [2025-09-16T18:16:51.580056] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-06. [2025-09-16T18:16:51.828954] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T18:16:51.834105] [INFO ] [REMOTE ] EMN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:16:52.374924] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-02. [2025-09-16T18:16:52.790801] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T18:16:53.174946] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-05. [2025-09-16T18:16:53.974872] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-07. [2025-09-16T18:16:54.079388] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-09-16T18:16:54.664207] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T18:16:54.774923] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-06. [2025-09-16T18:16:55.156927] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-09-16T18:16:55.574899] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-08. [2025-09-16T18:16:56.374897] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-03. [2025-09-16T18:16:56.599619] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-09-16T18:16:56.627241] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T18:16:57.037753] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-09-16T18:16:57.174891] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-04. [2025-09-16T18:16:57.779255] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-09-16T18:16:57.974939] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-09. [2025-09-16T18:16:58.774898] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-05. [2025-09-16T18:16:58.951569] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T18:16:58.954785] [INFO ] [REMOTE ] EMD: Full fetch complete (23945 records). [2025-09-16T18:16:58.954888] [DEBUG] [VALIDATOR ] EMD: Validating primary data. [2025-09-16T18:16:59.198720] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T18:16:59.336808] [DEBUG] [VALIDATOR ] EMD: All expected trading days are present. [2025-09-16T18:16:59.336892] [DEBUG] [VALIDATOR ] EMD: Primary data validation passed. [2025-09-16T18:16:59.337349] [INFO ] [MAIN ] EMD: Fetched primary data is valid. [2025-09-16T18:16:59.337390] [DEBUG] [LOCAL ] EMD: Saving primary data. [2025-09-16T18:16:59.338118] [DEBUG] [LOCAL ] EMD: Trimmed 23945 records to 15174 (keeping 150 days of data) [2025-09-16T18:16:59.437826] [DEBUG] [LOCAL ] EMD: Primary data saved (15174 records). [2025-09-16T18:16:59.437976] [INFO ] [MAIN ] EMD: Full history saved. [2025-09-16T18:16:59.438065] [INFO ] [MAIN ] Update Progress: 729/2369 symbols completed. EMD took 23.5s. Estimated remaining: 2h 53m 5s. Rate: 9.5 symbols/min [2025-09-16T18:16:59.586865] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-07. [2025-09-16T18:16:59.619085] [INFO ] [MAIN ] EMO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:16:59.629972] [DEBUG] [LOCAL ] EMO: Loaded 9432 existing records from primary storage. [2025-09-16T18:16:59.630182] [INFO ] [MAIN ] EMO: Loaded 9432 existing records. [2025-09-16T18:16:59.808291] [WARN ] [MAIN ] EMO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:16:59.808551] [INFO ] [MAIN ] EMO: Wiping existing primary data before full refresh. [2025-09-16T18:16:59.808599] [INFO ] [LOCAL ] EMO: Starting deletion of primary data. [2025-09-16T18:16:59.809081] [DEBUG] [LOCAL ] EMO: Deleted primary data file. [2025-09-16T18:16:59.809142] [INFO ] [MAIN ] EMO: Starting full primary refresh. [2025-09-16T18:17:00.166878] [INFO ] [REMOTE ] EMO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:00.374910] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-06. [2025-09-16T18:17:00.911844] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-09-16T18:17:01.174909] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-02. [2025-09-16T18:17:01.346383] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T18:17:01.974960] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-07. [2025-09-16T18:17:02.082933] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-09-16T18:17:02.364619] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-09-16T18:17:02.774914] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-08. [2025-09-16T18:17:03.241793] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-09-16T18:17:03.574920] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-03. [2025-09-16T18:17:04.374921] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-08. [2025-09-16T18:17:04.440507] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-09-16T18:17:05.174881] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-09. [2025-09-16T18:17:05.249704] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-09-16T18:17:05.774145] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-09-16T18:17:05.775013] [INFO ] [REMOTE ] EMF: Full fetch complete (4164 records). [2025-09-16T18:17:05.775084] [DEBUG] [VALIDATOR ] EMF: Validating primary data. [2025-09-16T18:17:05.775456] [FATAL] [VALIDATOR ] EMF: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:17:05.775573] [FATAL] [MAIN ] EMF: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:17:05.775612] [WARN ] [MAIN ] EMF: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:17:05.775671] [INFO ] [LOCAL ] EMF: Starting deletion of primary data. [2025-09-16T18:17:05.775733] [DEBUG] [LOCAL ] EMF: No primary data file found to delete. [2025-09-16T18:17:05.775771] [INFO ] [LOCAL ] EMF: Starting deletion of derivative data. [2025-09-16T18:17:05.776049] [DEBUG] [LOCAL ] EMF: Deleted derivative data file. [2025-09-16T18:17:05.776128] [INFO ] [MAIN ] Update Progress: 730/2369 symbols completed. EMF took 17.1s. Estimated remaining: 2h 52m 59s. Rate: 9.5 symbols/min [2025-09-16T18:17:05.952818] [INFO ] [MAIN ] EMP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:17:05.953034] [DEBUG] [LOCAL ] EMP: No existing data file found in primary storage. [2025-09-16T18:17:05.953074] [WARN ] [MAIN ] EMP: No existing data found, performing full refresh instead. [2025-09-16T18:17:05.953203] [INFO ] [MAIN ] EMP: Wiping existing primary data before full refresh. [2025-09-16T18:17:05.953245] [INFO ] [LOCAL ] EMP: Starting deletion of primary data. [2025-09-16T18:17:05.953289] [DEBUG] [LOCAL ] EMP: No primary data file found to delete. [2025-09-16T18:17:05.953323] [INFO ] [MAIN ] EMP: Starting full primary refresh. [2025-09-16T18:17:05.983679] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-03. [2025-09-16T18:17:06.322246] [INFO ] [REMOTE ] EMP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:06.774913] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-02. [2025-09-16T18:17:07.385853] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-09-16T18:17:07.574890] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-09. [2025-09-16T18:17:07.969513] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-09-16T18:17:08.374904] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-04. [2025-09-16T18:17:08.882336] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T18:17:08.886381] [INFO ] [REMOTE ] EME: Full fetch complete (35205 records). [2025-09-16T18:17:08.886468] [DEBUG] [VALIDATOR ] EME: Validating primary data. [2025-09-16T18:17:09.180033] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-04. [2025-09-16T18:17:09.261723] [DEBUG] [VALIDATOR ] EME: All expected trading days are present. [2025-09-16T18:17:09.261819] [DEBUG] [VALIDATOR ] EME: Primary data validation passed. [2025-09-16T18:17:09.262436] [INFO ] [MAIN ] EME: Fetched primary data is valid. [2025-09-16T18:17:09.262481] [DEBUG] [LOCAL ] EME: Saving primary data. [2025-09-16T18:17:09.263464] [DEBUG] [LOCAL ] EME: Trimmed 35205 records to 22441 (keeping 150 days of data) [2025-09-16T18:17:09.409907] [DEBUG] [LOCAL ] EME: Primary data saved (22441 records). [2025-09-16T18:17:09.410234] [INFO ] [MAIN ] EME: Full history saved. [2025-09-16T18:17:09.410397] [INFO ] [MAIN ] Update Progress: 731/2369 symbols completed. EME took 29.9s. Estimated remaining: 2h 52m 47s. Rate: 9.5 symbols/min [2025-09-16T18:17:09.578422] [INFO ] [MAIN ] EMR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:17:09.617523] [DEBUG] [LOCAL ] EMR: Loaded 39273 existing records from primary storage. [2025-09-16T18:17:09.617686] [INFO ] [MAIN ] EMR: Loaded 39273 existing records. [2025-09-16T18:17:09.797066] [WARN ] [MAIN ] EMR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:17:09.797426] [INFO ] [MAIN ] EMR: Wiping existing primary data before full refresh. [2025-09-16T18:17:09.797511] [INFO ] [LOCAL ] EMR: Starting deletion of primary data. [2025-09-16T18:17:09.798528] [DEBUG] [LOCAL ] EMR: Deleted primary data file. [2025-09-16T18:17:09.798582] [INFO ] [MAIN ] EMR: Starting full primary refresh. [2025-09-16T18:17:09.981594] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-03. [2025-09-16T18:17:10.191843] [INFO ] [REMOTE ] EMR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:10.194187] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T18:17:10.212430] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T18:17:10.474310] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-09-16T18:17:10.774926] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-05. [2025-09-16T18:17:11.574900] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-04. [2025-09-16T18:17:12.127699] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T18:17:12.374938] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-02. [2025-09-16T18:17:12.540371] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-09-16T18:17:13.174948] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-05. [2025-09-16T18:17:13.974909] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-06. [2025-09-16T18:17:14.102524] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T18:17:14.145104] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T18:17:14.774907] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-05. [2025-09-16T18:17:15.355810] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-09-16T18:17:15.574947] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-06. [2025-09-16T18:17:15.793840] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-09-16T18:17:15.920773] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-09-16T18:17:16.374968] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-06. [2025-09-16T18:17:16.969205] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T18:17:17.174908] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-03. [2025-09-16T18:17:17.974929] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-07. [2025-09-16T18:17:18.645337] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-09-16T18:17:18.774938] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-07. [2025-09-16T18:17:19.018476] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T18:17:19.574906] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-08. [2025-09-16T18:17:19.786782] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-09-16T18:17:20.010588] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-09-16T18:17:20.374916] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-08. [2025-09-16T18:17:21.174917] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-09. [2025-09-16T18:17:21.300450] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T18:17:21.736976] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T18:17:21.738038] [INFO ] [REMOTE ] EMP: Full fetch complete (3181 records). [2025-09-16T18:17:21.738111] [DEBUG] [VALIDATOR ] EMP: Validating primary data. [2025-09-16T18:17:21.980132] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-04. [2025-09-16T18:17:22.143230] [DEBUG] [VALIDATOR ] EMP: All expected trading days are present. [2025-09-16T18:17:22.143322] [DEBUG] [VALIDATOR ] EMP: Primary data validation passed. [2025-09-16T18:17:22.143452] [INFO ] [MAIN ] EMP: Fetched primary data is valid. [2025-09-16T18:17:22.143499] [DEBUG] [LOCAL ] EMP: Saving primary data. [2025-09-16T18:17:22.143965] [DEBUG] [LOCAL ] EMP: Trimmed 3181 records to 2224 (keeping 150 days of data) [2025-09-16T18:17:22.158100] [DEBUG] [LOCAL ] EMP: Primary data saved (2224 records). [2025-09-16T18:17:22.158194] [INFO ] [MAIN ] EMP: Full history saved. [2025-09-16T18:17:22.158265] [INFO ] [MAIN ] Update Progress: 732/2369 symbols completed. EMP took 16.4s. Estimated remaining: 2h 52m 55s. Rate: 9.5 symbols/min [2025-09-16T18:17:22.332411] [INFO ] [MAIN ] ENB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:17:22.372579] [DEBUG] [LOCAL ] ENB: Loaded 39511 existing records from primary storage. [2025-09-16T18:17:22.372766] [INFO ] [MAIN ] ENB: Loaded 39511 existing records. [2025-09-16T18:17:22.547744] [WARN ] [MAIN ] ENB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:17:22.548004] [INFO ] [MAIN ] ENB: Wiping existing primary data before full refresh. [2025-09-16T18:17:22.548049] [INFO ] [LOCAL ] ENB: Starting deletion of primary data. [2025-09-16T18:17:22.549002] [DEBUG] [LOCAL ] ENB: Deleted primary data file. [2025-09-16T18:17:22.549054] [INFO ] [MAIN ] ENB: Starting full primary refresh. [2025-09-16T18:17:22.784209] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-07. [2025-09-16T18:17:22.903608] [INFO ] [REMOTE ] ENB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:23.495782] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T18:17:23.574907] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-05. [2025-09-16T18:17:24.374902] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-09. [2025-09-16T18:17:24.657909] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T18:17:25.174896] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-08. [2025-09-16T18:17:25.515486] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T18:17:25.518297] [INFO ] [REMOTE ] EMO: Full fetch complete (12961 records). [2025-09-16T18:17:25.518372] [DEBUG] [VALIDATOR ] EMO: Validating primary data. [2025-09-16T18:17:25.936405] [DEBUG] [VALIDATOR ] EMO: All expected trading days are present. [2025-09-16T18:17:25.936526] [DEBUG] [VALIDATOR ] EMO: Primary data validation passed. [2025-09-16T18:17:25.937088] [INFO ] [MAIN ] EMO: Fetched primary data is valid. [2025-09-16T18:17:25.937596] [DEBUG] [LOCAL ] EMO: Saving primary data. [2025-09-16T18:17:25.939188] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-05 (2.36 seconds). [2025-09-16T18:17:25.951115] [DEBUG] [LOCAL ] EMO: Trimmed 12961 records to 9509 (keeping 150 days of data) [2025-09-16T18:17:25.978911] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-02. [2025-09-16T18:17:26.016365] [DEBUG] [LOCAL ] EMO: Primary data saved (9509 records). [2025-09-16T18:17:26.016501] [INFO ] [MAIN ] EMO: Full history saved. [2025-09-16T18:17:26.016596] [INFO ] [MAIN ] Update Progress: 733/2369 symbols completed. EMO took 26.6s. Estimated remaining: 2h 52m 43s. Rate: 9.5 symbols/min [2025-09-16T18:17:26.206010] [INFO ] [MAIN ] ENFY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:17:26.206182] [DEBUG] [LOCAL ] ENFY: No existing data file found in primary storage. [2025-09-16T18:17:26.206219] [WARN ] [MAIN ] ENFY: No existing data found, performing full refresh instead. [2025-09-16T18:17:26.206378] [INFO ] [MAIN ] ENFY: Wiping existing primary data before full refresh. [2025-09-16T18:17:26.206417] [INFO ] [LOCAL ] ENFY: Starting deletion of primary data. [2025-09-16T18:17:26.206459] [DEBUG] [LOCAL ] ENFY: No primary data file found to delete. [2025-09-16T18:17:26.206494] [INFO ] [MAIN ] ENFY: Starting full primary refresh. [2025-09-16T18:17:26.572929] [INFO ] [REMOTE ] ENFY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:26.658297] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T18:17:26.774857] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-09. [2025-09-16T18:17:27.574942] [INFO ] [REMOTE ] ENFY: Fetching primary data for 2025-02. [2025-09-16T18:17:27.880427] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-09-16T18:17:28.309607] [INFO ] [REMOTE ] ENFY: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-09-16T18:17:28.374914] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-03. [2025-09-16T18:17:28.428325] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-09-16T18:17:28.434687] [INFO ] [REMOTE ] EMN: Full fetch complete (54282 records). [2025-09-16T18:17:28.434793] [DEBUG] [VALIDATOR ] EMN: Validating primary data. [2025-09-16T18:17:28.817240] [DEBUG] [VALIDATOR ] EMN: All expected trading days are present. [2025-09-16T18:17:28.817328] [DEBUG] [VALIDATOR ] EMN: Primary data validation passed. [2025-09-16T18:17:28.818008] [INFO ] [MAIN ] EMN: Fetched primary data is valid. [2025-09-16T18:17:28.818067] [DEBUG] [LOCAL ] EMN: Saving primary data. [2025-09-16T18:17:28.819142] [DEBUG] [LOCAL ] EMN: Trimmed 54282 records to 37814 (keeping 150 days of data) [2025-09-16T18:17:29.055931] [DEBUG] [LOCAL ] EMN: Primary data saved (37814 records). [2025-09-16T18:17:29.056082] [INFO ] [MAIN ] EMN: Full history saved. [2025-09-16T18:17:29.057079] [INFO ] [MAIN ] Update Progress: 734/2369 symbols completed. EMN took 38.0s. Estimated remaining: 2h 52m 29s. Rate: 9.5 symbols/min [2025-09-16T18:17:29.180036] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-06. [2025-09-16T18:17:29.231267] [INFO ] [MAIN ] ENIC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:17:29.258595] [DEBUG] [LOCAL ] ENIC: Loaded 27637 existing records from primary storage. [2025-09-16T18:17:29.258758] [INFO ] [MAIN ] ENIC: Loaded 27637 existing records. [2025-09-16T18:17:29.432275] [WARN ] [MAIN ] ENIC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:17:29.432549] [INFO ] [MAIN ] ENIC: Wiping existing primary data before full refresh. [2025-09-16T18:17:29.432596] [INFO ] [LOCAL ] ENIC: Starting deletion of primary data. [2025-09-16T18:17:29.433292] [DEBUG] [LOCAL ] ENIC: Deleted primary data file. [2025-09-16T18:17:29.433341] [INFO ] [MAIN ] ENIC: Starting full primary refresh. [2025-09-16T18:17:29.792766] [INFO ] [REMOTE ] ENIC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:29.981205] [INFO ] [REMOTE ] ENFY: Fetching primary data for 2025-03. [2025-09-16T18:17:30.002158] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T18:17:30.451841] [INFO ] [REMOTE ] ENFY: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-09-16T18:17:30.774890] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-04. [2025-09-16T18:17:31.005636] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T18:17:31.574903] [INFO ] [REMOTE ] ENFY: Fetching primary data for 2025-04. [2025-09-16T18:17:31.911520] [INFO ] [REMOTE ] ENFY: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-09-16T18:17:32.374898] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-07. [2025-09-16T18:17:32.433760] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T18:17:33.174892] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-05. [2025-09-16T18:17:33.972115] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T18:17:33.982671] [INFO ] [REMOTE ] ENFY: Fetching primary data for 2025-05. [2025-09-16T18:17:34.332169] [INFO ] [REMOTE ] ENFY: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-09-16T18:17:34.774967] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-02. [2025-09-16T18:17:35.182673] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-05 (2.01 seconds). [2025-09-16T18:17:35.574908] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-06. [2025-09-16T18:17:35.960143] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T18:17:36.374912] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-03. [2025-09-16T18:17:37.174901] [INFO ] [REMOTE ] ENFY: Fetching primary data for 2025-06. [2025-09-16T18:17:37.400721] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-09-16T18:17:37.506689] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-09-16T18:17:37.535487] [INFO ] [REMOTE ] ENFY: Incremental fetch complete for 2025-06 (0.36 seconds). [2025-09-16T18:17:37.974909] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-07. [2025-09-16T18:17:38.774937] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-04. [2025-09-16T18:17:38.876779] [DEBUG] [REMOTE ] ENIC: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-09-16T18:17:38.877113] [WARN ] [REMOTE ] ENIC: No data returned for month 2025-04 [2025-09-16T18:17:39.574859] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-08. [2025-09-16T18:17:40.135522] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-09-16T18:17:40.374909] [INFO ] [REMOTE ] ENFY: Fetching primary data for 2025-07. [2025-09-16T18:17:41.174922] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-08. [2025-09-16T18:17:41.368006] [INFO ] [REMOTE ] ENFY: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-09-16T18:17:41.539525] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-09-16T18:17:41.974906] [INFO ] [REMOTE ] ENFY: Fetching primary data for 2025-08. [2025-09-16T18:17:42.353327] [INFO ] [REMOTE ] ENFY: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-09-16T18:17:42.749206] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T18:17:42.774881] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-09. [2025-09-16T18:17:43.574905] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-05. [2025-09-16T18:17:44.374907] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-09. [2025-09-16T18:17:44.607973] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-09-16T18:17:44.614520] [INFO ] [REMOTE ] ENB: Full fetch complete (58208 records). [2025-09-16T18:17:44.614724] [DEBUG] [VALIDATOR ] ENB: Validating primary data. [2025-09-16T18:17:45.005032] [DEBUG] [VALIDATOR ] ENB: All expected trading days are present. [2025-09-16T18:17:45.005228] [DEBUG] [VALIDATOR ] ENB: Primary data validation passed. [2025-09-16T18:17:45.006101] [INFO ] [MAIN ] ENB: Fetched primary data is valid. [2025-09-16T18:17:45.006227] [DEBUG] [LOCAL ] ENB: Saving primary data. [2025-09-16T18:17:45.035295] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T18:17:45.033865] [DEBUG] [LOCAL ] ENB: Trimmed 58208 records to 39899 (keeping 150 days of data) [2025-09-16T18:17:45.177308] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-06. [2025-09-16T18:17:45.312267] [DEBUG] [LOCAL ] ENB: Primary data saved (39899 records). [2025-09-16T18:17:45.312443] [INFO ] [MAIN ] ENB: Full history saved. [2025-09-16T18:17:45.312571] [INFO ] [MAIN ] Update Progress: 735/2369 symbols completed. ENB took 23.2s. Estimated remaining: 2h 52m 45s. Rate: 9.5 symbols/min [2025-09-16T18:17:45.497563] [INFO ] [MAIN ] ENO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:17:45.500664] [DEBUG] [LOCAL ] ENO: Loaded 1228 existing records from primary storage. [2025-09-16T18:17:45.500743] [INFO ] [MAIN ] ENO: Loaded 1228 existing records. [2025-09-16T18:17:45.688262] [WARN ] [MAIN ] ENO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:17:45.688535] [INFO ] [MAIN ] ENO: Wiping existing primary data before full refresh. [2025-09-16T18:17:45.688598] [INFO ] [LOCAL ] ENO: Starting deletion of primary data. [2025-09-16T18:17:45.688817] [DEBUG] [LOCAL ] ENO: Deleted primary data file. [2025-09-16T18:17:45.688876] [INFO ] [MAIN ] ENO: Starting full primary refresh. [2025-09-16T18:17:45.974968] [INFO ] [REMOTE ] ENFY: Fetching primary data for 2025-09. [2025-09-16T18:17:46.067662] [INFO ] [REMOTE ] ENO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:46.088047] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T18:17:46.094545] [INFO ] [REMOTE ] EMR: Full fetch complete (57500 records). [2025-09-16T18:17:46.094661] [DEBUG] [VALIDATOR ] EMR: Validating primary data. [2025-09-16T18:17:46.095345] [FATAL] [VALIDATOR ] EMR: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:17:46.095427] [FATAL] [MAIN ] EMR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:17:46.095465] [WARN ] [MAIN ] EMR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:17:46.095504] [INFO ] [LOCAL ] EMR: Starting deletion of primary data. [2025-09-16T18:17:46.095561] [DEBUG] [LOCAL ] EMR: No primary data file found to delete. [2025-09-16T18:17:46.095606] [INFO ] [LOCAL ] EMR: Starting deletion of derivative data. [2025-09-16T18:17:46.095937] [DEBUG] [LOCAL ] EMR: Deleted derivative data file. [2025-09-16T18:17:46.096031] [INFO ] [MAIN ] Update Progress: 736/2369 symbols completed. EMR took 36.7s. Estimated remaining: 2h 52m 26s. Rate: 9.5 symbols/min [2025-09-16T18:17:46.264393] [INFO ] [MAIN ] ENOV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:17:46.264591] [DEBUG] [LOCAL ] ENOV: No existing data file found in primary storage. [2025-09-16T18:17:46.264633] [WARN ] [MAIN ] ENOV: No existing data found, performing full refresh instead. [2025-09-16T18:17:46.264771] [INFO ] [MAIN ] ENOV: Wiping existing primary data before full refresh. [2025-09-16T18:17:46.264811] [INFO ] [LOCAL ] ENOV: Starting deletion of primary data. [2025-09-16T18:17:46.264851] [DEBUG] [LOCAL ] ENOV: No primary data file found to delete. [2025-09-16T18:17:46.264884] [INFO ] [MAIN ] ENOV: Starting full primary refresh. [2025-09-16T18:17:46.619962] [INFO ] [REMOTE ] ENOV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:46.642276] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T18:17:46.653347] [INFO ] [REMOTE ] ENFY: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-09-16T18:17:46.654790] [INFO ] [REMOTE ] ENFY: Full fetch complete (3539 records). [2025-09-16T18:17:46.654860] [DEBUG] [VALIDATOR ] ENFY: Validating primary data. [2025-09-16T18:17:46.655261] [FATAL] [VALIDATOR ] ENFY: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:17:46.655340] [FATAL] [MAIN ] ENFY: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:17:46.655377] [WARN ] [MAIN ] ENFY: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:17:46.655421] [INFO ] [LOCAL ] ENFY: Starting deletion of primary data. [2025-09-16T18:17:46.655479] [DEBUG] [LOCAL ] ENFY: No primary data file found to delete. [2025-09-16T18:17:46.655513] [INFO ] [LOCAL ] ENFY: Starting deletion of derivative data. [2025-09-16T18:17:46.655565] [DEBUG] [LOCAL ] ENFY: No derivative data file found to delete. [2025-09-16T18:17:46.655635] [INFO ] [MAIN ] Update Progress: 737/2369 symbols completed. ENFY took 20.6s. Estimated remaining: 2h 52m 7s. Rate: 9.5 symbols/min [2025-09-16T18:17:46.780022] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-07. [2025-09-16T18:17:46.826139] [INFO ] [MAIN ] ENR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:17:46.864365] [DEBUG] [LOCAL ] ENR: Loaded 32871 existing records from primary storage. [2025-09-16T18:17:46.864508] [INFO ] [MAIN ] ENR: Loaded 32871 existing records. [2025-09-16T18:17:47.041438] [WARN ] [MAIN ] ENR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:17:47.041715] [INFO ] [MAIN ] ENR: Wiping existing primary data before full refresh. [2025-09-16T18:17:47.041954] [INFO ] [LOCAL ] ENR: Starting deletion of primary data. [2025-09-16T18:17:47.042774] [DEBUG] [LOCAL ] ENR: Deleted primary data file. [2025-09-16T18:17:47.042824] [INFO ] [MAIN ] ENR: Starting full primary refresh. [2025-09-16T18:17:47.400334] [INFO ] [REMOTE ] ENR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:47.574888] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-02. [2025-09-16T18:17:48.047186] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-09-16T18:17:48.374908] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-02. [2025-09-16T18:17:49.094029] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T18:17:49.174948] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-03. [2025-09-16T18:17:49.784117] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-09-16T18:17:49.974972] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-03. [2025-09-16T18:17:50.573978] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T18:17:50.774931] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-02. [2025-09-16T18:17:51.328493] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-09-16T18:17:51.585007] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-08. [2025-09-16T18:17:51.589804] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T18:17:52.374918] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-04. [2025-09-16T18:17:53.015635] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-09-16T18:17:53.174886] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-09. [2025-09-16T18:17:53.550895] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T18:17:53.974911] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-05. [2025-09-16T18:17:54.135894] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T18:17:54.140040] [INFO ] [REMOTE ] ENIC: Full fetch complete (33472 records). [2025-09-16T18:17:54.140115] [DEBUG] [VALIDATOR ] ENIC: Validating primary data. [2025-09-16T18:17:54.331974] [WARN ] [VALIDATOR ] ENIC: Validation failed - insufficient historical data. Have 129 unique days, need 150 days [2025-09-16T18:17:54.332435] [FATAL] [MAIN ] ENIC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:17:54.332490] [WARN ] [MAIN ] ENIC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:17:54.332537] [INFO ] [LOCAL ] ENIC: Starting deletion of primary data. [2025-09-16T18:17:54.332605] [DEBUG] [LOCAL ] ENIC: No primary data file found to delete. [2025-09-16T18:17:54.332660] [INFO ] [LOCAL ] ENIC: Starting deletion of derivative data. [2025-09-16T18:17:54.332922] [DEBUG] [LOCAL ] ENIC: Deleted derivative data file. [2025-09-16T18:17:54.333003] [INFO ] [MAIN ] Update Progress: 738/2369 symbols completed. ENIC took 25.3s. Estimated remaining: 2h 52m 4s. Rate: 9.5 symbols/min [2025-09-16T18:17:54.524090] [INFO ] [MAIN ] ENS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:17:54.544364] [DEBUG] [LOCAL ] ENS: Loaded 21455 existing records from primary storage. [2025-09-16T18:17:54.544488] [INFO ] [MAIN ] ENS: Loaded 21455 existing records. [2025-09-16T18:17:54.718227] [WARN ] [MAIN ] ENS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:17:54.718470] [INFO ] [MAIN ] ENS: Wiping existing primary data before full refresh. [2025-09-16T18:17:54.718517] [INFO ] [LOCAL ] ENS: Starting deletion of primary data. [2025-09-16T18:17:54.719077] [DEBUG] [LOCAL ] ENS: Deleted primary data file. [2025-09-16T18:17:54.719136] [INFO ] [MAIN ] ENS: Starting full primary refresh. [2025-09-16T18:17:54.780856] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-03. [2025-09-16T18:17:55.103606] [INFO ] [REMOTE ] ENS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:17:55.248834] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-09-16T18:17:55.556433] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T18:17:55.574855] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-06. [2025-09-16T18:17:56.374889] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-02. [2025-09-16T18:17:57.140564] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T18:17:57.174848] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-07. [2025-09-16T18:17:57.790616] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T18:17:57.974922] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-03. [2025-09-16T18:17:58.774915] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-04. [2025-09-16T18:17:58.813161] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T18:17:59.286581] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T18:17:59.289579] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-09-16T18:17:59.574905] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-04. [2025-09-16T18:18:00.374917] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-05. [2025-09-16T18:18:00.935195] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-09-16T18:18:00.988405] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-09-16T18:18:01.174899] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-04. [2025-09-16T18:18:01.974957] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-06. [2025-09-16T18:18:02.606775] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-09-16T18:18:02.774935] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-05. [2025-09-16T18:18:03.023414] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-09-16T18:18:03.574937] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-05. [2025-09-16T18:18:04.316002] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T18:18:04.374860] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-06. [2025-09-16T18:18:05.076878] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-09-16T18:18:05.174960] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-06. [2025-09-16T18:18:05.687064] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T18:18:05.974988] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-08. [2025-09-16T18:18:06.781744] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-07. [2025-09-16T18:18:06.787508] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-09-16T18:18:07.574914] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-07. [2025-09-16T18:18:07.769293] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T18:18:08.120360] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-09-16T18:18:08.374900] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-09. [2025-09-16T18:18:08.909542] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-09-16T18:18:09.174944] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-07. [2025-09-16T18:18:09.750312] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-09-16T18:18:09.868040] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T18:18:09.875738] [INFO ] [REMOTE ] ENR: Full fetch complete (45284 records). [2025-09-16T18:18:09.875963] [DEBUG] [VALIDATOR ] ENR: Validating primary data. [2025-09-16T18:18:09.876976] [FATAL] [VALIDATOR ] ENR: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:18:09.877103] [FATAL] [MAIN ] ENR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:18:09.877173] [WARN ] [MAIN ] ENR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:18:09.877252] [INFO ] [LOCAL ] ENR: Starting deletion of primary data. [2025-09-16T18:18:09.877344] [DEBUG] [LOCAL ] ENR: No primary data file found to delete. [2025-09-16T18:18:09.877401] [INFO ] [LOCAL ] ENR: Starting deletion of derivative data. [2025-09-16T18:18:09.877771] [DEBUG] [LOCAL ] ENR: Deleted derivative data file. [2025-09-16T18:18:09.877912] [INFO ] [MAIN ] Update Progress: 739/2369 symbols completed. ENR took 23.2s. Estimated remaining: 2h 52m 18s. Rate: 9.5 symbols/min [2025-09-16T18:18:09.983711] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-08. [2025-09-16T18:18:10.173617] [INFO ] [MAIN ] ENVA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:18:10.206759] [DEBUG] [LOCAL ] ENVA: Loaded 18662 existing records from primary storage. [2025-09-16T18:18:10.206952] [INFO ] [MAIN ] ENVA: Loaded 18662 existing records. [2025-09-16T18:18:10.439740] [WARN ] [MAIN ] ENVA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:18:10.440098] [INFO ] [MAIN ] ENVA: Wiping existing primary data before full refresh. [2025-09-16T18:18:10.440167] [INFO ] [LOCAL ] ENVA: Starting deletion of primary data. [2025-09-16T18:18:10.440916] [DEBUG] [LOCAL ] ENVA: Deleted primary data file. [2025-09-16T18:18:10.441004] [INFO ] [MAIN ] ENVA: Starting full primary refresh. [2025-09-16T18:18:10.788071] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-08. [2025-09-16T18:18:10.827863] [INFO ] [REMOTE ] ENVA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:18:11.222272] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-09-16T18:18:11.575227] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-02. [2025-09-16T18:18:11.581197] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-09-16T18:18:12.374937] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-09. [2025-09-16T18:18:13.166158] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T18:18:13.167179] [INFO ] [REMOTE ] ENO: Full fetch complete (2221 records). [2025-09-16T18:18:13.167352] [DEBUG] [VALIDATOR ] ENO: Validating primary data. [2025-09-16T18:18:13.182568] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-08. [2025-09-16T18:18:13.174182] [FATAL] [VALIDATOR ] ENO: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:18:13.187620] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T18:18:13.186397] [FATAL] [MAIN ] ENO: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:18:13.194366] [WARN ] [MAIN ] ENO: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:18:13.194524] [INFO ] [LOCAL ] ENO: Starting deletion of primary data. [2025-09-16T18:18:13.194613] [DEBUG] [LOCAL ] ENO: No primary data file found to delete. [2025-09-16T18:18:13.194710] [INFO ] [LOCAL ] ENO: Starting deletion of derivative data. [2025-09-16T18:18:13.195030] [DEBUG] [LOCAL ] ENO: Deleted derivative data file. [2025-09-16T18:18:13.195167] [INFO ] [MAIN ] Update Progress: 740/2369 symbols completed. ENO took 27.9s. Estimated remaining: 2h 52m 5s. Rate: 9.5 symbols/min [2025-09-16T18:18:13.388546] [INFO ] [MAIN ] EOD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:18:13.401270] [DEBUG] [LOCAL ] EOD: Loaded 10829 existing records from primary storage. [2025-09-16T18:18:13.401413] [INFO ] [MAIN ] EOD: Loaded 10829 existing records. [2025-09-16T18:18:13.582601] [WARN ] [MAIN ] EOD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:18:13.582870] [INFO ] [MAIN ] EOD: Wiping existing primary data before full refresh. [2025-09-16T18:18:13.582940] [INFO ] [LOCAL ] EOD: Starting deletion of primary data. [2025-09-16T18:18:13.583315] [DEBUG] [LOCAL ] EOD: Deleted primary data file. [2025-09-16T18:18:13.583373] [INFO ] [MAIN ] EOD: Starting full primary refresh. [2025-09-16T18:18:13.927459] [INFO ] [REMOTE ] EOD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:18:13.974862] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-02. [2025-09-16T18:18:14.568526] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-09-16T18:18:14.774935] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-09. [2025-09-16T18:18:14.843810] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T18:18:15.574941] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-03. [2025-09-16T18:18:15.942497] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-09-16T18:18:15.946050] [INFO ] [REMOTE ] ENS: Full fetch complete (31744 records). [2025-09-16T18:18:15.946162] [DEBUG] [VALIDATOR ] ENS: Validating primary data. [2025-09-16T18:18:15.946681] [FATAL] [VALIDATOR ] ENS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:18:15.946775] [FATAL] [MAIN ] ENS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:18:15.946818] [WARN ] [MAIN ] ENS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:18:15.946861] [INFO ] [LOCAL ] ENS: Starting deletion of primary data. [2025-09-16T18:18:15.946916] [DEBUG] [LOCAL ] ENS: No primary data file found to delete. [2025-09-16T18:18:15.946952] [INFO ] [LOCAL ] ENS: Starting deletion of derivative data. [2025-09-16T18:18:15.947164] [DEBUG] [LOCAL ] ENS: Deleted derivative data file. [2025-09-16T18:18:15.947269] [INFO ] [MAIN ] Update Progress: 741/2369 symbols completed. ENS took 21.6s. Estimated remaining: 2h 51m 50s. Rate: 9.5 symbols/min [2025-09-16T18:18:16.119504] [INFO ] [MAIN ] EOG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:18:16.128898] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-09-16T18:18:16.173139] [DEBUG] [LOCAL ] EOG: Loaded 39357 existing records from primary storage. [2025-09-16T18:18:16.173307] [INFO ] [MAIN ] EOG: Loaded 39357 existing records. [2025-09-16T18:18:16.350899] [WARN ] [MAIN ] EOG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:18:16.351148] [INFO ] [MAIN ] EOG: Wiping existing primary data before full refresh. [2025-09-16T18:18:16.351195] [INFO ] [LOCAL ] EOG: Starting deletion of primary data. [2025-09-16T18:18:16.352089] [DEBUG] [LOCAL ] EOG: Deleted primary data file. [2025-09-16T18:18:16.352139] [INFO ] [MAIN ] EOG: Starting full primary refresh. [2025-09-16T18:18:16.381167] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-09. [2025-09-16T18:18:16.705558] [INFO ] [REMOTE ] EOG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:18:17.174874] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-03. [2025-09-16T18:18:17.974917] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-02. [2025-09-16T18:18:18.001593] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T18:18:18.007100] [INFO ] [REMOTE ] ENOV: Full fetch complete (48344 records). [2025-09-16T18:18:18.007193] [DEBUG] [VALIDATOR ] ENOV: Validating primary data. [2025-09-16T18:18:18.009211] [FATAL] [VALIDATOR ] ENOV: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:18:18.009287] [FATAL] [MAIN ] ENOV: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:18:18.009325] [WARN ] [MAIN ] ENOV: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:18:18.009370] [INFO ] [LOCAL ] ENOV: Starting deletion of primary data. [2025-09-16T18:18:18.009439] [DEBUG] [LOCAL ] ENOV: No primary data file found to delete. [2025-09-16T18:18:18.009476] [INFO ] [LOCAL ] ENOV: Starting deletion of derivative data. [2025-09-16T18:18:18.009517] [DEBUG] [LOCAL ] ENOV: No derivative data file found to delete. [2025-09-16T18:18:18.009585] [INFO ] [MAIN ] Update Progress: 742/2369 symbols completed. ENOV took 31.9s. Estimated remaining: 2h 51m 35s. Rate: 9.5 symbols/min [2025-09-16T18:18:18.177692] [INFO ] [MAIN ] EOI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:18:18.187399] [DEBUG] [LOCAL ] EOI: Loaded 6869 existing records from primary storage. [2025-09-16T18:18:18.187499] [INFO ] [MAIN ] EOI: Loaded 6869 existing records. [2025-09-16T18:18:18.362764] [WARN ] [MAIN ] EOI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:18:18.363030] [INFO ] [MAIN ] EOI: Wiping existing primary data before full refresh. [2025-09-16T18:18:18.363227] [INFO ] [LOCAL ] EOI: Starting deletion of primary data. [2025-09-16T18:18:18.363583] [DEBUG] [LOCAL ] EOI: Deleted primary data file. [2025-09-16T18:18:18.363761] [INFO ] [MAIN ] EOI: Starting full primary refresh. [2025-09-16T18:18:18.503423] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T18:18:18.741055] [INFO ] [REMOTE ] EOI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:18:18.774872] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-02. [2025-09-16T18:18:19.574896] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-04. [2025-09-16T18:18:19.614943] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T18:18:19.807220] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T18:18:20.374946] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-03. [2025-09-16T18:18:20.961698] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T18:18:21.174895] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-05. [2025-09-16T18:18:21.910592] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-09-16T18:18:21.974869] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-04. [2025-09-16T18:18:22.140330] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T18:18:22.774917] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-04. [2025-09-16T18:18:23.574914] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-03. [2025-09-16T18:18:23.598569] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-09-16T18:18:23.996090] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-09-16T18:18:24.374904] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-05. [2025-09-16T18:18:24.389659] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-09-16T18:18:25.174896] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-04. [2025-09-16T18:18:25.271628] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-09-16T18:18:25.974870] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-05. [2025-09-16T18:18:26.344410] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-09-16T18:18:26.774911] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-06. [2025-09-16T18:18:27.574910] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-06. [2025-09-16T18:18:27.771705] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-09-16T18:18:27.849797] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T18:18:28.374871] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-06. [2025-09-16T18:18:28.743779] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-09-16T18:18:29.174897] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-07. [2025-09-16T18:18:29.974893] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-05. [2025-09-16T18:18:30.169678] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-09-16T18:18:30.581683] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-06 (2.21 seconds). [2025-09-16T18:18:30.774895] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-07. [2025-09-16T18:18:30.927914] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-09-16T18:18:31.574898] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-07. [2025-09-16T18:18:31.835123] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-09-16T18:18:32.374893] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-08. [2025-09-16T18:18:33.174894] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-06. [2025-09-16T18:18:33.217053] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-09-16T18:18:33.348512] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-09-16T18:18:33.941316] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T18:18:33.974852] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-07. [2025-09-16T18:18:34.729794] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-09-16T18:18:34.774856] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-08. [2025-09-16T18:18:35.530345] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-09-16T18:18:35.574855] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-09. [2025-09-16T18:18:36.374953] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-09. [2025-09-16T18:18:36.422302] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T18:18:36.423943] [INFO ] [REMOTE ] EOI: Full fetch complete (11526 records). [2025-09-16T18:18:36.424021] [DEBUG] [VALIDATOR ] EOI: Validating primary data. [2025-09-16T18:18:36.786529] [DEBUG] [VALIDATOR ] EOI: All expected trading days are present. [2025-09-16T18:18:36.786624] [DEBUG] [VALIDATOR ] EOI: Primary data validation passed. [2025-09-16T18:18:36.786926] [INFO ] [MAIN ] EOI: Fetched primary data is valid. [2025-09-16T18:18:36.786969] [DEBUG] [LOCAL ] EOI: Saving primary data. [2025-09-16T18:18:36.787448] [DEBUG] [LOCAL ] EOI: Trimmed 11526 records to 6949 (keeping 150 days of data) [2025-09-16T18:18:36.831471] [DEBUG] [LOCAL ] EOI: Primary data saved (6949 records). [2025-09-16T18:18:36.831615] [INFO ] [MAIN ] EOI: Full history saved. [2025-09-16T18:18:36.831719] [INFO ] [MAIN ] Update Progress: 743/2369 symbols completed. EOI took 18.8s. Estimated remaining: 2h 51m 56s. Rate: 9.5 symbols/min [2025-09-16T18:18:37.002522] [INFO ] [MAIN ] EOS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:18:37.013836] [DEBUG] [LOCAL ] EOS: Loaded 10119 existing records from primary storage. [2025-09-16T18:18:37.014009] [INFO ] [MAIN ] EOS: Loaded 10119 existing records. [2025-09-16T18:18:37.191927] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-08. [2025-09-16T18:18:37.196246] [WARN ] [MAIN ] EOS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:18:37.197351] [INFO ] [MAIN ] EOS: Wiping existing primary data before full refresh. [2025-09-16T18:18:37.197400] [INFO ] [LOCAL ] EOS: Starting deletion of primary data. [2025-09-16T18:18:37.197757] [DEBUG] [LOCAL ] EOS: Deleted primary data file. [2025-09-16T18:18:37.197805] [INFO ] [MAIN ] EOS: Starting full primary refresh. [2025-09-16T18:18:37.457903] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T18:18:37.493475] [INFO ] [REMOTE ] EOD: Full fetch complete (15414 records). [2025-09-16T18:18:37.493739] [DEBUG] [VALIDATOR ] EOD: Validating primary data. [2025-09-16T18:18:37.687156] [INFO ] [REMOTE ] EOS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:18:37.957612] [DEBUG] [VALIDATOR ] EOD: All expected trading days are present. [2025-09-16T18:18:37.957720] [DEBUG] [VALIDATOR ] EOD: Primary data validation passed. [2025-09-16T18:18:37.958131] [INFO ] [MAIN ] EOD: Fetched primary data is valid. [2025-09-16T18:18:37.958174] [DEBUG] [LOCAL ] EOD: Saving primary data. [2025-09-16T18:18:37.958943] [DEBUG] [LOCAL ] EOD: Trimmed 15414 records to 10922 (keeping 150 days of data) [2025-09-16T18:18:37.989182] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-02. [2025-09-16T18:18:38.029070] [DEBUG] [LOCAL ] EOD: Primary data saved (10922 records). [2025-09-16T18:18:38.029260] [INFO ] [MAIN ] EOD: Full history saved. [2025-09-16T18:18:38.029441] [INFO ] [MAIN ] Update Progress: 744/2369 symbols completed. EOD took 24.8s. Estimated remaining: 2h 51m 38s. Rate: 9.5 symbols/min [2025-09-16T18:18:38.214433] [INFO ] [MAIN ] EOT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:18:38.221665] [DEBUG] [LOCAL ] EOT: Loaded 4995 existing records from primary storage. [2025-09-16T18:18:38.221795] [INFO ] [MAIN ] EOT: Loaded 4995 existing records. [2025-09-16T18:18:38.401592] [WARN ] [MAIN ] EOT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:18:38.401871] [INFO ] [MAIN ] EOT: Wiping existing primary data before full refresh. [2025-09-16T18:18:38.401940] [INFO ] [LOCAL ] EOT: Starting deletion of primary data. [2025-09-16T18:18:38.402216] [DEBUG] [LOCAL ] EOT: Deleted primary data file. [2025-09-16T18:18:38.402263] [INFO ] [MAIN ] EOT: Starting full primary refresh. [2025-09-16T18:18:38.765789] [INFO ] [REMOTE ] EOT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:18:38.774866] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-02. [2025-09-16T18:18:38.820936] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-09-16T18:18:39.121952] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T18:18:39.369801] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-09-16T18:18:39.574926] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-09. [2025-09-16T18:18:40.374885] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-03. [2025-09-16T18:18:40.694597] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-09-16T18:18:40.697798] [INFO ] [REMOTE ] ENVA: Full fetch complete (27733 records). [2025-09-16T18:18:40.697885] [DEBUG] [VALIDATOR ] ENVA: Validating primary data. [2025-09-16T18:18:40.698408] [FATAL] [VALIDATOR ] ENVA: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:18:40.698491] [FATAL] [MAIN ] ENVA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:18:40.698529] [WARN ] [MAIN ] ENVA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:18:40.698568] [INFO ] [LOCAL ] ENVA: Starting deletion of primary data. [2025-09-16T18:18:40.698624] [DEBUG] [LOCAL ] ENVA: No primary data file found to delete. [2025-09-16T18:18:40.698698] [INFO ] [LOCAL ] ENVA: Starting deletion of derivative data. [2025-09-16T18:18:40.698924] [DEBUG] [LOCAL ] ENVA: Deleted derivative data file. [2025-09-16T18:18:40.699004] [INFO ] [MAIN ] Update Progress: 745/2369 symbols completed. ENVA took 30.8s. Estimated remaining: 2h 51m 24s. Rate: 9.5 symbols/min [2025-09-16T18:18:40.868703] [INFO ] [MAIN ] EPAC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:18:40.888273] [DEBUG] [LOCAL ] EPAC: Loaded 18981 existing records from primary storage. [2025-09-16T18:18:40.888403] [INFO ] [MAIN ] EPAC: Loaded 18981 existing records. [2025-09-16T18:18:41.061957] [WARN ] [MAIN ] EPAC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:18:41.062175] [INFO ] [MAIN ] EPAC: Wiping existing primary data before full refresh. [2025-09-16T18:18:41.062220] [INFO ] [LOCAL ] EPAC: Starting deletion of primary data. [2025-09-16T18:18:41.062794] [DEBUG] [LOCAL ] EPAC: Deleted primary data file. [2025-09-16T18:18:41.062844] [INFO ] [MAIN ] EPAC: Starting full primary refresh. [2025-09-16T18:18:41.180025] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-08. [2025-09-16T18:18:41.412628] [INFO ] [REMOTE ] EPAC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:18:41.425996] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-09-16T18:18:41.974903] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-04. [2025-09-16T18:18:42.774874] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-02. [2025-09-16T18:18:43.108971] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-09-16T18:18:43.153411] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-09-16T18:18:43.574909] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-05. [2025-09-16T18:18:43.813037] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T18:18:44.280239] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-09-16T18:18:44.374928] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-06. [2025-09-16T18:18:45.174902] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-09. [2025-09-16T18:18:45.205558] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-09-16T18:18:45.974925] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-03. [2025-09-16T18:18:46.663118] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-09-16T18:18:46.774960] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-04. [2025-09-16T18:18:46.931660] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T18:18:46.940852] [INFO ] [REMOTE ] EOG: Full fetch complete (58010 records). [2025-09-16T18:18:46.941120] [DEBUG] [VALIDATOR ] EOG: Validating primary data. [2025-09-16T18:18:47.339598] [DEBUG] [VALIDATOR ] EOG: All expected trading days are present. [2025-09-16T18:18:47.339735] [DEBUG] [VALIDATOR ] EOG: Primary data validation passed. [2025-09-16T18:18:47.340728] [INFO ] [MAIN ] EOG: Fetched primary data is valid. [2025-09-16T18:18:47.340823] [DEBUG] [LOCAL ] EOG: Saving primary data. [2025-09-16T18:18:47.343844] [DEBUG] [LOCAL ] EOG: Trimmed 58010 records to 39747 (keeping 150 days of data) [2025-09-16T18:18:47.581772] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-03. [2025-09-16T18:18:47.597861] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-09-16T18:18:47.630923] [DEBUG] [LOCAL ] EOG: Primary data saved (39747 records). [2025-09-16T18:18:47.631099] [INFO ] [MAIN ] EOG: Full history saved. [2025-09-16T18:18:47.632384] [INFO ] [MAIN ] Update Progress: 746/2369 symbols completed. EOG took 31.7s. Estimated remaining: 2h 51m 19s. Rate: 9.5 symbols/min [2025-09-16T18:18:47.806758] [INFO ] [MAIN ] EPAM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:18:47.842959] [DEBUG] [LOCAL ] EPAM: Loaded 28631 existing records from primary storage. [2025-09-16T18:18:47.843104] [INFO ] [MAIN ] EPAM: Loaded 28631 existing records. [2025-09-16T18:18:48.019871] [WARN ] [MAIN ] EPAM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:18:48.020152] [INFO ] [MAIN ] EPAM: Wiping existing primary data before full refresh. [2025-09-16T18:18:48.020199] [INFO ] [LOCAL ] EPAM: Starting deletion of primary data. [2025-09-16T18:18:48.020949] [DEBUG] [LOCAL ] EPAM: Deleted primary data file. [2025-09-16T18:18:48.021000] [INFO ] [MAIN ] EPAM: Starting full primary refresh. [2025-09-16T18:18:48.375917] [INFO ] [REMOTE ] EPAM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:18:48.376052] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-07. [2025-09-16T18:18:48.891857] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T18:18:49.174926] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-02. [2025-09-16T18:18:49.490106] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-09-16T18:18:49.974915] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-04. [2025-09-16T18:18:50.630692] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-09-16T18:18:50.774893] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-03. [2025-09-16T18:18:51.270768] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-09-16T18:18:51.574924] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-05. [2025-09-16T18:18:52.386975] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-05. [2025-09-16T18:18:52.390207] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T18:18:52.723412] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T18:18:53.104708] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-09-16T18:18:53.174863] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-06. [2025-09-16T18:18:53.810843] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-09-16T18:18:53.974920] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-07. [2025-09-16T18:18:54.772506] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T18:18:54.774838] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-08. [2025-09-16T18:18:55.558083] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-09-16T18:18:55.574855] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-09. [2025-09-16T18:18:56.334036] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-09-16T18:18:56.335580] [INFO ] [REMOTE ] EOT: Full fetch complete (7484 records). [2025-09-16T18:18:56.335667] [DEBUG] [VALIDATOR ] EOT: Validating primary data. [2025-09-16T18:18:56.379961] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-08. [2025-09-16T18:18:56.691517] [DEBUG] [VALIDATOR ] EOT: All expected trading days are present. [2025-09-16T18:18:56.691598] [DEBUG] [VALIDATOR ] EOT: Primary data validation passed. [2025-09-16T18:18:56.691872] [INFO ] [MAIN ] EOT: Fetched primary data is valid. [2025-09-16T18:18:56.691916] [DEBUG] [LOCAL ] EOT: Saving primary data. [2025-09-16T18:18:56.692321] [DEBUG] [LOCAL ] EOT: Trimmed 7484 records to 5065 (keeping 150 days of data) [2025-09-16T18:18:56.724032] [DEBUG] [LOCAL ] EOT: Primary data saved (5065 records). [2025-09-16T18:18:56.724144] [INFO ] [MAIN ] EOT: Full history saved. [2025-09-16T18:18:56.724227] [INFO ] [MAIN ] Update Progress: 747/2369 symbols completed. EOT took 18.7s. Estimated remaining: 2h 51m 18s. Rate: 9.5 symbols/min [2025-09-16T18:18:56.902008] [INFO ] [MAIN ] EPC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:18:56.936862] [DEBUG] [LOCAL ] EPC: Loaded 29617 existing records from primary storage. [2025-09-16T18:18:56.937035] [INFO ] [MAIN ] EPC: Loaded 29617 existing records. [2025-09-16T18:18:57.122217] [WARN ] [MAIN ] EPC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:18:57.122463] [INFO ] [MAIN ] EPC: Wiping existing primary data before full refresh. [2025-09-16T18:18:57.122510] [INFO ] [LOCAL ] EPC: Starting deletion of primary data. [2025-09-16T18:18:57.123278] [DEBUG] [LOCAL ] EPC: Deleted primary data file. [2025-09-16T18:18:57.123329] [INFO ] [MAIN ] EPC: Starting full primary refresh. [2025-09-16T18:18:57.179972] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-04. [2025-09-16T18:18:57.500696] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T18:18:57.504223] [INFO ] [REMOTE ] EPC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:18:57.974932] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-06. [2025-09-16T18:18:58.774924] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-02. [2025-09-16T18:18:58.999661] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T18:18:59.301483] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T18:18:59.574963] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-07. [2025-09-16T18:18:59.969438] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T18:19:00.374953] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-03. [2025-09-16T18:19:01.174918] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-09. [2025-09-16T18:19:01.232375] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-09-16T18:19:01.864809] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-09-16T18:19:01.974873] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-04. [2025-09-16T18:19:01.985478] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-09-16T18:19:01.987673] [INFO ] [REMOTE ] EOS: Full fetch complete (16870 records). [2025-09-16T18:19:01.987847] [DEBUG] [VALIDATOR ] EOS: Validating primary data. [2025-09-16T18:19:01.988301] [FATAL] [VALIDATOR ] EOS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:19:01.988383] [FATAL] [MAIN ] EOS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:19:01.988420] [WARN ] [MAIN ] EOS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:19:01.988463] [INFO ] [LOCAL ] EOS: Starting deletion of primary data. [2025-09-16T18:19:01.988519] [DEBUG] [LOCAL ] EOS: No primary data file found to delete. [2025-09-16T18:19:01.988554] [INFO ] [LOCAL ] EOS: Starting deletion of derivative data. [2025-09-16T18:19:01.988852] [DEBUG] [LOCAL ] EOS: Deleted derivative data file. [2025-09-16T18:19:01.988936] [INFO ] [MAIN ] Update Progress: 748/2369 symbols completed. EOS took 25.2s. Estimated remaining: 2h 51m 10s. Rate: 9.5 symbols/min [2025-09-16T18:19:02.165084] [INFO ] [MAIN ] EPD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:19:02.211057] [DEBUG] [LOCAL ] EPD: Loaded 39582 existing records from primary storage. [2025-09-16T18:19:02.211241] [INFO ] [MAIN ] EPD: Loaded 39582 existing records. [2025-09-16T18:19:02.399568] [WARN ] [MAIN ] EPD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:19:02.399825] [INFO ] [MAIN ] EPD: Wiping existing primary data before full refresh. [2025-09-16T18:19:02.399871] [INFO ] [LOCAL ] EPD: Starting deletion of primary data. [2025-09-16T18:19:02.400969] [DEBUG] [LOCAL ] EPD: Deleted primary data file. [2025-09-16T18:19:02.401026] [INFO ] [MAIN ] EPD: Starting full primary refresh. [2025-09-16T18:19:02.761693] [INFO ] [REMOTE ] EPD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:19:02.774857] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-02. [2025-09-16T18:19:03.580184] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-09-16T18:19:03.578871] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-08. [2025-09-16T18:19:04.374863] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-05. [2025-09-16T18:19:04.512885] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T18:19:04.843059] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-09-16T18:19:05.174917] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-09. [2025-09-16T18:19:05.908434] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T18:19:05.974876] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-06. [2025-09-16T18:19:06.267831] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T18:19:06.271205] [INFO ] [REMOTE ] EPAC: Full fetch complete (29093 records). [2025-09-16T18:19:06.271285] [DEBUG] [VALIDATOR ] EPAC: Validating primary data. [2025-09-16T18:19:06.637851] [DEBUG] [VALIDATOR ] EPAC: All expected trading days are present. [2025-09-16T18:19:06.637928] [DEBUG] [VALIDATOR ] EPAC: Primary data validation passed. [2025-09-16T18:19:06.638458] [INFO ] [MAIN ] EPAC: Fetched primary data is valid. [2025-09-16T18:19:06.638498] [DEBUG] [LOCAL ] EPAC: Saving primary data. [2025-09-16T18:19:06.639228] [DEBUG] [LOCAL ] EPAC: Trimmed 29093 records to 19167 (keeping 150 days of data) [2025-09-16T18:19:06.759195] [DEBUG] [LOCAL ] EPAC: Primary data saved (19167 records). [2025-09-16T18:19:06.759344] [INFO ] [MAIN ] EPAC: Full history saved. [2025-09-16T18:19:06.759431] [INFO ] [MAIN ] Update Progress: 749/2369 symbols completed. EPAC took 26.1s. Estimated remaining: 2h 51m 0s. Rate: 9.5 symbols/min [2025-09-16T18:19:06.780073] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-03. [2025-09-16T18:19:06.933452] [INFO ] [MAIN ] EPR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:19:06.933683] [DEBUG] [LOCAL ] EPR: No existing data file found in primary storage. [2025-09-16T18:19:06.933727] [WARN ] [MAIN ] EPR: No existing data found, performing full refresh instead. [2025-09-16T18:19:06.933880] [INFO ] [MAIN ] EPR: Wiping existing primary data before full refresh. [2025-09-16T18:19:06.933970] [INFO ] [LOCAL ] EPR: Starting deletion of primary data. [2025-09-16T18:19:06.934030] [DEBUG] [LOCAL ] EPR: No primary data file found to delete. [2025-09-16T18:19:06.934077] [INFO ] [MAIN ] EPR: Starting full primary refresh. [2025-09-16T18:19:07.300062] [INFO ] [REMOTE ] EPR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:19:07.574932] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-05. [2025-09-16T18:19:07.691567] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-09-16T18:19:08.374883] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-02. [2025-09-16T18:19:08.794480] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-09-16T18:19:09.088076] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-09-16T18:19:09.174973] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-04. [2025-09-16T18:19:09.719287] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-09-16T18:19:09.974965] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-03. [2025-09-16T18:19:10.774915] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-06. [2025-09-16T18:19:11.365360] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-09-16T18:19:11.474576] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-09-16T18:19:11.574866] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-04. [2025-09-16T18:19:12.122442] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T18:19:12.374907] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-07. [2025-09-16T18:19:13.002515] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T18:19:13.174914] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-05. [2025-09-16T18:19:13.974916] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-05. [2025-09-16T18:19:14.234366] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-09-16T18:19:14.774910] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-07. [2025-09-16T18:19:14.828338] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T18:19:15.574908] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-06. [2025-09-16T18:19:16.119749] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-09-16T18:19:16.183306] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-09-16T18:19:16.374917] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-08. [2025-09-16T18:19:16.944115] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T18:19:17.174924] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-07. [2025-09-16T18:19:17.932457] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T18:19:17.974904] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-09. [2025-09-16T18:19:18.774934] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-08. [2025-09-16T18:19:18.905722] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T18:19:19.517468] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T18:19:19.522225] [INFO ] [REMOTE ] EPAM: Full fetch complete (43054 records). [2025-09-16T18:19:19.522326] [DEBUG] [VALIDATOR ] EPAM: Validating primary data. [2025-09-16T18:19:19.522964] [FATAL] [VALIDATOR ] EPAM: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:19:19.523044] [FATAL] [MAIN ] EPAM: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:19:19.523081] [WARN ] [MAIN ] EPAM: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:19:19.523126] [INFO ] [LOCAL ] EPAM: Starting deletion of primary data. [2025-09-16T18:19:19.523184] [DEBUG] [LOCAL ] EPAM: No primary data file found to delete. [2025-09-16T18:19:19.523220] [INFO ] [LOCAL ] EPAM: Starting deletion of derivative data. [2025-09-16T18:19:19.523473] [DEBUG] [LOCAL ] EPAM: Deleted derivative data file. [2025-09-16T18:19:19.524533] [INFO ] [MAIN ] Update Progress: 750/2369 symbols completed. EPAM took 31.9s. Estimated remaining: 2h 51m 8s. Rate: 9.5 symbols/min [2025-09-16T18:19:19.579984] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-08. [2025-09-16T18:19:19.699538] [INFO ] [MAIN ] EPRT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:19:19.740891] [DEBUG] [LOCAL ] EPRT: Loaded 35533 existing records from primary storage. [2025-09-16T18:19:19.741052] [INFO ] [MAIN ] EPRT: Loaded 35533 existing records. [2025-09-16T18:19:19.917456] [WARN ] [MAIN ] EPRT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:19:19.917723] [INFO ] [MAIN ] EPRT: Wiping existing primary data before full refresh. [2025-09-16T18:19:19.917771] [INFO ] [LOCAL ] EPRT: Starting deletion of primary data. [2025-09-16T18:19:19.918530] [DEBUG] [LOCAL ] EPRT: Deleted primary data file. [2025-09-16T18:19:19.918575] [INFO ] [MAIN ] EPRT: Starting full primary refresh. [2025-09-16T18:19:20.275228] [INFO ] [REMOTE ] EPRT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:19:20.324474] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T18:19:20.374893] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-09. [2025-09-16T18:19:21.174945] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-06. [2025-09-16T18:19:21.323729] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-09-16T18:19:21.682556] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T18:19:21.689088] [INFO ] [REMOTE ] EPC: Full fetch complete (43364 records). [2025-09-16T18:19:21.689285] [DEBUG] [VALIDATOR ] EPC: Validating primary data. [2025-09-16T18:19:21.690136] [FATAL] [VALIDATOR ] EPC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:19:21.690237] [FATAL] [MAIN ] EPC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:19:21.690283] [WARN ] [MAIN ] EPC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:19:21.690342] [INFO ] [LOCAL ] EPC: Starting deletion of primary data. [2025-09-16T18:19:21.690412] [DEBUG] [LOCAL ] EPC: No primary data file found to delete. [2025-09-16T18:19:21.690460] [INFO ] [LOCAL ] EPC: Starting deletion of derivative data. [2025-09-16T18:19:21.690832] [DEBUG] [LOCAL ] EPC: Deleted derivative data file. [2025-09-16T18:19:21.690944] [INFO ] [MAIN ] Update Progress: 751/2369 symbols completed. EPC took 25.0s. Estimated remaining: 2h 50m 52s. Rate: 9.5 symbols/min [2025-09-16T18:19:21.889871] [INFO ] [MAIN ] EQBK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:19:21.900736] [DEBUG] [LOCAL ] EQBK: Loaded 7957 existing records from primary storage. [2025-09-16T18:19:21.900887] [INFO ] [MAIN ] EQBK: Loaded 7957 existing records. [2025-09-16T18:19:21.980160] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-09. [2025-09-16T18:19:22.143975] [WARN ] [MAIN ] EQBK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:19:22.144295] [INFO ] [MAIN ] EQBK: Wiping existing primary data before full refresh. [2025-09-16T18:19:22.144367] [INFO ] [LOCAL ] EQBK: Starting deletion of primary data. [2025-09-16T18:19:22.144853] [DEBUG] [LOCAL ] EQBK: Deleted primary data file. [2025-09-16T18:19:22.144929] [INFO ] [MAIN ] EQBK: Starting full primary refresh. [2025-09-16T18:19:22.692344] [INFO ] [REMOTE ] EQBK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:19:22.775007] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-02. [2025-09-16T18:19:22.874383] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T18:19:23.389033] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T18:19:23.394213] [INFO ] [REMOTE ] EPR: Full fetch complete (47927 records). [2025-09-16T18:19:23.394301] [DEBUG] [VALIDATOR ] EPR: Validating primary data. [2025-09-16T18:19:23.394856] [FATAL] [VALIDATOR ] EPR: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:19:23.394975] [FATAL] [MAIN ] EPR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:19:23.395014] [WARN ] [MAIN ] EPR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:19:23.395053] [INFO ] [LOCAL ] EPR: Starting deletion of primary data. [2025-09-16T18:19:23.395109] [DEBUG] [LOCAL ] EPR: No primary data file found to delete. [2025-09-16T18:19:23.395145] [INFO ] [LOCAL ] EPR: Starting deletion of derivative data. [2025-09-16T18:19:23.395183] [DEBUG] [LOCAL ] EPR: No derivative data file found to delete. [2025-09-16T18:19:23.395248] [INFO ] [MAIN ] Update Progress: 752/2369 symbols completed. EPR took 16.6s. Estimated remaining: 2h 50m 36s. Rate: 9.5 symbols/min [2025-09-16T18:19:23.564465] [INFO ] [MAIN ] EQH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:19:23.574919] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-07. [2025-09-16T18:19:23.602987] [DEBUG] [LOCAL ] EQH: Loaded 38767 existing records from primary storage. [2025-09-16T18:19:23.603120] [INFO ] [MAIN ] EQH: Loaded 38767 existing records. [2025-09-16T18:19:23.777496] [WARN ] [MAIN ] EQH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:19:23.777808] [INFO ] [MAIN ] EQH: Wiping existing primary data before full refresh. [2025-09-16T18:19:23.777855] [INFO ] [LOCAL ] EQH: Starting deletion of primary data. [2025-09-16T18:19:23.778639] [DEBUG] [LOCAL ] EQH: Deleted primary data file. [2025-09-16T18:19:23.778714] [INFO ] [MAIN ] EQH: Starting full primary refresh. [2025-09-16T18:19:24.126022] [INFO ] [REMOTE ] EQH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:19:24.315756] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-09-16T18:19:24.374897] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-02. [2025-09-16T18:19:25.053361] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T18:19:25.174878] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-08. [2025-09-16T18:19:25.974893] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-02. [2025-09-16T18:19:26.329256] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-09-16T18:19:26.774933] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-03. [2025-09-16T18:19:26.879423] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-09-16T18:19:27.144399] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-09-16T18:19:27.574907] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-09. [2025-09-16T18:19:28.256722] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T18:19:28.374896] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-03. [2025-09-16T18:19:28.993493] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T18:19:28.999900] [INFO ] [REMOTE ] EPD: Full fetch complete (58293 records). [2025-09-16T18:19:29.000006] [DEBUG] [VALIDATOR ] EPD: Validating primary data. [2025-09-16T18:19:29.178907] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-04. [2025-09-16T18:19:29.310579] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T18:19:29.403426] [DEBUG] [VALIDATOR ] EPD: All expected trading days are present. [2025-09-16T18:19:29.403521] [DEBUG] [VALIDATOR ] EPD: Primary data validation passed. [2025-09-16T18:19:29.404373] [INFO ] [MAIN ] EPD: Fetched primary data is valid. [2025-09-16T18:19:29.404431] [DEBUG] [LOCAL ] EPD: Saving primary data. [2025-09-16T18:19:29.406795] [DEBUG] [LOCAL ] EPD: Trimmed 58293 records to 39972 (keeping 150 days of data) [2025-09-16T18:19:29.660448] [DEBUG] [LOCAL ] EPD: Primary data saved (39972 records). [2025-09-16T18:19:29.660813] [INFO ] [MAIN ] EPD: Full history saved. [2025-09-16T18:19:29.660972] [INFO ] [MAIN ] Update Progress: 753/2369 symbols completed. EPD took 27.7s. Estimated remaining: 2h 50m 29s. Rate: 9.5 symbols/min [2025-09-16T18:19:29.837320] [INFO ] [MAIN ] EQNR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:19:29.837513] [DEBUG] [LOCAL ] EQNR: No existing data file found in primary storage. [2025-09-16T18:19:29.837551] [WARN ] [MAIN ] EQNR: No existing data found, performing full refresh instead. [2025-09-16T18:19:29.837706] [INFO ] [MAIN ] EQNR: Wiping existing primary data before full refresh. [2025-09-16T18:19:29.837752] [INFO ] [LOCAL ] EQNR: Starting deletion of primary data. [2025-09-16T18:19:29.837795] [DEBUG] [LOCAL ] EQNR: No primary data file found to delete. [2025-09-16T18:19:29.837829] [INFO ] [MAIN ] EQNR: Starting full primary refresh. [2025-09-16T18:19:29.987072] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-04. [2025-09-16T18:19:30.215509] [INFO ] [REMOTE ] EQNR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:19:30.774881] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-02. [2025-09-16T18:19:30.985773] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-09-16T18:19:31.075479] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-09-16T18:19:31.574901] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-05. [2025-09-16T18:19:32.374862] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-05. [2025-09-16T18:19:32.861301] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-02 (2.09 seconds). [2025-09-16T18:19:32.892013] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-09-16T18:19:33.174931] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-03. [2025-09-16T18:19:33.289752] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-09-16T18:19:33.974936] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-06. [2025-09-16T18:19:34.774935] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-06. [2025-09-16T18:19:35.079301] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-09-16T18:19:35.185401] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T18:19:35.553196] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T18:19:35.574913] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-07. [2025-09-16T18:19:36.269812] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-09-16T18:19:36.374881] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-08. [2025-09-16T18:19:37.174901] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-04. [2025-09-16T18:19:37.357771] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-09-16T18:19:37.974942] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-09. [2025-09-16T18:19:38.774911] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-03. [2025-09-16T18:19:39.057669] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T18:19:39.081120] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T18:19:39.083048] [INFO ] [REMOTE ] EQBK: Full fetch complete (11459 records). [2025-09-16T18:19:39.083117] [DEBUG] [VALIDATOR ] EQBK: Validating primary data. [2025-09-16T18:19:39.455028] [DEBUG] [VALIDATOR ] EQBK: All expected trading days are present. [2025-09-16T18:19:39.455109] [DEBUG] [VALIDATOR ] EQBK: Primary data validation passed. [2025-09-16T18:19:39.455387] [INFO ] [MAIN ] EQBK: Fetched primary data is valid. [2025-09-16T18:19:39.455437] [DEBUG] [LOCAL ] EQBK: Saving primary data. [2025-09-16T18:19:39.455961] [DEBUG] [LOCAL ] EQBK: Trimmed 11459 records to 8045 (keeping 150 days of data) [2025-09-16T18:19:39.508041] [DEBUG] [LOCAL ] EQBK: Primary data saved (8045 records). [2025-09-16T18:19:39.508206] [INFO ] [MAIN ] EQBK: Full history saved. [2025-09-16T18:19:39.508329] [INFO ] [MAIN ] Update Progress: 754/2369 symbols completed. EQBK took 17.8s. Estimated remaining: 2h 50m 31s. Rate: 9.5 symbols/min [2025-09-16T18:19:39.580065] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-05. [2025-09-16T18:19:39.692075] [INFO ] [MAIN ] EQR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:19:39.731118] [DEBUG] [LOCAL ] EQR: Loaded 37964 existing records from primary storage. [2025-09-16T18:19:39.731282] [INFO ] [MAIN ] EQR: Loaded 37964 existing records. [2025-09-16T18:19:39.921296] [WARN ] [MAIN ] EQR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:19:39.921600] [INFO ] [MAIN ] EQR: Wiping existing primary data before full refresh. [2025-09-16T18:19:39.921683] [INFO ] [LOCAL ] EQR: Starting deletion of primary data. [2025-09-16T18:19:39.923274] [DEBUG] [LOCAL ] EQR: Deleted primary data file. [2025-09-16T18:19:39.923368] [INFO ] [MAIN ] EQR: Starting full primary refresh. [2025-09-16T18:19:40.303222] [INFO ] [REMOTE ] EQR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:19:40.374915] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-02. [2025-09-16T18:19:40.481855] [DEBUG] [REMOTE ] EQR: Error fetching primary data for 2025-02: 'API call returned an empty response.' [2025-09-16T18:19:40.482271] [WARN ] [REMOTE ] EQR: No data returned for month 2025-02 [2025-09-16T18:19:40.595844] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-09-16T18:19:41.174909] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-04. [2025-09-16T18:19:41.509738] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-09-16T18:19:41.974916] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-03. [2025-09-16T18:19:42.774905] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-06. [2025-09-16T18:19:43.024477] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-09-16T18:19:43.336314] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T18:19:43.574937] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-05. [2025-09-16T18:19:44.369041] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T18:19:44.379175] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-04. [2025-09-16T18:19:45.174900] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-07. [2025-09-16T18:19:45.211453] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T18:19:45.974917] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-06. [2025-09-16T18:19:46.147977] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T18:19:46.774989] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-05. [2025-09-16T18:19:46.808176] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-09-16T18:19:47.574883] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-08. [2025-09-16T18:19:47.606369] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T18:19:48.374938] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-07. [2025-09-16T18:19:48.532717] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T18:19:49.174942] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-07. [2025-09-16T18:19:49.388043] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-09-16T18:19:49.974925] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-09. [2025-09-16T18:19:50.337944] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-09-16T18:19:50.774929] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-08. [2025-09-16T18:19:50.808334] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-09-16T18:19:51.156688] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T18:19:51.163000] [INFO ] [REMOTE ] EQH: Full fetch complete (57287 records). [2025-09-16T18:19:51.163151] [DEBUG] [VALIDATOR ] EQH: Validating primary data. [2025-09-16T18:19:51.543464] [DEBUG] [VALIDATOR ] EQH: All expected trading days are present. [2025-09-16T18:19:51.543549] [DEBUG] [VALIDATOR ] EQH: Primary data validation passed. [2025-09-16T18:19:51.544310] [INFO ] [MAIN ] EQH: Fetched primary data is valid. [2025-09-16T18:19:51.544379] [DEBUG] [LOCAL ] EQH: Saving primary data. [2025-09-16T18:19:51.545735] [DEBUG] [LOCAL ] EQH: Trimmed 57287 records to 39156 (keeping 150 days of data) [2025-09-16T18:19:51.603488] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-08. [2025-09-16T18:19:51.793054] [DEBUG] [LOCAL ] EQH: Primary data saved (39156 records). [2025-09-16T18:19:51.793251] [INFO ] [MAIN ] EQH: Full history saved. [2025-09-16T18:19:51.793347] [INFO ] [MAIN ] Update Progress: 755/2369 symbols completed. EQH took 28.4s. Estimated remaining: 2h 50m 37s. Rate: 9.5 symbols/min [2025-09-16T18:19:51.968215] [INFO ] [MAIN ] EQS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:19:51.968380] [DEBUG] [LOCAL ] EQS: No existing data file found in primary storage. [2025-09-16T18:19:51.968418] [WARN ] [MAIN ] EQS: No existing data found, performing full refresh instead. [2025-09-16T18:19:51.968552] [INFO ] [MAIN ] EQS: Wiping existing primary data before full refresh. [2025-09-16T18:19:51.968600] [INFO ] [LOCAL ] EQS: Starting deletion of primary data. [2025-09-16T18:19:51.968671] [DEBUG] [LOCAL ] EQS: No primary data file found to delete. [2025-09-16T18:19:51.968708] [INFO ] [MAIN ] EQS: Starting full primary refresh. [2025-09-16T18:19:52.326252] [INFO ] [REMOTE ] EQS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:19:52.374917] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-06. [2025-09-16T18:19:52.637536] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-09-16T18:19:53.174910] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-02. [2025-09-16T18:19:53.264713] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T18:19:53.565524] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-09-16T18:19:53.974879] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-03. [2025-09-16T18:19:54.131499] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T18:19:54.309447] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-09-16T18:19:54.774913] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-04. [2025-09-16T18:19:55.017831] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-04 (0.24 seconds). [2025-09-16T18:19:55.574904] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-09. [2025-09-16T18:19:56.374895] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-07. [2025-09-16T18:19:57.181009] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-05. [2025-09-16T18:19:57.182767] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T18:19:57.188875] [INFO ] [REMOTE ] EPRT: Full fetch complete (52903 records). [2025-09-16T18:19:57.188965] [DEBUG] [VALIDATOR ] EPRT: Validating primary data. [2025-09-16T18:19:57.574510] [DEBUG] [VALIDATOR ] EPRT: All expected trading days are present. [2025-09-16T18:19:57.574598] [DEBUG] [VALIDATOR ] EPRT: Primary data validation passed. [2025-09-16T18:19:57.575534] [INFO ] [MAIN ] EPRT: Fetched primary data is valid. [2025-09-16T18:19:57.575595] [DEBUG] [LOCAL ] EPRT: Saving primary data. [2025-09-16T18:19:57.576857] [DEBUG] [LOCAL ] EPRT: Trimmed 52903 records to 35910 (keeping 150 days of data) [2025-09-16T18:19:57.644384] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-09-16T18:19:57.796939] [DEBUG] [LOCAL ] EPRT: Primary data saved (35910 records). [2025-09-16T18:19:57.797079] [INFO ] [MAIN ] EPRT: Full history saved. [2025-09-16T18:19:57.797711] [INFO ] [MAIN ] Update Progress: 756/2369 symbols completed. EPRT took 38.3s. Estimated remaining: 2h 50m 30s. Rate: 9.5 symbols/min [2025-09-16T18:19:57.965751] [INFO ] [MAIN ] EQT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:19:57.974923] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-06. [2025-09-16T18:19:58.005619] [DEBUG] [LOCAL ] EQT: Loaded 39645 existing records from primary storage. [2025-09-16T18:19:58.005794] [INFO ] [MAIN ] EQT: Loaded 39645 existing records. [2025-09-16T18:19:58.188082] [WARN ] [MAIN ] EQT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:19:58.188547] [INFO ] [MAIN ] EQT: Wiping existing primary data before full refresh. [2025-09-16T18:19:58.188601] [INFO ] [LOCAL ] EQT: Starting deletion of primary data. [2025-09-16T18:19:58.189429] [DEBUG] [LOCAL ] EQT: Deleted primary data file. [2025-09-16T18:19:58.189476] [INFO ] [MAIN ] EQT: Starting full primary refresh. [2025-09-16T18:19:58.503781] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-09-16T18:19:58.511277] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-09-16T18:19:58.595740] [INFO ] [REMOTE ] EQT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:19:58.774922] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-02. [2025-09-16T18:19:59.574947] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-09. [2025-09-16T18:20:00.374972] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-07. [2025-09-16T18:20:00.728498] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-09-16T18:20:00.873226] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-09-16T18:20:01.174887] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-03. [2025-09-16T18:20:01.559308] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-09-16T18:20:01.566733] [INFO ] [REMOTE ] EQNR: Full fetch complete (57717 records). [2025-09-16T18:20:01.566932] [DEBUG] [VALIDATOR ] EQNR: Validating primary data. [2025-09-16T18:20:01.967083] [DEBUG] [VALIDATOR ] EQNR: All expected trading days are present. [2025-09-16T18:20:01.967178] [DEBUG] [VALIDATOR ] EQNR: Primary data validation passed. [2025-09-16T18:20:01.967930] [INFO ] [MAIN ] EQNR: Fetched primary data is valid. [2025-09-16T18:20:01.967978] [DEBUG] [LOCAL ] EQNR: Saving primary data. [2025-09-16T18:20:01.969241] [DEBUG] [LOCAL ] EQNR: Trimmed 57717 records to 39521 (keeping 150 days of data) [2025-09-16T18:20:02.026541] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-08. [2025-09-16T18:20:02.223288] [DEBUG] [LOCAL ] EQNR: Primary data saved (39521 records). [2025-09-16T18:20:02.223453] [INFO ] [MAIN ] EQNR: Full history saved. [2025-09-16T18:20:02.223553] [INFO ] [MAIN ] Update Progress: 757/2369 symbols completed. EQNR took 32.6s. Estimated remaining: 2h 50m 20s. Rate: 9.5 symbols/min [2025-09-16T18:20:02.732753] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-09-16T18:20:02.774860] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-09. [2025-09-16T18:20:03.170758] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-09-16T18:20:03.501868] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T18:20:03.503147] [INFO ] [REMOTE ] EQS: Full fetch complete (1856 records). [2025-09-16T18:20:03.503216] [DEBUG] [VALIDATOR ] EQS: Validating primary data. [2025-09-16T18:20:03.580122] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-04. [2025-09-16T18:20:03.686871] [WARN ] [VALIDATOR ] EQS: Validation failed - insufficient historical data. Have 139 unique days, need 150 days [2025-09-16T18:20:03.687107] [FATAL] [MAIN ] EQS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:20:03.687178] [WARN ] [MAIN ] EQS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:20:03.687245] [INFO ] [LOCAL ] EQS: Starting deletion of primary data. [2025-09-16T18:20:03.687315] [DEBUG] [LOCAL ] EQS: No primary data file found to delete. [2025-09-16T18:20:03.687363] [INFO ] [LOCAL ] EQS: Starting deletion of derivative data. [2025-09-16T18:20:03.687406] [DEBUG] [LOCAL ] EQS: No derivative data file found to delete. [2025-09-16T18:20:03.687467] [INFO ] [MAIN ] Update Progress: 758/2369 symbols completed. EQS took 11.9s. Estimated remaining: 2h 50m 3s. Rate: 9.5 symbols/min [2025-09-16T18:20:04.374928] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-08. [2025-09-16T18:20:05.700108] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-09-16T18:20:05.711553] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-05. [2025-09-16T18:20:06.262484] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-09-16T18:20:06.273463] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-09. [2025-09-16T18:20:07.694087] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-09-16T18:20:07.705450] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-06. [2025-09-16T18:20:07.853447] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-09-16T18:20:07.859622] [INFO ] [REMOTE ] EQR: Full fetch complete (50978 records). [2025-09-16T18:20:07.859744] [DEBUG] [VALIDATOR ] EQR: Validating primary data. [2025-09-16T18:20:08.053530] [WARN ] [VALIDATOR ] EQR: Validation failed - insufficient historical data. Have 137 unique days, need 150 days [2025-09-16T18:20:08.054189] [FATAL] [MAIN ] EQR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:20:08.054247] [WARN ] [MAIN ] EQR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:20:08.054295] [INFO ] [LOCAL ] EQR: Starting deletion of primary data. [2025-09-16T18:20:08.054354] [DEBUG] [LOCAL ] EQR: No primary data file found to delete. [2025-09-16T18:20:08.054390] [INFO ] [LOCAL ] EQR: Starting deletion of derivative data. [2025-09-16T18:20:08.054698] [DEBUG] [LOCAL ] EQR: Deleted derivative data file. [2025-09-16T18:20:08.054784] [INFO ] [MAIN ] Update Progress: 759/2369 symbols completed. EQR took 28.5s. Estimated remaining: 2h 49m 52s. Rate: 9.5 symbols/min [2025-09-16T18:20:09.688485] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-09-16T18:20:09.699281] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-07. [2025-09-16T18:20:11.823341] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-09-16T18:20:11.835072] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-08. [2025-09-16T18:20:13.856149] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-09-16T18:20:13.867713] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-09. [2025-09-16T18:20:15.624020] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T18:20:15.634391] [INFO ] [REMOTE ] EQT: Full fetch complete (58361 records). [2025-09-16T18:20:15.634694] [DEBUG] [VALIDATOR ] EQT: Validating primary data. [2025-09-16T18:20:16.231682] [DEBUG] [VALIDATOR ] EQT: All expected trading days are present. [2025-09-16T18:20:16.231796] [DEBUG] [VALIDATOR ] EQT: Primary data validation passed. [2025-09-16T18:20:16.232553] [INFO ] [MAIN ] EQT: Fetched primary data is valid. [2025-09-16T18:20:16.232599] [DEBUG] [LOCAL ] EQT: Saving primary data. [2025-09-16T18:20:16.234043] [DEBUG] [LOCAL ] EQT: Trimmed 58361 records to 40035 (keeping 150 days of data) [2025-09-16T18:20:16.509594] [DEBUG] [LOCAL ] EQT: Primary data saved (40035 records). [2025-09-16T18:20:16.509780] [INFO ] [MAIN ] EQT: Full history saved. [2025-09-16T18:20:16.510862] [INFO ] [MAIN ] Update Progress: 760/2369 symbols completed. EQT took 18.7s. Estimated remaining: 2h 49m 51s. Rate: 9.5 symbols/min [2025-09-16T18:20:17.137188] [INFO ] [MAIN ] ERO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:17.116088] [INFO ] [MAIN ] EQV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:17.208877] [INFO ] [MAIN ] ERJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:17.213525] [INFO ] [MAIN ] ES: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:17.217113] [DEBUG] [LOCAL ] EQV: Loaded 2157 existing records from primary storage. [2025-09-16T18:20:17.218799] [INFO ] [MAIN ] EQV: Loaded 2157 existing records. [2025-09-16T18:20:17.447564] [WARN ] [MAIN ] EQV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:20:17.448466] [DEBUG] [LOCAL ] ERO: Loaded 29843 existing records from primary storage. [2025-09-16T18:20:17.459118] [INFO ] [MAIN ] EQV: Wiping existing primary data before full refresh. [2025-09-16T18:20:17.459354] [INFO ] [MAIN ] ERO: Loaded 29843 existing records. [2025-09-16T18:20:17.459294] [INFO ] [LOCAL ] EQV: Starting deletion of primary data. [2025-09-16T18:20:17.518971] [DEBUG] [LOCAL ] ES: Loaded 38782 existing records from primary storage. [2025-09-16T18:20:17.501903] [DEBUG] [LOCAL ] EQV: Deleted primary data file. [2025-09-16T18:20:17.513222] [DEBUG] [LOCAL ] ERJ: Loaded 37326 existing records from primary storage. [2025-09-16T18:20:17.519318] [INFO ] [MAIN ] ERJ: Loaded 37326 existing records. [2025-09-16T18:20:17.519386] [INFO ] [MAIN ] ES: Loaded 38782 existing records. [2025-09-16T18:20:17.600113] [INFO ] [MAIN ] EQV: Starting full primary refresh. [2025-09-16T18:20:18.174077] [WARN ] [MAIN ] ES: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:20:18.174185] [WARN ] [MAIN ] ERO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:20:18.245479] [INFO ] [MAIN ] ES: Wiping existing primary data before full refresh. [2025-09-16T18:20:18.253281] [WARN ] [MAIN ] ERJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:20:18.255539] [INFO ] [LOCAL ] ES: Starting deletion of primary data. [2025-09-16T18:20:18.253604] [INFO ] [MAIN ] ERO: Wiping existing primary data before full refresh. [2025-09-16T18:20:18.279254] [INFO ] [LOCAL ] ERO: Starting deletion of primary data. [2025-09-16T18:20:18.305220] [DEBUG] [LOCAL ] ES: Deleted primary data file. [2025-09-16T18:20:18.305443] [INFO ] [MAIN ] ERJ: Wiping existing primary data before full refresh. [2025-09-16T18:20:18.344735] [INFO ] [LOCAL ] ERJ: Starting deletion of primary data. [2025-09-16T18:20:18.339432] [DEBUG] [LOCAL ] ERO: Deleted primary data file. [2025-09-16T18:20:18.344817] [INFO ] [MAIN ] ES: Starting full primary refresh. [2025-09-16T18:20:18.366578] [DEBUG] [LOCAL ] ERJ: Deleted primary data file. [2025-09-16T18:20:18.360486] [INFO ] [MAIN ] ERO: Starting full primary refresh. [2025-09-16T18:20:18.443764] [INFO ] [MAIN ] ERJ: Starting full primary refresh. [2025-09-16T18:20:18.707264] [INFO ] [REMOTE ] EQV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:18.734575] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-02. [2025-09-16T18:20:19.495167] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T18:20:19.518954] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-03. [2025-09-16T18:20:19.527576] [INFO ] [REMOTE ] ES: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:19.589797] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-02. [2025-09-16T18:20:19.638081] [INFO ] [REMOTE ] ERJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:19.659708] [INFO ] [REMOTE ] ERO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:19.660268] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-02. [2025-09-16T18:20:19.660068] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-02. [2025-09-16T18:20:19.961118] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-09-16T18:20:19.962537] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-04. [2025-09-16T18:20:20.287958] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-09-16T18:20:20.289275] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-05. [2025-09-16T18:20:20.628606] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-05 (0.34 seconds). [2025-09-16T18:20:20.630046] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-06. [2025-09-16T18:20:20.929803] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T18:20:20.939150] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-03. [2025-09-16T18:20:20.989937] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-06 (0.36 seconds). [2025-09-16T18:20:20.991387] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-07. [2025-09-16T18:20:21.317744] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-09-16T18:20:21.318964] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-08. [2025-09-16T18:20:21.547069] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-09-16T18:20:21.555089] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-03. [2025-09-16T18:20:21.634475] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-09-16T18:20:21.636034] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-09. [2025-09-16T18:20:22.141199] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-09-16T18:20:22.142094] [INFO ] [REMOTE ] EQV: Full fetch complete (3340 records). [2025-09-16T18:20:22.142164] [DEBUG] [VALIDATOR ] EQV: Validating primary data. [2025-09-16T18:20:22.142540] [FATAL] [VALIDATOR ] EQV: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:20:22.142628] [FATAL] [MAIN ] EQV: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:20:22.142713] [WARN ] [MAIN ] EQV: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:20:22.142767] [INFO ] [LOCAL ] EQV: Starting deletion of primary data. [2025-09-16T18:20:22.142825] [DEBUG] [LOCAL ] EQV: No primary data file found to delete. [2025-09-16T18:20:22.142862] [INFO ] [LOCAL ] EQV: Starting deletion of derivative data. [2025-09-16T18:20:22.143185] [DEBUG] [LOCAL ] EQV: Deleted derivative data file. [2025-09-16T18:20:22.143269] [INFO ] [MAIN ] Update Progress: 761/2369 symbols completed. EQV took 5.6s. Estimated remaining: 2h 49m 43s. Rate: 9.5 symbols/min [2025-09-16T18:20:22.369212] [INFO ] [MAIN ] ESAB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:22.389075] [DEBUG] [LOCAL ] ESAB: Loaded 18188 existing records from primary storage. [2025-09-16T18:20:22.389331] [INFO ] [MAIN ] ESAB: Loaded 18188 existing records. [2025-09-16T18:20:22.539655] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-02 (2.95 seconds). [2025-09-16T18:20:22.584239] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-03. [2025-09-16T18:20:22.619213] [WARN ] [MAIN ] ESAB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:20:22.619447] [INFO ] [MAIN ] ESAB: Wiping existing primary data before full refresh. [2025-09-16T18:20:22.619502] [INFO ] [LOCAL ] ESAB: Starting deletion of primary data. [2025-09-16T18:20:22.620013] [DEBUG] [LOCAL ] ESAB: Deleted primary data file. [2025-09-16T18:20:22.620063] [INFO ] [MAIN ] ESAB: Starting full primary refresh. [2025-09-16T18:20:22.996980] [INFO ] [REMOTE ] ESAB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:22.997433] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-02. [2025-09-16T18:20:23.033107] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-09-16T18:20:23.044441] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-04. [2025-09-16T18:20:23.321039] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T18:20:23.330194] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-04. [2025-09-16T18:20:24.335841] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-09-16T18:20:24.340878] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-03. [2025-09-16T18:20:24.370977] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-09-16T18:20:24.382431] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-04. [2025-09-16T18:20:24.475673] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T18:20:24.514004] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T18:20:25.174899] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-05. [2025-09-16T18:20:25.301008] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-09-16T18:20:25.976183] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-05. [2025-09-16T18:20:25.982230] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T18:20:26.315501] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T18:20:26.774893] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-06. [2025-09-16T18:20:27.378160] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T18:20:27.574895] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-05. [2025-09-16T18:20:27.751125] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T18:20:28.374899] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-06. [2025-09-16T18:20:29.021732] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T18:20:29.174902] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-07. [2025-09-16T18:20:29.614974] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T18:20:29.974901] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-07. [2025-09-16T18:20:30.436114] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T18:20:30.774905] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-08. [2025-09-16T18:20:31.574896] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-04. [2025-09-16T18:20:31.626877] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T18:20:32.062611] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T18:20:32.374878] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-09. [2025-09-16T18:20:32.469572] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-09-16T18:20:33.174902] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-08. [2025-09-16T18:20:33.506134] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T18:20:33.511619] [INFO ] [REMOTE ] ERO: Full fetch complete (44885 records). [2025-09-16T18:20:33.511770] [DEBUG] [VALIDATOR ] ERO: Validating primary data. [2025-09-16T18:20:33.900750] [DEBUG] [VALIDATOR ] ERO: All expected trading days are present. [2025-09-16T18:20:33.900858] [DEBUG] [VALIDATOR ] ERO: Primary data validation passed. [2025-09-16T18:20:33.901555] [INFO ] [MAIN ] ERO: Fetched primary data is valid. [2025-09-16T18:20:33.901608] [DEBUG] [LOCAL ] ERO: Saving primary data. [2025-09-16T18:20:33.902703] [DEBUG] [LOCAL ] ERO: Trimmed 44885 records to 30125 (keeping 150 days of data) [2025-09-16T18:20:33.990525] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-06. [2025-09-16T18:20:34.097385] [DEBUG] [LOCAL ] ERO: Primary data saved (30125 records). [2025-09-16T18:20:34.097575] [INFO ] [MAIN ] ERO: Full history saved. [2025-09-16T18:20:34.097715] [INFO ] [MAIN ] Update Progress: 762/2369 symbols completed. ERO took 17.6s. Estimated remaining: 2h 49m 48s. Rate: 9.5 symbols/min [2025-09-16T18:20:34.283723] [INFO ] [MAIN ] ESE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:34.303140] [DEBUG] [LOCAL ] ESE: Loaded 12123 existing records from primary storage. [2025-09-16T18:20:34.303298] [INFO ] [MAIN ] ESE: Loaded 12123 existing records. [2025-09-16T18:20:34.486592] [WARN ] [MAIN ] ESE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:20:34.486961] [INFO ] [MAIN ] ESE: Wiping existing primary data before full refresh. [2025-09-16T18:20:34.487034] [INFO ] [LOCAL ] ESE: Starting deletion of primary data. [2025-09-16T18:20:34.487598] [DEBUG] [LOCAL ] ESE: Deleted primary data file. [2025-09-16T18:20:34.487774] [INFO ] [MAIN ] ESE: Starting full primary refresh. [2025-09-16T18:20:34.776143] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-05. [2025-09-16T18:20:34.826699] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T18:20:34.894244] [INFO ] [REMOTE ] ESE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:35.276341] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T18:20:35.574881] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-07. [2025-09-16T18:20:35.775008] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T18:20:36.374953] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-02. [2025-09-16T18:20:37.174894] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-09. [2025-09-16T18:20:37.464102] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T18:20:37.484491] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-09-16T18:20:37.974916] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-08. [2025-09-16T18:20:38.467560] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T18:20:38.473578] [INFO ] [REMOTE ] ERJ: Full fetch complete (55378 records). [2025-09-16T18:20:38.473756] [DEBUG] [VALIDATOR ] ERJ: Validating primary data. [2025-09-16T18:20:38.780155] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-06. [2025-09-16T18:20:38.864672] [DEBUG] [VALIDATOR ] ERJ: All expected trading days are present. [2025-09-16T18:20:38.864784] [DEBUG] [VALIDATOR ] ERJ: Primary data validation passed. [2025-09-16T18:20:38.865851] [INFO ] [MAIN ] ERJ: Fetched primary data is valid. [2025-09-16T18:20:38.865922] [DEBUG] [LOCAL ] ERJ: Saving primary data. [2025-09-16T18:20:38.867580] [DEBUG] [LOCAL ] ERJ: Trimmed 55378 records to 37655 (keeping 150 days of data) [2025-09-16T18:20:39.100991] [DEBUG] [LOCAL ] ERJ: Primary data saved (37655 records). [2025-09-16T18:20:39.101152] [INFO ] [MAIN ] ERJ: Full history saved. [2025-09-16T18:20:39.101282] [INFO ] [MAIN ] Update Progress: 763/2369 symbols completed. ERJ took 22.6s. Estimated remaining: 2h 49m 39s. Rate: 9.5 symbols/min [2025-09-16T18:20:39.274266] [INFO ] [MAIN ] ESI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:39.314329] [DEBUG] [LOCAL ] ESI: Loaded 37061 existing records from primary storage. [2025-09-16T18:20:39.314504] [INFO ] [MAIN ] ESI: Loaded 37061 existing records. [2025-09-16T18:20:39.501029] [WARN ] [MAIN ] ESI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:20:39.501451] [INFO ] [MAIN ] ESI: Wiping existing primary data before full refresh. [2025-09-16T18:20:39.501511] [INFO ] [LOCAL ] ESI: Starting deletion of primary data. [2025-09-16T18:20:39.502362] [DEBUG] [LOCAL ] ESI: Deleted primary data file. [2025-09-16T18:20:39.502412] [INFO ] [MAIN ] ESI: Starting full primary refresh. [2025-09-16T18:20:39.582422] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-03. [2025-09-16T18:20:39.772409] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-09-16T18:20:39.912057] [INFO ] [REMOTE ] ESI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:39.920402] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:20:40.374949] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-07. [2025-09-16T18:20:40.491982] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-09-16T18:20:41.175068] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-02. [2025-09-16T18:20:41.175863] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T18:20:41.974915] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-04. [2025-09-16T18:20:42.658412] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T18:20:42.774967] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-09. [2025-09-16T18:20:42.984061] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-09-16T18:20:43.574926] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-03. [2025-09-16T18:20:44.374966] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-05. [2025-09-16T18:20:44.531592] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T18:20:44.540619] [INFO ] [REMOTE ] ES: Full fetch complete (57162 records). [2025-09-16T18:20:44.540820] [DEBUG] [VALIDATOR ] ES: Validating primary data. [2025-09-16T18:20:44.932386] [DEBUG] [VALIDATOR ] ES: All expected trading days are present. [2025-09-16T18:20:44.932483] [DEBUG] [VALIDATOR ] ES: Primary data validation passed. [2025-09-16T18:20:44.933310] [INFO ] [MAIN ] ES: Fetched primary data is valid. [2025-09-16T18:20:44.933364] [DEBUG] [LOCAL ] ES: Saving primary data. [2025-09-16T18:20:44.934802] [DEBUG] [LOCAL ] ES: Trimmed 57162 records to 39171 (keeping 150 days of data) [2025-09-16T18:20:45.175398] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-08. [2025-09-16T18:20:45.185590] [DEBUG] [LOCAL ] ES: Primary data saved (39171 records). [2025-09-16T18:20:45.185776] [INFO ] [MAIN ] ES: Full history saved. [2025-09-16T18:20:45.185885] [INFO ] [MAIN ] Update Progress: 764/2369 symbols completed. ES took 28.7s. Estimated remaining: 2h 49m 32s. Rate: 9.5 symbols/min [2025-09-16T18:20:45.416734] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T18:20:45.418193] [INFO ] [MAIN ] ESNT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:45.429494] [DEBUG] [LOCAL ] ESNT: No existing data file found in primary storage. [2025-09-16T18:20:45.429558] [WARN ] [MAIN ] ESNT: No existing data found, performing full refresh instead. [2025-09-16T18:20:45.430048] [INFO ] [MAIN ] ESNT: Wiping existing primary data before full refresh. [2025-09-16T18:20:45.430090] [INFO ] [LOCAL ] ESNT: Starting deletion of primary data. [2025-09-16T18:20:45.430131] [DEBUG] [LOCAL ] ESNT: No primary data file found to delete. [2025-09-16T18:20:45.430165] [INFO ] [MAIN ] ESNT: Starting full primary refresh. [2025-09-16T18:20:45.818299] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T18:20:45.823844] [INFO ] [REMOTE ] ESNT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:45.974956] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-06. [2025-09-16T18:20:46.642914] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-09-16T18:20:46.774898] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-09. [2025-09-16T18:20:47.050132] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T18:20:47.574922] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-04. [2025-09-16T18:20:47.935173] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-09-16T18:20:47.938250] [INFO ] [REMOTE ] ESAB: Full fetch complete (27359 records). [2025-09-16T18:20:47.938338] [DEBUG] [VALIDATOR ] ESAB: Validating primary data. [2025-09-16T18:20:47.938845] [FATAL] [VALIDATOR ] ESAB: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:20:47.938932] [FATAL] [MAIN ] ESAB: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:20:47.938999] [WARN ] [MAIN ] ESAB: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:20:47.939068] [INFO ] [LOCAL ] ESAB: Starting deletion of primary data. [2025-09-16T18:20:47.939137] [DEBUG] [LOCAL ] ESAB: No primary data file found to delete. [2025-09-16T18:20:47.939172] [INFO ] [LOCAL ] ESAB: Starting deletion of derivative data. [2025-09-16T18:20:47.939411] [DEBUG] [LOCAL ] ESAB: Deleted derivative data file. [2025-09-16T18:20:47.939500] [INFO ] [MAIN ] Update Progress: 765/2369 symbols completed. ESAB took 25.8s. Estimated remaining: 2h 49m 18s. Rate: 9.5 symbols/min [2025-09-16T18:20:48.118933] [INFO ] [MAIN ] ESRT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:48.155779] [DEBUG] [LOCAL ] ESRT: Loaded 34095 existing records from primary storage. [2025-09-16T18:20:48.155925] [INFO ] [MAIN ] ESRT: Loaded 34095 existing records. [2025-09-16T18:20:48.332068] [WARN ] [MAIN ] ESRT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:20:48.332322] [INFO ] [MAIN ] ESRT: Wiping existing primary data before full refresh. [2025-09-16T18:20:48.332371] [INFO ] [LOCAL ] ESRT: Starting deletion of primary data. [2025-09-16T18:20:48.333022] [DEBUG] [LOCAL ] ESRT: Deleted primary data file. [2025-09-16T18:20:48.333075] [INFO ] [MAIN ] ESRT: Starting full primary refresh. [2025-09-16T18:20:48.387105] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-07. [2025-09-16T18:20:48.695308] [INFO ] [REMOTE ] ESRT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:49.171950] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T18:20:49.186819] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-02. [2025-09-16T18:20:49.300301] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-09-16T18:20:49.974927] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-08. [2025-09-16T18:20:50.774949] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-02. [2025-09-16T18:20:50.841900] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T18:20:51.009590] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T18:20:51.574946] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-09. [2025-09-16T18:20:52.322098] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T18:20:52.374860] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-03. [2025-09-16T18:20:52.727748] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T18:20:52.730721] [INFO ] [REMOTE ] ESE: Full fetch complete (17819 records). [2025-09-16T18:20:52.730930] [DEBUG] [VALIDATOR ] ESE: Validating primary data. [2025-09-16T18:20:53.097340] [DEBUG] [VALIDATOR ] ESE: All expected trading days are present. [2025-09-16T18:20:53.097441] [DEBUG] [VALIDATOR ] ESE: Primary data validation passed. [2025-09-16T18:20:53.097888] [INFO ] [MAIN ] ESE: Fetched primary data is valid. [2025-09-16T18:20:53.097933] [DEBUG] [LOCAL ] ESE: Saving primary data. [2025-09-16T18:20:53.098549] [DEBUG] [LOCAL ] ESE: Trimmed 17819 records to 12263 (keeping 150 days of data) [2025-09-16T18:20:53.176878] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-03. [2025-09-16T18:20:53.182723] [DEBUG] [LOCAL ] ESE: Primary data saved (12263 records). [2025-09-16T18:20:53.183009] [INFO ] [MAIN ] ESE: Full history saved. [2025-09-16T18:20:53.183117] [INFO ] [MAIN ] Update Progress: 766/2369 symbols completed. ESE took 19.1s. Estimated remaining: 2h 49m 10s. Rate: 9.5 symbols/min [2025-09-16T18:20:53.352738] [INFO ] [MAIN ] ESS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:20:53.375209] [DEBUG] [LOCAL ] ESS: Loaded 21846 existing records from primary storage. [2025-09-16T18:20:53.375351] [INFO ] [MAIN ] ESS: Loaded 21846 existing records. [2025-09-16T18:20:53.556703] [WARN ] [MAIN ] ESS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:20:53.556945] [INFO ] [MAIN ] ESS: Wiping existing primary data before full refresh. [2025-09-16T18:20:53.557014] [INFO ] [LOCAL ] ESS: Starting deletion of primary data. [2025-09-16T18:20:53.557824] [DEBUG] [LOCAL ] ESS: Deleted primary data file. [2025-09-16T18:20:53.557874] [INFO ] [MAIN ] ESS: Starting full primary refresh. [2025-09-16T18:20:53.926450] [INFO ] [REMOTE ] ESS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:20:53.951539] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-09-16T18:20:53.974863] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-04. [2025-09-16T18:20:54.756508] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-09-16T18:20:54.774841] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-04. [2025-09-16T18:20:55.486893] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T18:20:55.574897] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-05. [2025-09-16T18:20:56.374948] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-05. [2025-09-16T18:20:56.598351] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T18:20:56.949770] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-09-16T18:20:57.174946] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-02. [2025-09-16T18:20:57.974944] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-06. [2025-09-16T18:20:58.031948] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T18:20:58.576334] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-09-16T18:20:58.774955] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-05. [2025-09-16T18:20:59.063495] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-09-16T18:20:59.574945] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-03. [2025-09-16T18:21:00.333677] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T18:21:00.374854] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-06. [2025-09-16T18:21:01.062634] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-09-16T18:21:01.174847] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-04. [2025-09-16T18:21:01.974912] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-06. [2025-09-16T18:21:02.135371] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T18:21:02.362670] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T18:21:02.774927] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-07. [2025-09-16T18:21:03.192337] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-09-16T18:21:03.574937] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-07. [2025-09-16T18:21:04.220536] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T18:21:04.374868] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-08. [2025-09-16T18:21:05.174859] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-07. [2025-09-16T18:21:05.272882] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T18:21:05.480310] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-09-16T18:21:05.974892] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-05. [2025-09-16T18:21:06.750042] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-09-16T18:21:06.774853] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-08. [2025-09-16T18:21:07.386751] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-09-16T18:21:07.574970] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-08. [2025-09-16T18:21:08.374923] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-09. [2025-09-16T18:21:08.497034] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-09-16T18:21:08.955497] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T18:21:09.174985] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-09. [2025-09-16T18:21:09.714876] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T18:21:09.724104] [INFO ] [REMOTE ] ESRT: Full fetch complete (50176 records). [2025-09-16T18:21:09.724428] [DEBUG] [VALIDATOR ] ESRT: Validating primary data. [2025-09-16T18:21:09.983611] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-09. [2025-09-16T18:21:10.154263] [DEBUG] [VALIDATOR ] ESRT: All expected trading days are present. [2025-09-16T18:21:10.154353] [DEBUG] [VALIDATOR ] ESRT: Primary data validation passed. [2025-09-16T18:21:10.155141] [INFO ] [MAIN ] ESRT: Fetched primary data is valid. [2025-09-16T18:21:10.155190] [DEBUG] [LOCAL ] ESRT: Saving primary data. [2025-09-16T18:21:10.156297] [DEBUG] [LOCAL ] ESRT: Trimmed 50176 records to 34466 (keeping 150 days of data) [2025-09-16T18:21:10.363843] [DEBUG] [LOCAL ] ESRT: Primary data saved (34466 records). [2025-09-16T18:21:10.364018] [INFO ] [MAIN ] ESRT: Full history saved. [2025-09-16T18:21:10.364110] [INFO ] [MAIN ] Update Progress: 767/2369 symbols completed. ESRT took 22.4s. Estimated remaining: 2h 49m 26s. Rate: 9.5 symbols/min [2025-09-16T18:21:10.544514] [INFO ] [MAIN ] ESTC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:21:10.580723] [DEBUG] [LOCAL ] ESTC: Loaded 37222 existing records from primary storage. [2025-09-16T18:21:10.580876] [INFO ] [MAIN ] ESTC: Loaded 37222 existing records. [2025-09-16T18:21:10.772241] [WARN ] [MAIN ] ESTC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:21:10.778728] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-06. [2025-09-16T18:21:10.787541] [INFO ] [MAIN ] ESTC: Wiping existing primary data before full refresh. [2025-09-16T18:21:10.788767] [INFO ] [LOCAL ] ESTC: Starting deletion of primary data. [2025-09-16T18:21:10.789866] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T18:21:10.793970] [DEBUG] [LOCAL ] ESTC: Deleted primary data file. [2025-09-16T18:21:10.794544] [INFO ] [MAIN ] ESTC: Starting full primary refresh. [2025-09-16T18:21:10.799928] [INFO ] [REMOTE ] ESNT: Full fetch complete (42800 records). [2025-09-16T18:21:10.866453] [DEBUG] [VALIDATOR ] ESNT: Validating primary data. [2025-09-16T18:21:11.444632] [INFO ] [REMOTE ] ESTC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:21:11.584760] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-02. [2025-09-16T18:21:11.585101] [DEBUG] [VALIDATOR ] ESNT: All expected trading days are present. [2025-09-16T18:21:11.586472] [DEBUG] [VALIDATOR ] ESNT: Primary data validation passed. [2025-09-16T18:21:11.588019] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T18:21:11.588967] [INFO ] [MAIN ] ESNT: Fetched primary data is valid. [2025-09-16T18:21:11.598186] [DEBUG] [LOCAL ] ESNT: Saving primary data. [2025-09-16T18:21:11.598530] [INFO ] [REMOTE ] ESI: Full fetch complete (55218 records). [2025-09-16T18:21:11.599062] [DEBUG] [VALIDATOR ] ESI: Validating primary data. [2025-09-16T18:21:11.600020] [DEBUG] [LOCAL ] ESNT: Trimmed 42800 records to 28016 (keeping 150 days of data) [2025-09-16T18:21:12.132159] [DEBUG] [VALIDATOR ] ESI: All expected trading days are present. [2025-09-16T18:21:12.132868] [DEBUG] [VALIDATOR ] ESI: Primary data validation passed. [2025-09-16T18:21:12.134540] [INFO ] [MAIN ] ESI: Fetched primary data is valid. [2025-09-16T18:21:12.134964] [DEBUG] [LOCAL ] ESI: Saving primary data. [2025-09-16T18:21:12.156604] [DEBUG] [LOCAL ] ESI: Trimmed 55218 records to 37431 (keeping 150 days of data) [2025-09-16T18:21:12.307859] [DEBUG] [LOCAL ] ESNT: Primary data saved (28016 records). [2025-09-16T18:21:12.315526] [INFO ] [MAIN ] ESNT: Full history saved. [2025-09-16T18:21:12.326212] [INFO ] [MAIN ] Update Progress: 768/2369 symbols completed. ESNT took 27.1s. Estimated remaining: 2h 49m 11s. Rate: 9.5 symbols/min [2025-09-16T18:21:12.350237] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T18:21:12.414278] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-07. [2025-09-16T18:21:12.550961] [INFO ] [MAIN ] ET: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:21:12.679156] [DEBUG] [LOCAL ] ET: Loaded 39750 existing records from primary storage. [2025-09-16T18:21:12.679802] [INFO ] [MAIN ] ET: Loaded 39750 existing records. [2025-09-16T18:21:12.874903] [WARN ] [MAIN ] ET: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:21:12.876399] [INFO ] [MAIN ] ET: Wiping existing primary data before full refresh. [2025-09-16T18:21:12.877023] [INFO ] [LOCAL ] ET: Starting deletion of primary data. [2025-09-16T18:21:12.878107] [DEBUG] [LOCAL ] ET: Deleted primary data file. [2025-09-16T18:21:12.878684] [INFO ] [MAIN ] ET: Starting full primary refresh. [2025-09-16T18:21:13.219395] [DEBUG] [LOCAL ] ESI: Primary data saved (37431 records). [2025-09-16T18:21:13.240902] [INFO ] [MAIN ] ESI: Full history saved. [2025-09-16T18:21:13.273605] [INFO ] [MAIN ] Update Progress: 769/2369 symbols completed. ESI took 34.2s. Estimated remaining: 2h 48m 53s. Rate: 9.5 symbols/min [2025-09-16T18:21:13.278124] [INFO ] [REMOTE ] ET: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:21:13.342788] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-02. [2025-09-16T18:21:13.476336] [INFO ] [MAIN ] ETB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:21:13.476560] [DEBUG] [LOCAL ] ETB: No existing data file found in primary storage. [2025-09-16T18:21:13.476665] [WARN ] [MAIN ] ETB: No existing data found, performing full refresh instead. [2025-09-16T18:21:13.476926] [INFO ] [MAIN ] ETB: Wiping existing primary data before full refresh. [2025-09-16T18:21:13.477072] [INFO ] [LOCAL ] ETB: Starting deletion of primary data. [2025-09-16T18:21:13.477146] [DEBUG] [LOCAL ] ETB: No primary data file found to delete. [2025-09-16T18:21:13.477258] [INFO ] [MAIN ] ETB: Starting full primary refresh. [2025-09-16T18:21:13.698973] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-02 (2.11 seconds). [2025-09-16T18:21:13.898061] [INFO ] [REMOTE ] ETB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:21:13.920003] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T18:21:13.974869] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-02. [2025-09-16T18:21:14.774923] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-03. [2025-09-16T18:21:15.174516] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-09-16T18:21:15.574911] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-08. [2025-09-16T18:21:15.750162] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-02 (2.41 seconds). [2025-09-16T18:21:16.379969] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-03. [2025-09-16T18:21:16.397810] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T18:21:16.994969] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T18:21:17.174904] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-03. [2025-09-16T18:21:17.484403] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T18:21:17.974907] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-09. [2025-09-16T18:21:18.774896] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-04. [2025-09-16T18:21:19.448252] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-03 (2.27 seconds). [2025-09-16T18:21:19.574913] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-04. [2025-09-16T18:21:19.592163] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T18:21:19.595881] [INFO ] [REMOTE ] ESS: Full fetch complete (32748 records). [2025-09-16T18:21:19.595963] [DEBUG] [VALIDATOR ] ESS: Validating primary data. [2025-09-16T18:21:19.812302] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T18:21:20.005055] [DEBUG] [VALIDATOR ] ESS: All expected trading days are present. [2025-09-16T18:21:20.005138] [DEBUG] [VALIDATOR ] ESS: Primary data validation passed. [2025-09-16T18:21:20.005703] [INFO ] [MAIN ] ESS: Fetched primary data is valid. [2025-09-16T18:21:20.005748] [DEBUG] [LOCAL ] ESS: Saving primary data. [2025-09-16T18:21:20.006500] [DEBUG] [LOCAL ] ESS: Trimmed 32748 records to 22109 (keeping 150 days of data) [2025-09-16T18:21:20.149487] [DEBUG] [LOCAL ] ESS: Primary data saved (22109 records). [2025-09-16T18:21:20.149630] [INFO ] [MAIN ] ESS: Full history saved. [2025-09-16T18:21:20.149752] [INFO ] [MAIN ] Update Progress: 770/2369 symbols completed. ESS took 27.0s. Estimated remaining: 2h 48m 48s. Rate: 9.5 symbols/min [2025-09-16T18:21:20.320095] [INFO ] [MAIN ] ETD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:21:20.339203] [DEBUG] [LOCAL ] ETD: Loaded 20372 existing records from primary storage. [2025-09-16T18:21:20.339353] [INFO ] [MAIN ] ETD: Loaded 20372 existing records. [2025-09-16T18:21:20.382270] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-05. [2025-09-16T18:21:20.518355] [WARN ] [MAIN ] ETD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:21:20.518599] [INFO ] [MAIN ] ETD: Wiping existing primary data before full refresh. [2025-09-16T18:21:20.518662] [INFO ] [LOCAL ] ETD: Starting deletion of primary data. [2025-09-16T18:21:20.519238] [DEBUG] [LOCAL ] ETD: Deleted primary data file. [2025-09-16T18:21:20.519313] [INFO ] [MAIN ] ETD: Starting full primary refresh. [2025-09-16T18:21:20.873720] [INFO ] [REMOTE ] ETD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:21:21.174902] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-04. [2025-09-16T18:21:21.271063] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-09-16T18:21:21.974878] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-06. [2025-09-16T18:21:22.065263] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-04 (2.49 seconds). [2025-09-16T18:21:22.477627] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-09-16T18:21:22.774904] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-07. [2025-09-16T18:21:22.923533] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-09-16T18:21:23.574906] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-02. [2025-09-16T18:21:23.607868] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-09-16T18:21:24.374893] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-05. [2025-09-16T18:21:24.935374] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T18:21:25.174890] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-03. [2025-09-16T18:21:25.974898] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-08. [2025-09-16T18:21:26.446425] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T18:21:26.511680] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-09-16T18:21:26.774966] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-05. [2025-09-16T18:21:27.165537] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-09-16T18:21:27.574909] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-04. [2025-09-16T18:21:28.374922] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-09. [2025-09-16T18:21:28.419224] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T18:21:29.062012] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-09-16T18:21:29.063469] [INFO ] [REMOTE ] ETB: Full fetch complete (13729 records). [2025-09-16T18:21:29.063538] [DEBUG] [VALIDATOR ] ETB: Validating primary data. [2025-09-16T18:21:29.180084] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-06. [2025-09-16T18:21:29.425211] [DEBUG] [VALIDATOR ] ETB: All expected trading days are present. [2025-09-16T18:21:29.425293] [DEBUG] [VALIDATOR ] ETB: Primary data validation passed. [2025-09-16T18:21:29.425585] [INFO ] [MAIN ] ETB: Fetched primary data is valid. [2025-09-16T18:21:29.425627] [DEBUG] [LOCAL ] ETB: Saving primary data. [2025-09-16T18:21:29.426176] [DEBUG] [LOCAL ] ETB: Trimmed 13729 records to 8126 (keeping 150 days of data) [2025-09-16T18:21:29.479211] [DEBUG] [LOCAL ] ETB: Primary data saved (8126 records). [2025-09-16T18:21:29.479371] [INFO ] [MAIN ] ETB: Full history saved. [2025-09-16T18:21:29.479454] [INFO ] [MAIN ] Update Progress: 771/2369 symbols completed. ETB took 16.2s. Estimated remaining: 2h 48m 48s. Rate: 9.5 symbols/min [2025-09-16T18:21:29.664286] [INFO ] [MAIN ] ETG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:21:29.676989] [DEBUG] [LOCAL ] ETG: Loaded 12094 existing records from primary storage. [2025-09-16T18:21:29.677112] [INFO ] [MAIN ] ETG: Loaded 12094 existing records. [2025-09-16T18:21:29.851225] [WARN ] [MAIN ] ETG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:21:29.851504] [INFO ] [MAIN ] ETG: Wiping existing primary data before full refresh. [2025-09-16T18:21:29.851579] [INFO ] [LOCAL ] ETG: Starting deletion of primary data. [2025-09-16T18:21:29.852076] [DEBUG] [LOCAL ] ETG: Deleted primary data file. [2025-09-16T18:21:29.852125] [INFO ] [MAIN ] ETG: Starting full primary refresh. [2025-09-16T18:21:29.980077] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-06. [2025-09-16T18:21:30.208364] [INFO ] [REMOTE ] ETG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:21:30.747075] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-04 (3.17 seconds). [2025-09-16T18:21:30.774846] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-05. [2025-09-16T18:21:31.141150] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-06 (1.96 seconds). [2025-09-16T18:21:31.574894] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-07. [2025-09-16T18:21:31.881526] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-09-16T18:21:32.374902] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-02. [2025-09-16T18:21:32.515341] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T18:21:33.132810] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T18:21:33.174894] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-03. [2025-09-16T18:21:33.454418] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T18:21:33.974905] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-08. [2025-09-16T18:21:34.398605] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-09-16T18:21:34.774943] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-07. [2025-09-16T18:21:35.574912] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-04. [2025-09-16T18:21:35.995289] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-09-16T18:21:36.238115] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T18:21:36.374906] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-08. [2025-09-16T18:21:36.715827] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-09-16T18:21:37.174907] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-06. [2025-09-16T18:21:37.974863] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-05. [2025-09-16T18:21:38.105735] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-09-16T18:21:38.174314] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T18:21:38.588852] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-09-16T18:21:38.774909] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-07. [2025-09-16T18:21:39.574908] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-09. [2025-09-16T18:21:39.816229] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-09-16T18:21:40.374941] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-09. [2025-09-16T18:21:41.154697] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-09-16T18:21:41.160817] [INFO ] [REMOTE ] ESTC: Full fetch complete (54518 records). [2025-09-16T18:21:41.160917] [DEBUG] [VALIDATOR ] ESTC: Validating primary data. [2025-09-16T18:21:41.182165] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-08. [2025-09-16T18:21:41.563713] [DEBUG] [VALIDATOR ] ESTC: All expected trading days are present. [2025-09-16T18:21:41.563803] [DEBUG] [VALIDATOR ] ESTC: Primary data validation passed. [2025-09-16T18:21:41.564681] [INFO ] [MAIN ] ESTC: Fetched primary data is valid. [2025-09-16T18:21:41.564753] [DEBUG] [LOCAL ] ESTC: Saving primary data. [2025-09-16T18:21:41.566032] [DEBUG] [LOCAL ] ESTC: Trimmed 54518 records to 37574 (keeping 150 days of data) [2025-09-16T18:21:41.789339] [DEBUG] [LOCAL ] ESTC: Primary data saved (37574 records). [2025-09-16T18:21:41.789518] [INFO ] [MAIN ] ESTC: Full history saved. [2025-09-16T18:21:41.789610] [INFO ] [MAIN ] Update Progress: 772/2369 symbols completed. ESTC took 31.4s. Estimated remaining: 2h 48m 54s. Rate: 9.5 symbols/min [2025-09-16T18:21:41.962431] [INFO ] [MAIN ] ETJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:21:41.973966] [DEBUG] [LOCAL ] ETJ: Loaded 10939 existing records from primary storage. [2025-09-16T18:21:41.974088] [INFO ] [MAIN ] ETJ: Loaded 10939 existing records. [2025-09-16T18:21:42.005786] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-06. [2025-09-16T18:21:42.160974] [WARN ] [MAIN ] ETJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:21:42.161915] [INFO ] [MAIN ] ETJ: Wiping existing primary data before full refresh. [2025-09-16T18:21:42.161970] [INFO ] [LOCAL ] ETJ: Starting deletion of primary data. [2025-09-16T18:21:42.167776] [DEBUG] [LOCAL ] ETJ: Deleted primary data file. [2025-09-16T18:21:42.177357] [INFO ] [MAIN ] ETJ: Starting full primary refresh. [2025-09-16T18:21:42.223479] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-09-16T18:21:42.555992] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-09-16T18:21:42.588022] [INFO ] [REMOTE ] ET: Full fetch complete (58080 records). [2025-09-16T18:21:42.592057] [INFO ] [REMOTE ] ETJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:21:42.592274] [DEBUG] [VALIDATOR ] ET: Validating primary data. [2025-09-16T18:21:42.593390] [FATAL] [VALIDATOR ] ET: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:21:42.593470] [FATAL] [MAIN ] ET: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:21:42.593507] [WARN ] [MAIN ] ET: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:21:42.593547] [INFO ] [LOCAL ] ET: Starting deletion of primary data. [2025-09-16T18:21:42.593604] [DEBUG] [LOCAL ] ET: No primary data file found to delete. [2025-09-16T18:21:42.593658] [INFO ] [LOCAL ] ET: Starting deletion of derivative data. [2025-09-16T18:21:42.593976] [DEBUG] [LOCAL ] ET: Deleted derivative data file. [2025-09-16T18:21:42.595248] [INFO ] [MAIN ] Update Progress: 773/2369 symbols completed. ET took 30.3s. Estimated remaining: 2h 48m 36s. Rate: 9.5 symbols/min [2025-09-16T18:21:42.762499] [INFO ] [MAIN ] ETN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:21:42.774941] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-09. [2025-09-16T18:21:42.802087] [DEBUG] [LOCAL ] ETN: Loaded 38883 existing records from primary storage. [2025-09-16T18:21:42.802226] [INFO ] [MAIN ] ETN: Loaded 38883 existing records. [2025-09-16T18:21:42.980287] [WARN ] [MAIN ] ETN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:21:42.992253] [INFO ] [MAIN ] ETN: Wiping existing primary data before full refresh. [2025-09-16T18:21:42.992332] [INFO ] [LOCAL ] ETN: Starting deletion of primary data. [2025-09-16T18:21:42.992807] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-09-16T18:21:42.996829] [DEBUG] [LOCAL ] ETN: Deleted primary data file. [2025-09-16T18:21:42.996930] [INFO ] [MAIN ] ETN: Starting full primary refresh. [2025-09-16T18:21:43.343712] [INFO ] [REMOTE ] ETN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:21:43.574848] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-02. [2025-09-16T18:21:43.619949] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T18:21:43.623799] [INFO ] [REMOTE ] ETD: Full fetch complete (29560 records). [2025-09-16T18:21:43.623885] [DEBUG] [VALIDATOR ] ETD: Validating primary data. [2025-09-16T18:21:43.998032] [DEBUG] [VALIDATOR ] ETD: All expected trading days are present. [2025-09-16T18:21:43.998117] [DEBUG] [VALIDATOR ] ETD: Primary data validation passed. [2025-09-16T18:21:43.998627] [INFO ] [MAIN ] ETD: Fetched primary data is valid. [2025-09-16T18:21:43.998689] [DEBUG] [LOCAL ] ETD: Saving primary data. [2025-09-16T18:21:43.999556] [DEBUG] [LOCAL ] ETD: Trimmed 29560 records to 20572 (keeping 150 days of data) [2025-09-16T18:21:44.125063] [DEBUG] [LOCAL ] ETD: Primary data saved (20572 records). [2025-09-16T18:21:44.125192] [INFO ] [MAIN ] ETD: Full history saved. [2025-09-16T18:21:44.125273] [INFO ] [MAIN ] Update Progress: 774/2369 symbols completed. ETD took 24.0s. Estimated remaining: 2h 48m 20s. Rate: 9.5 symbols/min [2025-09-16T18:21:44.301669] [INFO ] [MAIN ] ETO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:21:44.308673] [DEBUG] [LOCAL ] ETO: Loaded 4841 existing records from primary storage. [2025-09-16T18:21:44.308772] [INFO ] [MAIN ] ETO: Loaded 4841 existing records. [2025-09-16T18:21:44.384273] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-02. [2025-09-16T18:21:44.494046] [WARN ] [MAIN ] ETO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:21:44.494300] [INFO ] [MAIN ] ETO: Wiping existing primary data before full refresh. [2025-09-16T18:21:44.494354] [INFO ] [LOCAL ] ETO: Starting deletion of primary data. [2025-09-16T18:21:44.494614] [DEBUG] [LOCAL ] ETO: Deleted primary data file. [2025-09-16T18:21:44.494691] [INFO ] [MAIN ] ETO: Starting full primary refresh. [2025-09-16T18:21:44.854178] [INFO ] [REMOTE ] ETO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:21:45.174954] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-02. [2025-09-16T18:21:45.588462] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-09-16T18:21:45.608966] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T18:21:45.974953] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-03. [2025-09-16T18:21:46.030259] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T18:21:46.774920] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-07. [2025-09-16T18:21:47.001451] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-09-16T18:21:47.574942] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-04. [2025-09-16T18:21:47.673831] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-09-16T18:21:48.359823] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-09-16T18:21:48.374854] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-05. [2025-09-16T18:21:49.156319] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-09-16T18:21:49.174860] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-06. [2025-09-16T18:21:49.876771] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-09-16T18:21:49.974920] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-03. [2025-09-16T18:21:50.774868] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-03. [2025-09-16T18:21:50.783993] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-09-16T18:21:51.574922] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-08. [2025-09-16T18:21:52.374911] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-04. [2025-09-16T18:21:52.643252] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T18:21:52.910521] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-09-16T18:21:53.174937] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-09. [2025-09-16T18:21:53.699005] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T18:21:53.974945] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-05. [2025-09-16T18:21:54.774904] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-07. [2025-09-16T18:21:54.893060] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-09-16T18:21:54.895090] [INFO ] [REMOTE ] ETG: Full fetch complete (18297 records). [2025-09-16T18:21:54.895166] [DEBUG] [VALIDATOR ] ETG: Validating primary data. [2025-09-16T18:21:54.895736] [FATAL] [VALIDATOR ] ETG: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:21:54.895822] [FATAL] [MAIN ] ETG: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:21:54.895876] [WARN ] [MAIN ] ETG: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:21:54.895940] [INFO ] [LOCAL ] ETG: Starting deletion of primary data. [2025-09-16T18:21:54.896006] [DEBUG] [LOCAL ] ETG: No primary data file found to delete. [2025-09-16T18:21:54.896043] [INFO ] [LOCAL ] ETG: Starting deletion of derivative data. [2025-09-16T18:21:54.896326] [DEBUG] [LOCAL ] ETG: Deleted derivative data file. [2025-09-16T18:21:54.896411] [INFO ] [MAIN ] Update Progress: 775/2369 symbols completed. ETG took 25.4s. Estimated remaining: 2h 48m 22s. Rate: 9.5 symbols/min [2025-09-16T18:21:55.075841] [INFO ] [MAIN ] ETR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:21:55.111961] [DEBUG] [LOCAL ] ETR: Loaded 39268 existing records from primary storage. [2025-09-16T18:21:55.112095] [INFO ] [MAIN ] ETR: Loaded 39268 existing records. [2025-09-16T18:21:55.286795] [WARN ] [MAIN ] ETR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:21:55.287008] [INFO ] [MAIN ] ETR: Wiping existing primary data before full refresh. [2025-09-16T18:21:55.287054] [INFO ] [LOCAL ] ETR: Starting deletion of primary data. [2025-09-16T18:21:55.287894] [DEBUG] [LOCAL ] ETR: Deleted primary data file. [2025-09-16T18:21:55.287943] [INFO ] [MAIN ] ETR: Starting full primary refresh. [2025-09-16T18:21:55.577700] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-04. [2025-09-16T18:21:55.646203] [INFO ] [REMOTE ] ETR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:21:55.657401] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-09-16T18:21:55.669827] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-09-16T18:21:56.374919] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-02. [2025-09-16T18:21:57.174907] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-08. [2025-09-16T18:21:57.974903] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-06. [2025-09-16T18:21:58.549678] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-04 (2.97 seconds). [2025-09-16T18:21:58.588510] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-09-16T18:21:58.744073] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-02 (2.37 seconds). [2025-09-16T18:21:58.745074] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T18:21:58.774852] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-07. [2025-09-16T18:21:59.240552] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-09-16T18:21:59.574904] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-09. [2025-09-16T18:22:00.367598] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T18:22:00.371194] [INFO ] [REMOTE ] ETJ: Full fetch complete (18385 records). [2025-09-16T18:22:00.371329] [DEBUG] [VALIDATOR ] ETJ: Validating primary data. [2025-09-16T18:22:00.372064] [FATAL] [VALIDATOR ] ETJ: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:22:00.372167] [FATAL] [MAIN ] ETJ: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:22:00.372223] [WARN ] [MAIN ] ETJ: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:22:00.372281] [INFO ] [LOCAL ] ETJ: Starting deletion of primary data. [2025-09-16T18:22:00.372362] [DEBUG] [LOCAL ] ETJ: No primary data file found to delete. [2025-09-16T18:22:00.372417] [INFO ] [LOCAL ] ETJ: Starting deletion of derivative data. [2025-09-16T18:22:00.372713] [DEBUG] [LOCAL ] ETJ: Deleted derivative data file. [2025-09-16T18:22:00.372837] [INFO ] [MAIN ] Update Progress: 776/2369 symbols completed. ETJ took 18.6s. Estimated remaining: 2h 48m 14s. Rate: 9.5 symbols/min [2025-09-16T18:22:00.379962] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-03. [2025-09-16T18:22:00.558095] [INFO ] [MAIN ] ETV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:00.578035] [DEBUG] [LOCAL ] ETV: Loaded 14880 existing records from primary storage. [2025-09-16T18:22:00.578226] [INFO ] [MAIN ] ETV: Loaded 14880 existing records. [2025-09-16T18:22:00.759599] [WARN ] [MAIN ] ETV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:22:00.759907] [INFO ] [MAIN ] ETV: Wiping existing primary data before full refresh. [2025-09-16T18:22:00.759958] [INFO ] [LOCAL ] ETV: Starting deletion of primary data. [2025-09-16T18:22:00.760464] [DEBUG] [LOCAL ] ETV: Deleted primary data file. [2025-09-16T18:22:00.760511] [INFO ] [MAIN ] ETV: Starting full primary refresh. [2025-09-16T18:22:01.116061] [INFO ] [REMOTE ] ETV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:01.174862] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-02. [2025-09-16T18:22:01.974960] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-05. [2025-09-16T18:22:02.324188] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-09-16T18:22:02.541518] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-03 (2.16 seconds). [2025-09-16T18:22:02.774974] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-04. [2025-09-16T18:22:03.574908] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-03. [2025-09-16T18:22:04.037279] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-05 (2.06 seconds). [2025-09-16T18:22:04.258570] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T18:22:04.374918] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-06. [2025-09-16T18:22:04.651543] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T18:22:05.174886] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-05. [2025-09-16T18:22:05.974912] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-08. [2025-09-16T18:22:06.161975] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-09-16T18:22:06.605109] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-09-16T18:22:06.774886] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-09. [2025-09-16T18:22:06.858174] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T18:22:07.509397] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T18:22:07.510599] [INFO ] [REMOTE ] ETO: Full fetch complete (7153 records). [2025-09-16T18:22:07.510686] [DEBUG] [VALIDATOR ] ETO: Validating primary data. [2025-09-16T18:22:07.580022] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-04. [2025-09-16T18:22:07.875295] [DEBUG] [VALIDATOR ] ETO: All expected trading days are present. [2025-09-16T18:22:07.875371] [DEBUG] [VALIDATOR ] ETO: Primary data validation passed. [2025-09-16T18:22:07.875569] [INFO ] [MAIN ] ETO: Fetched primary data is valid. [2025-09-16T18:22:07.875623] [DEBUG] [LOCAL ] ETO: Saving primary data. [2025-09-16T18:22:07.876027] [DEBUG] [LOCAL ] ETO: Trimmed 7153 records to 4864 (keeping 150 days of data) [2025-09-16T18:22:07.906133] [DEBUG] [LOCAL ] ETO: Primary data saved (4864 records). [2025-09-16T18:22:07.906232] [INFO ] [MAIN ] ETO: Full history saved. [2025-09-16T18:22:07.906326] [INFO ] [MAIN ] Update Progress: 777/2369 symbols completed. ETO took 23.8s. Estimated remaining: 2h 48m 10s. Rate: 9.5 symbols/min [2025-09-16T18:22:08.105436] [INFO ] [MAIN ] ETW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:08.122568] [DEBUG] [LOCAL ] ETW: Loaded 17970 existing records from primary storage. [2025-09-16T18:22:08.122715] [INFO ] [MAIN ] ETW: Loaded 17970 existing records. [2025-09-16T18:22:08.303506] [WARN ] [MAIN ] ETW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:22:08.303808] [INFO ] [MAIN ] ETW: Wiping existing primary data before full refresh. [2025-09-16T18:22:08.303878] [INFO ] [LOCAL ] ETW: Starting deletion of primary data. [2025-09-16T18:22:08.304371] [DEBUG] [LOCAL ] ETW: Deleted primary data file. [2025-09-16T18:22:08.304418] [INFO ] [MAIN ] ETW: Starting full primary refresh. [2025-09-16T18:22:08.380094] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-07. [2025-09-16T18:22:08.662488] [INFO ] [REMOTE ] ETW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:08.916135] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-09-16T18:22:09.174897] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-05. [2025-09-16T18:22:09.974916] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-02. [2025-09-16T18:22:10.179130] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T18:22:10.733510] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-09-16T18:22:10.774851] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-08. [2025-09-16T18:22:11.398189] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T18:22:11.574917] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-06. [2025-09-16T18:22:12.374921] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-03. [2025-09-16T18:22:12.740769] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-09-16T18:22:12.773518] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-09-16T18:22:13.174883] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-09. [2025-09-16T18:22:13.563299] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-09-16T18:22:13.974917] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-07. [2025-09-16T18:22:14.774973] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-04. [2025-09-16T18:22:15.366320] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-09-16T18:22:15.398578] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-09-16T18:22:15.404980] [INFO ] [REMOTE ] ETN: Full fetch complete (57547 records). [2025-09-16T18:22:15.405122] [DEBUG] [VALIDATOR ] ETN: Validating primary data. [2025-09-16T18:22:15.585273] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-08. [2025-09-16T18:22:15.799454] [DEBUG] [VALIDATOR ] ETN: All expected trading days are present. [2025-09-16T18:22:15.799578] [DEBUG] [VALIDATOR ] ETN: Primary data validation passed. [2025-09-16T18:22:15.800369] [INFO ] [MAIN ] ETN: Fetched primary data is valid. [2025-09-16T18:22:15.800425] [DEBUG] [LOCAL ] ETN: Saving primary data. [2025-09-16T18:22:15.801825] [DEBUG] [LOCAL ] ETN: Trimmed 57547 records to 39261 (keeping 150 days of data) [2025-09-16T18:22:15.918625] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-09-16T18:22:16.083985] [DEBUG] [LOCAL ] ETN: Primary data saved (39261 records). [2025-09-16T18:22:16.084146] [INFO ] [MAIN ] ETN: Full history saved. [2025-09-16T18:22:16.085047] [INFO ] [MAIN ] Update Progress: 778/2369 symbols completed. ETN took 33.5s. Estimated remaining: 2h 48m 8s. Rate: 9.5 symbols/min [2025-09-16T18:22:16.263354] [INFO ] [MAIN ] ETX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:16.263590] [DEBUG] [LOCAL ] ETX: No existing data file found in primary storage. [2025-09-16T18:22:16.263631] [WARN ] [MAIN ] ETX: No existing data found, performing full refresh instead. [2025-09-16T18:22:16.263805] [INFO ] [MAIN ] ETX: Wiping existing primary data before full refresh. [2025-09-16T18:22:16.263869] [INFO ] [LOCAL ] ETX: Starting deletion of primary data. [2025-09-16T18:22:16.263948] [DEBUG] [LOCAL ] ETX: No primary data file found to delete. [2025-09-16T18:22:16.263999] [INFO ] [MAIN ] ETX: Starting full primary refresh. [2025-09-16T18:22:16.380138] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-05. [2025-09-16T18:22:16.627725] [INFO ] [REMOTE ] ETX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:16.641349] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-09-16T18:22:17.174932] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-09. [2025-09-16T18:22:17.662709] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-09-16T18:22:17.974939] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-02. [2025-09-16T18:22:18.154273] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T18:22:18.156988] [INFO ] [REMOTE ] ETV: Full fetch complete (23277 records). [2025-09-16T18:22:18.157184] [DEBUG] [VALIDATOR ] ETV: Validating primary data. [2025-09-16T18:22:18.545934] [DEBUG] [VALIDATOR ] ETV: All expected trading days are present. [2025-09-16T18:22:18.546013] [DEBUG] [VALIDATOR ] ETV: Primary data validation passed. [2025-09-16T18:22:18.546463] [INFO ] [MAIN ] ETV: Fetched primary data is valid. [2025-09-16T18:22:18.546503] [DEBUG] [LOCAL ] ETV: Saving primary data. [2025-09-16T18:22:18.547119] [DEBUG] [LOCAL ] ETV: Trimmed 23277 records to 15040 (keeping 150 days of data) [2025-09-16T18:22:18.639259] [DEBUG] [LOCAL ] ETV: Primary data saved (15040 records). [2025-09-16T18:22:18.639390] [INFO ] [MAIN ] ETV: Full history saved. [2025-09-16T18:22:18.639473] [INFO ] [MAIN ] Update Progress: 779/2369 symbols completed. ETV took 18.3s. Estimated remaining: 2h 47m 54s. Rate: 9.5 symbols/min [2025-09-16T18:22:18.781003] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-06. [2025-09-16T18:22:18.812445] [INFO ] [MAIN ] ETY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:18.813571] [DEBUG] [LOCAL ] ETY: No existing data file found in primary storage. [2025-09-16T18:22:18.818391] [WARN ] [MAIN ] ETY: No existing data found, performing full refresh instead. [2025-09-16T18:22:18.819330] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T18:22:18.819446] [INFO ] [MAIN ] ETY: Wiping existing primary data before full refresh. [2025-09-16T18:22:18.821874] [INFO ] [LOCAL ] ETY: Starting deletion of primary data. [2025-09-16T18:22:18.822142] [DEBUG] [LOCAL ] ETY: No primary data file found to delete. [2025-09-16T18:22:18.822241] [INFO ] [MAIN ] ETY: Starting full primary refresh. [2025-09-16T18:22:19.181680] [INFO ] [REMOTE ] ETY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:19.575073] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-03. [2025-09-16T18:22:20.029661] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-09-16T18:22:20.374952] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-06. [2025-09-16T18:22:20.427390] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T18:22:21.174921] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-04. [2025-09-16T18:22:21.420055] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T18:22:21.703016] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-09-16T18:22:21.974897] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-05. [2025-09-16T18:22:22.347860] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-09-16T18:22:22.774940] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-07. [2025-09-16T18:22:23.574908] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-02. [2025-09-16T18:22:24.144337] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T18:22:24.374907] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-08. [2025-09-16T18:22:24.862101] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T18:22:25.174898] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-03. [2025-09-16T18:22:25.452933] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T18:22:25.974907] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-06. [2025-09-16T18:22:26.241326] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-09-16T18:22:26.600173] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-09-16T18:22:26.774900] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-09. [2025-09-16T18:22:27.574894] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-07. [2025-09-16T18:22:28.067434] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T18:22:28.070366] [INFO ] [REMOTE ] ETW: Full fetch complete (27081 records). [2025-09-16T18:22:28.070439] [DEBUG] [VALIDATOR ] ETW: Validating primary data. [2025-09-16T18:22:28.380033] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-04. [2025-09-16T18:22:28.457280] [DEBUG] [VALIDATOR ] ETW: All expected trading days are present. [2025-09-16T18:22:28.457380] [DEBUG] [VALIDATOR ] ETW: Primary data validation passed. [2025-09-16T18:22:28.458005] [INFO ] [MAIN ] ETW: Fetched primary data is valid. [2025-09-16T18:22:28.458066] [DEBUG] [LOCAL ] ETW: Saving primary data. [2025-09-16T18:22:28.459015] [DEBUG] [LOCAL ] ETW: Trimmed 27081 records to 18084 (keeping 150 days of data) [2025-09-16T18:22:28.569274] [DEBUG] [LOCAL ] ETW: Primary data saved (18084 records). [2025-09-16T18:22:28.569433] [INFO ] [MAIN ] ETW: Full history saved. [2025-09-16T18:22:28.569525] [INFO ] [MAIN ] Update Progress: 780/2369 symbols completed. ETW took 20.7s. Estimated remaining: 2h 47m 55s. Rate: 9.5 symbols/min [2025-09-16T18:22:28.750450] [INFO ] [MAIN ] EVC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:28.764780] [DEBUG] [LOCAL ] EVC: Loaded 13742 existing records from primary storage. [2025-09-16T18:22:28.764928] [INFO ] [MAIN ] EVC: Loaded 13742 existing records. [2025-09-16T18:22:28.951944] [WARN ] [MAIN ] EVC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:22:28.952241] [INFO ] [MAIN ] EVC: Wiping existing primary data before full refresh. [2025-09-16T18:22:28.952298] [INFO ] [LOCAL ] EVC: Starting deletion of primary data. [2025-09-16T18:22:28.952744] [DEBUG] [LOCAL ] EVC: Deleted primary data file. [2025-09-16T18:22:28.952793] [INFO ] [MAIN ] EVC: Starting full primary refresh. [2025-09-16T18:22:29.183740] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-07. [2025-09-16T18:22:29.328192] [INFO ] [REMOTE ] EVC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:29.392444] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-09-16T18:22:29.460497] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T18:22:29.974910] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-02. [2025-09-16T18:22:29.984954] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T18:22:30.774902] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-08. [2025-09-16T18:22:31.249017] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T18:22:31.528659] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T18:22:31.574855] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-09. [2025-09-16T18:22:32.271784] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-09-16T18:22:32.273131] [INFO ] [REMOTE ] ETX: Full fetch complete (5332 records). [2025-09-16T18:22:32.273203] [DEBUG] [VALIDATOR ] ETX: Validating primary data. [2025-09-16T18:22:32.379946] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-03. [2025-09-16T18:22:32.648745] [DEBUG] [VALIDATOR ] ETX: All expected trading days are present. [2025-09-16T18:22:32.648822] [DEBUG] [VALIDATOR ] ETX: Primary data validation passed. [2025-09-16T18:22:32.648978] [INFO ] [MAIN ] ETX: Fetched primary data is valid. [2025-09-16T18:22:32.649018] [DEBUG] [LOCAL ] ETX: Saving primary data. [2025-09-16T18:22:32.649409] [DEBUG] [LOCAL ] ETX: Trimmed 5332 records to 4122 (keeping 150 days of data) [2025-09-16T18:22:32.677552] [DEBUG] [LOCAL ] ETX: Primary data saved (4122 records). [2025-09-16T18:22:32.677656] [INFO ] [MAIN ] ETX: Full history saved. [2025-09-16T18:22:32.677726] [INFO ] [MAIN ] Update Progress: 781/2369 symbols completed. ETX took 16.6s. Estimated remaining: 2h 47m 44s. Rate: 9.5 symbols/min [2025-09-16T18:22:32.846845] [INFO ] [MAIN ] EVEX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:32.871369] [DEBUG] [LOCAL ] EVEX: Loaded 22449 existing records from primary storage. [2025-09-16T18:22:32.871497] [INFO ] [MAIN ] EVEX: Loaded 22449 existing records. [2025-09-16T18:22:33.051247] [WARN ] [MAIN ] EVEX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:22:33.051461] [INFO ] [MAIN ] EVEX: Wiping existing primary data before full refresh. [2025-09-16T18:22:33.051534] [INFO ] [LOCAL ] EVEX: Starting deletion of primary data. [2025-09-16T18:22:33.052066] [DEBUG] [LOCAL ] EVEX: Deleted primary data file. [2025-09-16T18:22:33.052127] [INFO ] [MAIN ] EVEX: Starting full primary refresh. [2025-09-16T18:22:33.179262] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-05. [2025-09-16T18:22:33.413155] [INFO ] [REMOTE ] EVEX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:33.639018] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T18:22:33.974908] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-04. [2025-09-16T18:22:34.297383] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T18:22:34.661946] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-09-16T18:22:34.774868] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-05. [2025-09-16T18:22:35.574870] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-06. [2025-09-16T18:22:35.789283] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T18:22:36.374867] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-02. [2025-09-16T18:22:36.727792] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-09-16T18:22:37.174890] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-07. [2025-09-16T18:22:37.621006] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T18:22:37.942040] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T18:22:37.974861] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-08. [2025-09-16T18:22:38.682194] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-09-16T18:22:38.774902] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-03. [2025-09-16T18:22:39.574903] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-08. [2025-09-16T18:22:40.202534] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-09-16T18:22:40.374893] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-04. [2025-09-16T18:22:41.098466] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-09-16T18:22:41.174904] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-05. [2025-09-16T18:22:41.952616] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-08 (2.38 seconds). [2025-09-16T18:22:41.974876] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-09. [2025-09-16T18:22:42.115827] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-09-16T18:22:42.774896] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-06. [2025-09-16T18:22:43.286696] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T18:22:43.294466] [INFO ] [REMOTE ] ETR: Full fetch complete (57924 records). [2025-09-16T18:22:43.294603] [DEBUG] [VALIDATOR ] ETR: Validating primary data. [2025-09-16T18:22:43.580023] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-09. [2025-09-16T18:22:43.682170] [DEBUG] [VALIDATOR ] ETR: All expected trading days are present. [2025-09-16T18:22:43.682246] [DEBUG] [VALIDATOR ] ETR: Primary data validation passed. [2025-09-16T18:22:43.683009] [INFO ] [MAIN ] ETR: Fetched primary data is valid. [2025-09-16T18:22:43.683072] [DEBUG] [LOCAL ] ETR: Saving primary data. [2025-09-16T18:22:43.684228] [DEBUG] [LOCAL ] ETR: Trimmed 57924 records to 39652 (keeping 150 days of data) [2025-09-16T18:22:43.939577] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-09-16T18:22:43.956803] [DEBUG] [LOCAL ] ETR: Primary data saved (39652 records). [2025-09-16T18:22:43.956928] [INFO ] [MAIN ] ETR: Full history saved. [2025-09-16T18:22:43.957056] [INFO ] [MAIN ] Update Progress: 782/2369 symbols completed. ETR took 49.1s. Estimated remaining: 2h 47m 48s. Rate: 9.5 symbols/min [2025-09-16T18:22:44.134507] [INFO ] [MAIN ] EVF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:44.143152] [DEBUG] [LOCAL ] EVF: Loaded 6398 existing records from primary storage. [2025-09-16T18:22:44.143255] [INFO ] [MAIN ] EVF: Loaded 6398 existing records. [2025-09-16T18:22:44.329247] [WARN ] [MAIN ] EVF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:22:44.329541] [INFO ] [MAIN ] EVF: Wiping existing primary data before full refresh. [2025-09-16T18:22:44.329605] [INFO ] [LOCAL ] EVF: Starting deletion of primary data. [2025-09-16T18:22:44.329914] [DEBUG] [LOCAL ] EVF: Deleted primary data file. [2025-09-16T18:22:44.329985] [INFO ] [MAIN ] EVF: Starting full primary refresh. [2025-09-16T18:22:44.386162] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-06. [2025-09-16T18:22:44.633388] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T18:22:44.687184] [INFO ] [REMOTE ] ETY: Full fetch complete (25317 records). [2025-09-16T18:22:44.710768] [DEBUG] [VALIDATOR ] ETY: Validating primary data. [2025-09-16T18:22:44.715242] [INFO ] [REMOTE ] EVF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:45.094417] [DEBUG] [VALIDATOR ] ETY: All expected trading days are present. [2025-09-16T18:22:45.094522] [DEBUG] [VALIDATOR ] ETY: Primary data validation passed. [2025-09-16T18:22:45.095136] [INFO ] [MAIN ] ETY: Fetched primary data is valid. [2025-09-16T18:22:45.095184] [DEBUG] [LOCAL ] ETY: Saving primary data. [2025-09-16T18:22:45.096827] [DEBUG] [LOCAL ] ETY: Trimmed 25317 records to 16334 (keeping 150 days of data) [2025-09-16T18:22:45.175713] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-02. [2025-09-16T18:22:45.205625] [DEBUG] [LOCAL ] ETY: Primary data saved (16334 records). [2025-09-16T18:22:45.205803] [INFO ] [MAIN ] ETY: Full history saved. [2025-09-16T18:22:45.205895] [INFO ] [MAIN ] Update Progress: 783/2369 symbols completed. ETY took 26.6s. Estimated remaining: 2h 47m 31s. Rate: 9.5 symbols/min [2025-09-16T18:22:45.376670] [INFO ] [MAIN ] EVG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:45.392756] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T18:22:45.400683] [DEBUG] [LOCAL ] EVG: Loaded 4289 existing records from primary storage. [2025-09-16T18:22:45.401127] [INFO ] [MAIN ] EVG: Loaded 4289 existing records. [2025-09-16T18:22:45.574284] [WARN ] [MAIN ] EVG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:22:45.574528] [INFO ] [MAIN ] EVG: Wiping existing primary data before full refresh. [2025-09-16T18:22:45.574575] [INFO ] [LOCAL ] EVG: Starting deletion of primary data. [2025-09-16T18:22:45.574963] [DEBUG] [LOCAL ] EVG: Deleted primary data file. [2025-09-16T18:22:45.575012] [INFO ] [MAIN ] EVG: Starting full primary refresh. [2025-09-16T18:22:45.925617] [INFO ] [REMOTE ] EVG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:45.969324] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T18:22:45.974833] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-03. [2025-09-16T18:22:46.455069] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-09-16T18:22:46.774902] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-07. [2025-09-16T18:22:47.560780] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T18:22:47.574845] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-08. [2025-09-16T18:22:48.158253] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-09-16T18:22:48.374895] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-04. [2025-09-16T18:22:48.874597] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-09-16T18:22:49.174920] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-05. [2025-09-16T18:22:49.577602] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-09-16T18:22:49.974902] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-06. [2025-09-16T18:22:50.516152] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-09-16T18:22:50.774934] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-02. [2025-09-16T18:22:51.207452] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-09-16T18:22:51.574925] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-07. [2025-09-16T18:22:52.109534] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-09-16T18:22:52.374903] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-08. [2025-09-16T18:22:52.956308] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-09-16T18:22:53.174880] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-09. [2025-09-16T18:22:53.974923] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-07. [2025-09-16T18:22:54.032154] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T18:22:54.033669] [INFO ] [REMOTE ] EVF: Full fetch complete (9510 records). [2025-09-16T18:22:54.033740] [DEBUG] [VALIDATOR ] EVF: Validating primary data. [2025-09-16T18:22:54.399317] [DEBUG] [VALIDATOR ] EVF: All expected trading days are present. [2025-09-16T18:22:54.399395] [DEBUG] [VALIDATOR ] EVF: Primary data validation passed. [2025-09-16T18:22:54.399658] [INFO ] [MAIN ] EVF: Fetched primary data is valid. [2025-09-16T18:22:54.399703] [DEBUG] [LOCAL ] EVF: Saving primary data. [2025-09-16T18:22:54.400170] [DEBUG] [LOCAL ] EVF: Trimmed 9510 records to 6450 (keeping 150 days of data) [2025-09-16T18:22:54.437935] [DEBUG] [LOCAL ] EVF: Primary data saved (6450 records). [2025-09-16T18:22:54.438045] [INFO ] [MAIN ] EVF: Full history saved. [2025-09-16T18:22:54.438151] [INFO ] [MAIN ] Update Progress: 784/2369 symbols completed. EVF took 10.5s. Estimated remaining: 2h 47m 30s. Rate: 9.5 symbols/min [2025-09-16T18:22:54.609528] [INFO ] [MAIN ] EVH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:54.646242] [DEBUG] [LOCAL ] EVH: Loaded 36311 existing records from primary storage. [2025-09-16T18:22:54.646389] [INFO ] [MAIN ] EVH: Loaded 36311 existing records. [2025-09-16T18:22:54.780287] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-09. [2025-09-16T18:22:54.842472] [WARN ] [MAIN ] EVH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:22:54.842819] [INFO ] [MAIN ] EVH: Wiping existing primary data before full refresh. [2025-09-16T18:22:54.842889] [INFO ] [LOCAL ] EVH: Starting deletion of primary data. [2025-09-16T18:22:54.843757] [DEBUG] [LOCAL ] EVH: Deleted primary data file. [2025-09-16T18:22:54.843810] [INFO ] [MAIN ] EVH: Starting full primary refresh. [2025-09-16T18:22:55.204167] [INFO ] [REMOTE ] EVH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:55.395762] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-09-16T18:22:55.574999] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-03. [2025-09-16T18:22:55.951878] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-09-16T18:22:55.954504] [INFO ] [REMOTE ] EVC: Full fetch complete (21967 records). [2025-09-16T18:22:55.954602] [DEBUG] [VALIDATOR ] EVC: Validating primary data. [2025-09-16T18:22:55.955200] [FATAL] [VALIDATOR ] EVC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:22:55.955305] [FATAL] [MAIN ] EVC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:22:55.955376] [WARN ] [MAIN ] EVC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:22:55.955452] [INFO ] [LOCAL ] EVC: Starting deletion of primary data. [2025-09-16T18:22:55.955526] [DEBUG] [LOCAL ] EVC: No primary data file found to delete. [2025-09-16T18:22:55.955577] [INFO ] [LOCAL ] EVC: Starting deletion of derivative data. [2025-09-16T18:22:55.955942] [DEBUG] [LOCAL ] EVC: Deleted derivative data file. [2025-09-16T18:22:55.956069] [INFO ] [MAIN ] Update Progress: 785/2369 symbols completed. EVC took 27.4s. Estimated remaining: 2h 47m 14s. Rate: 9.5 symbols/min [2025-09-16T18:22:56.092225] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-09-16T18:22:56.148364] [INFO ] [MAIN ] EVN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:22:56.148675] [DEBUG] [LOCAL ] EVN: No existing data file found in primary storage. [2025-09-16T18:22:56.148720] [WARN ] [MAIN ] EVN: No existing data found, performing full refresh instead. [2025-09-16T18:22:56.148885] [INFO ] [MAIN ] EVN: Wiping existing primary data before full refresh. [2025-09-16T18:22:56.148930] [INFO ] [LOCAL ] EVN: Starting deletion of primary data. [2025-09-16T18:22:56.148973] [DEBUG] [LOCAL ] EVN: No primary data file found to delete. [2025-09-16T18:22:56.149009] [INFO ] [MAIN ] EVN: Starting full primary refresh. [2025-09-16T18:22:56.380111] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-02. [2025-09-16T18:22:56.568988] [INFO ] [REMOTE ] EVN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:22:57.174937] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-02. [2025-09-16T18:22:57.773470] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T18:22:57.794482] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-09-16T18:22:57.974990] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-04. [2025-09-16T18:22:58.422508] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-09-16T18:22:58.774954] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-03. [2025-09-16T18:22:59.574950] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-08. [2025-09-16T18:22:59.600880] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T18:23:00.374907] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-05. [2025-09-16T18:23:00.505057] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T18:23:00.782699] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-09-16T18:23:01.174931] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-09. [2025-09-16T18:23:01.974944] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-06. [2025-09-16T18:23:02.774913] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-04. [2025-09-16T18:23:03.574955] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-03. [2025-09-16T18:23:04.766383] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-06 (2.79 seconds). [2025-09-16T18:23:04.768234] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-07. [2025-09-16T18:23:04.862616] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-04 (2.09 seconds). [2025-09-16T18:23:05.174890] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-05. [2025-09-16T18:23:05.401584] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T18:23:05.565397] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T18:23:05.832721] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-09-16T18:23:05.965136] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-09 (4.79 seconds). [2025-09-16T18:23:05.970355] [INFO ] [REMOTE ] EVEX: Full fetch complete (30554 records). [2025-09-16T18:23:05.970441] [DEBUG] [VALIDATOR ] EVEX: Validating primary data. [2025-09-16T18:23:05.970975] [FATAL] [VALIDATOR ] EVEX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:23:05.971056] [FATAL] [MAIN ] EVEX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:23:05.971094] [WARN ] [MAIN ] EVEX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:23:05.971191] [INFO ] [LOCAL ] EVEX: Starting deletion of primary data. [2025-09-16T18:23:05.971253] [DEBUG] [LOCAL ] EVEX: No primary data file found to delete. [2025-09-16T18:23:05.971288] [INFO ] [LOCAL ] EVEX: Starting deletion of derivative data. [2025-09-16T18:23:05.971576] [DEBUG] [LOCAL ] EVEX: Deleted derivative data file. [2025-09-16T18:23:05.971677] [INFO ] [MAIN ] Update Progress: 786/2369 symbols completed. EVEX took 33.3s. Estimated remaining: 2h 47m 16s. Rate: 9.5 symbols/min [2025-09-16T18:23:05.982586] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-04. [2025-09-16T18:23:06.150540] [INFO ] [MAIN ] EVR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:23:06.150745] [DEBUG] [LOCAL ] EVR: No existing data file found in primary storage. [2025-09-16T18:23:06.150786] [WARN ] [MAIN ] EVR: No existing data found, performing full refresh instead. [2025-09-16T18:23:06.150901] [INFO ] [MAIN ] EVR: Wiping existing primary data before full refresh. [2025-09-16T18:23:06.150939] [INFO ] [LOCAL ] EVR: Starting deletion of primary data. [2025-09-16T18:23:06.150980] [DEBUG] [LOCAL ] EVR: No primary data file found to delete. [2025-09-16T18:23:06.151012] [INFO ] [MAIN ] EVR: Starting full primary refresh. [2025-09-16T18:23:06.538864] [INFO ] [REMOTE ] EVR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:23:06.774895] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-02. [2025-09-16T18:23:07.284167] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-09-16T18:23:07.574947] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-06. [2025-09-16T18:23:08.234375] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-09-16T18:23:08.384085] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-08. [2025-09-16T18:23:08.386391] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T18:23:09.043610] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-09-16T18:23:09.174919] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-03. [2025-09-16T18:23:09.974916] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-07. [2025-09-16T18:23:10.728769] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T18:23:10.774914] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-04. [2025-09-16T18:23:10.821723] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-09-16T18:23:11.574901] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-05. [2025-09-16T18:23:12.375913] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-08. [2025-09-16T18:23:12.378224] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T18:23:13.174867] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-09. [2025-09-16T18:23:13.280934] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-09-16T18:23:13.303608] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T18:23:13.961871] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T18:23:13.963174] [INFO ] [REMOTE ] EVG: Full fetch complete (6205 records). [2025-09-16T18:23:13.963242] [DEBUG] [VALIDATOR ] EVG: Validating primary data. [2025-09-16T18:23:13.984341] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-09. [2025-09-16T18:23:14.332185] [DEBUG] [VALIDATOR ] EVG: All expected trading days are present. [2025-09-16T18:23:14.332264] [DEBUG] [VALIDATOR ] EVG: Primary data validation passed. [2025-09-16T18:23:14.332450] [INFO ] [MAIN ] EVG: Fetched primary data is valid. [2025-09-16T18:23:14.332492] [DEBUG] [LOCAL ] EVG: Saving primary data. [2025-09-16T18:23:14.332901] [DEBUG] [LOCAL ] EVG: Trimmed 6205 records to 4332 (keeping 150 days of data) [2025-09-16T18:23:14.360633] [DEBUG] [LOCAL ] EVG: Primary data saved (4332 records). [2025-09-16T18:23:14.360734] [INFO ] [MAIN ] EVG: Full history saved. [2025-09-16T18:23:14.360815] [INFO ] [MAIN ] Update Progress: 787/2369 symbols completed. EVG took 29.2s. Estimated remaining: 2h 47m 13s. Rate: 9.5 symbols/min [2025-09-16T18:23:14.532987] [INFO ] [MAIN ] EVT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:23:14.547388] [DEBUG] [LOCAL ] EVT: Loaded 12619 existing records from primary storage. [2025-09-16T18:23:14.547505] [INFO ] [MAIN ] EVT: Loaded 12619 existing records. [2025-09-16T18:23:14.745185] [WARN ] [MAIN ] EVT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:23:14.745485] [INFO ] [MAIN ] EVT: Wiping existing primary data before full refresh. [2025-09-16T18:23:14.745551] [INFO ] [LOCAL ] EVT: Starting deletion of primary data. [2025-09-16T18:23:14.746135] [DEBUG] [LOCAL ] EVT: Deleted primary data file. [2025-09-16T18:23:14.746207] [INFO ] [MAIN ] EVT: Starting full primary refresh. [2025-09-16T18:23:14.781335] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-06. [2025-09-16T18:23:14.996496] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T18:23:15.066746] [INFO ] [REMOTE ] EVN: Full fetch complete (12037 records). [2025-09-16T18:23:15.066955] [DEBUG] [VALIDATOR ] EVN: Validating primary data. [2025-09-16T18:23:15.520929] [INFO ] [REMOTE ] EVT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:23:15.578940] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-02. [2025-09-16T18:23:15.770430] [DEBUG] [VALIDATOR ] EVN: All expected trading days are present. [2025-09-16T18:23:15.770526] [DEBUG] [VALIDATOR ] EVN: Primary data validation passed. [2025-09-16T18:23:15.770948] [INFO ] [MAIN ] EVN: Fetched primary data is valid. [2025-09-16T18:23:15.770996] [DEBUG] [LOCAL ] EVN: Saving primary data. [2025-09-16T18:23:15.771537] [DEBUG] [LOCAL ] EVN: Trimmed 12037 records to 7786 (keeping 150 days of data) [2025-09-16T18:23:15.824046] [DEBUG] [LOCAL ] EVN: Primary data saved (7786 records). [2025-09-16T18:23:15.824213] [INFO ] [MAIN ] EVN: Full history saved. [2025-09-16T18:23:15.824298] [INFO ] [MAIN ] Update Progress: 788/2369 symbols completed. EVN took 19.9s. Estimated remaining: 2h 46m 57s. Rate: 9.5 symbols/min [2025-09-16T18:23:16.003065] [INFO ] [MAIN ] EVTC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:23:16.023813] [DEBUG] [LOCAL ] EVTC: Loaded 19286 existing records from primary storage. [2025-09-16T18:23:16.023991] [INFO ] [MAIN ] EVTC: Loaded 19286 existing records. [2025-09-16T18:23:16.206568] [WARN ] [MAIN ] EVTC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:23:16.206966] [INFO ] [MAIN ] EVTC: Wiping existing primary data before full refresh. [2025-09-16T18:23:16.207089] [INFO ] [LOCAL ] EVTC: Starting deletion of primary data. [2025-09-16T18:23:16.207692] [DEBUG] [LOCAL ] EVTC: Deleted primary data file. [2025-09-16T18:23:16.207821] [INFO ] [MAIN ] EVTC: Starting full primary refresh. [2025-09-16T18:23:16.359590] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T18:23:16.389573] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-05. [2025-09-16T18:23:16.624291] [INFO ] [REMOTE ] EVTC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:23:16.799724] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T18:23:17.174909] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-03. [2025-09-16T18:23:17.678252] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-09-16T18:23:17.974910] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-06. [2025-09-16T18:23:18.006622] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T18:23:18.774906] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-04. [2025-09-16T18:23:19.431398] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T18:23:19.574883] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-02. [2025-09-16T18:23:19.669483] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-09-16T18:23:20.374877] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-05. [2025-09-16T18:23:20.786180] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-09-16T18:23:21.174885] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-07. [2025-09-16T18:23:21.214588] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T18:23:21.974899] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-03. [2025-09-16T18:23:22.652088] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T18:23:22.774880] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-08. [2025-09-16T18:23:23.143759] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T18:23:23.574906] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-04. [2025-09-16T18:23:24.374861] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-06. [2025-09-16T18:23:24.445780] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:23:24.991348] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T18:23:25.174893] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-05. [2025-09-16T18:23:25.208128] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-09-16T18:23:25.974878] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-07. [2025-09-16T18:23:26.080506] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T18:23:26.774906] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-09. [2025-09-16T18:23:27.353891] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-09-16T18:23:27.574877] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-08. [2025-09-16T18:23:27.779570] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T18:23:27.784983] [INFO ] [REMOTE ] EVH: Full fetch complete (53367 records). [2025-09-16T18:23:27.785094] [DEBUG] [VALIDATOR ] EVH: Validating primary data. [2025-09-16T18:23:27.785784] [FATAL] [VALIDATOR ] EVH: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:23:27.785863] [FATAL] [MAIN ] EVH: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:23:27.785901] [WARN ] [MAIN ] EVH: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:23:27.785951] [INFO ] [LOCAL ] EVH: Starting deletion of primary data. [2025-09-16T18:23:27.786014] [DEBUG] [LOCAL ] EVH: No primary data file found to delete. [2025-09-16T18:23:27.786052] [INFO ] [LOCAL ] EVH: Starting deletion of derivative data. [2025-09-16T18:23:27.786335] [DEBUG] [LOCAL ] EVH: Deleted derivative data file. [2025-09-16T18:23:27.786416] [INFO ] [MAIN ] Update Progress: 789/2369 symbols completed. EVH took 33.3s. Estimated remaining: 2h 47m 2s. Rate: 9.5 symbols/min [2025-09-16T18:23:27.964370] [INFO ] [MAIN ] EVTL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:23:27.992398] [DEBUG] [LOCAL ] EVTL: Loaded 29727 existing records from primary storage. [2025-09-16T18:23:27.992559] [INFO ] [MAIN ] EVTL: Loaded 29727 existing records. [2025-09-16T18:23:28.174585] [WARN ] [MAIN ] EVTL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:23:28.174850] [INFO ] [MAIN ] EVTL: Wiping existing primary data before full refresh. [2025-09-16T18:23:28.174897] [INFO ] [LOCAL ] EVTL: Starting deletion of primary data. [2025-09-16T18:23:28.175675] [DEBUG] [LOCAL ] EVTL: Deleted primary data file. [2025-09-16T18:23:28.175727] [INFO ] [MAIN ] EVTL: Starting full primary refresh. [2025-09-16T18:23:28.383496] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-07. [2025-09-16T18:23:28.546993] [INFO ] [REMOTE ] EVTL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:23:29.174882] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-02. [2025-09-16T18:23:29.205393] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-09-16T18:23:29.642225] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T18:23:29.974894] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-08. [2025-09-16T18:23:30.774874] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-06. [2025-09-16T18:23:30.814919] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:23:30.982275] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T18:23:31.574868] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-03. [2025-09-16T18:23:31.835307] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-09-16T18:23:32.374870] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-07. [2025-09-16T18:23:32.707735] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-09-16T18:23:33.174880] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-04. [2025-09-16T18:23:33.625938] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-09-16T18:23:33.974884] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-09. [2025-09-16T18:23:34.253686] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-09-16T18:23:34.774883] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-09. [2025-09-16T18:23:34.982874] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T18:23:34.985291] [INFO ] [REMOTE ] EVT: Full fetch complete (19236 records). [2025-09-16T18:23:34.985365] [DEBUG] [VALIDATOR ] EVT: Validating primary data. [2025-09-16T18:23:35.363174] [DEBUG] [VALIDATOR ] EVT: All expected trading days are present. [2025-09-16T18:23:35.363264] [DEBUG] [VALIDATOR ] EVT: Primary data validation passed. [2025-09-16T18:23:35.363745] [INFO ] [MAIN ] EVT: Fetched primary data is valid. [2025-09-16T18:23:35.363794] [DEBUG] [LOCAL ] EVT: Saving primary data. [2025-09-16T18:23:35.364383] [DEBUG] [LOCAL ] EVT: Trimmed 19236 records to 12703 (keeping 150 days of data) [2025-09-16T18:23:35.441834] [DEBUG] [LOCAL ] EVT: Primary data saved (12703 records). [2025-09-16T18:23:35.441979] [INFO ] [MAIN ] EVT: Full history saved. [2025-09-16T18:23:35.442064] [INFO ] [MAIN ] Update Progress: 790/2369 symbols completed. EVT took 21.1s. Estimated remaining: 2h 46m 58s. Rate: 9.5 symbols/min [2025-09-16T18:23:35.588022] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-08. [2025-09-16T18:23:35.621440] [INFO ] [MAIN ] EVTR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:23:35.639086] [DEBUG] [LOCAL ] EVTR: Loaded 17176 existing records from primary storage. [2025-09-16T18:23:35.639230] [INFO ] [MAIN ] EVTR: Loaded 17176 existing records. [2025-09-16T18:23:35.822000] [WARN ] [MAIN ] EVTR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:23:35.822307] [INFO ] [MAIN ] EVTR: Wiping existing primary data before full refresh. [2025-09-16T18:23:35.822357] [INFO ] [LOCAL ] EVTR: Starting deletion of primary data. [2025-09-16T18:23:35.822896] [DEBUG] [LOCAL ] EVTR: Deleted primary data file. [2025-09-16T18:23:35.822947] [INFO ] [MAIN ] EVTR: Starting full primary refresh. [2025-09-16T18:23:36.196459] [INFO ] [REMOTE ] EVTR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:23:36.287346] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-09-16T18:23:36.291275] [INFO ] [REMOTE ] EVR: Full fetch complete (39342 records). [2025-09-16T18:23:36.291373] [DEBUG] [VALIDATOR ] EVR: Validating primary data. [2025-09-16T18:23:36.379968] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-05. [2025-09-16T18:23:36.683911] [DEBUG] [VALIDATOR ] EVR: All expected trading days are present. [2025-09-16T18:23:36.684023] [DEBUG] [VALIDATOR ] EVR: Primary data validation passed. [2025-09-16T18:23:36.684860] [INFO ] [MAIN ] EVR: Fetched primary data is valid. [2025-09-16T18:23:36.684928] [DEBUG] [LOCAL ] EVR: Saving primary data. [2025-09-16T18:23:36.686048] [DEBUG] [LOCAL ] EVR: Trimmed 39342 records to 25174 (keeping 150 days of data) [2025-09-16T18:23:36.865027] [DEBUG] [LOCAL ] EVR: Primary data saved (25174 records). [2025-09-16T18:23:36.865466] [INFO ] [MAIN ] EVR: Full history saved. [2025-09-16T18:23:36.866013] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-09-16T18:23:36.866122] [INFO ] [MAIN ] Update Progress: 791/2369 symbols completed. EVR took 30.9s. Estimated remaining: 2h 46m 42s. Rate: 9.5 symbols/min [2025-09-16T18:23:37.046963] [INFO ] [MAIN ] EW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:23:37.087609] [DEBUG] [LOCAL ] EW: Loaded 39389 existing records from primary storage. [2025-09-16T18:23:37.087807] [INFO ] [MAIN ] EW: Loaded 39389 existing records. [2025-09-16T18:23:37.180051] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-02. [2025-09-16T18:23:37.266346] [WARN ] [MAIN ] EW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:23:37.266579] [INFO ] [MAIN ] EW: Wiping existing primary data before full refresh. [2025-09-16T18:23:37.266624] [INFO ] [LOCAL ] EW: Starting deletion of primary data. [2025-09-16T18:23:37.267483] [DEBUG] [LOCAL ] EW: Deleted primary data file. [2025-09-16T18:23:37.267535] [INFO ] [MAIN ] EW: Starting full primary refresh. [2025-09-16T18:23:37.630287] [INFO ] [REMOTE ] EW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:23:37.671341] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-09-16T18:23:37.974877] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-06. [2025-09-16T18:23:38.122048] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-09-16T18:23:38.774932] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-03. [2025-09-16T18:23:39.574881] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-02. [2025-09-16T18:23:39.724503] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-09-16T18:23:39.879504] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T18:23:40.374948] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-09. [2025-09-16T18:23:41.174871] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-04. [2025-09-16T18:23:41.258791] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T18:23:41.582282] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-09-16T18:23:41.586075] [INFO ] [REMOTE ] EVTC: Full fetch complete (28984 records). [2025-09-16T18:23:41.586173] [DEBUG] [VALIDATOR ] EVTC: Validating primary data. [2025-09-16T18:23:41.966450] [DEBUG] [VALIDATOR ] EVTC: All expected trading days are present. [2025-09-16T18:23:41.966544] [DEBUG] [VALIDATOR ] EVTC: Primary data validation passed. [2025-09-16T18:23:41.967096] [INFO ] [MAIN ] EVTC: Fetched primary data is valid. [2025-09-16T18:23:41.967213] [DEBUG] [LOCAL ] EVTC: Saving primary data. [2025-09-16T18:23:41.980403] [DEBUG] [LOCAL ] EVTC: Trimmed 28984 records to 19478 (keeping 150 days of data) [2025-09-16T18:23:41.980204] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-03. [2025-09-16T18:23:41.981695] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-09-16T18:23:42.109850] [DEBUG] [LOCAL ] EVTC: Primary data saved (19478 records). [2025-09-16T18:23:42.110011] [INFO ] [MAIN ] EVTC: Full history saved. [2025-09-16T18:23:42.110098] [INFO ] [MAIN ] Update Progress: 792/2369 symbols completed. EVTC took 26.3s. Estimated remaining: 2h 46m 34s. Rate: 9.5 symbols/min [2025-09-16T18:23:42.284094] [INFO ] [MAIN ] EXG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:23:42.284285] [DEBUG] [LOCAL ] EXG: No existing data file found in primary storage. [2025-09-16T18:23:42.284322] [WARN ] [MAIN ] EXG: No existing data found, performing full refresh instead. [2025-09-16T18:23:42.284485] [INFO ] [MAIN ] EXG: Wiping existing primary data before full refresh. [2025-09-16T18:23:42.284526] [INFO ] [LOCAL ] EXG: Starting deletion of primary data. [2025-09-16T18:23:42.284568] [DEBUG] [LOCAL ] EXG: No primary data file found to delete. [2025-09-16T18:23:42.284602] [INFO ] [MAIN ] EXG: Starting full primary refresh. [2025-09-16T18:23:42.655035] [INFO ] [REMOTE ] EXG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:23:42.774903] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-02. [2025-09-16T18:23:43.529228] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T18:23:43.574906] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-05. [2025-09-16T18:23:44.146451] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T18:23:44.374906] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-07. [2025-09-16T18:23:44.537727] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-09-16T18:23:45.174896] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-06. [2025-09-16T18:23:45.974903] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-03. [2025-09-16T18:23:46.241580] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T18:23:46.542836] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-09-16T18:23:46.774927] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-07. [2025-09-16T18:23:47.492926] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-09-16T18:23:47.519232] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-09-16T18:23:47.574908] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-08. [2025-09-16T18:23:48.374912] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-04. [2025-09-16T18:23:48.940890] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-09-16T18:23:49.174914] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-09. [2025-09-16T18:23:49.701862] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T18:23:49.974934] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-05. [2025-09-16T18:23:50.062482] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T18:23:50.067122] [INFO ] [REMOTE ] EVTR: Full fetch complete (24548 records). [2025-09-16T18:23:50.067230] [DEBUG] [VALIDATOR ] EVTR: Validating primary data. [2025-09-16T18:23:50.438291] [DEBUG] [VALIDATOR ] EVTR: All expected trading days are present. [2025-09-16T18:23:50.438390] [DEBUG] [VALIDATOR ] EVTR: Primary data validation passed. [2025-09-16T18:23:50.439016] [INFO ] [MAIN ] EVTR: Fetched primary data is valid. [2025-09-16T18:23:50.439064] [DEBUG] [LOCAL ] EVTR: Saving primary data. [2025-09-16T18:23:50.440350] [DEBUG] [LOCAL ] EVTR: Trimmed 24548 records to 17483 (keeping 150 days of data) [2025-09-16T18:23:50.545997] [DEBUG] [LOCAL ] EVTR: Primary data saved (17483 records). [2025-09-16T18:23:50.546133] [INFO ] [MAIN ] EVTR: Full history saved. [2025-09-16T18:23:50.546226] [INFO ] [MAIN ] Update Progress: 793/2369 symbols completed. EVTR took 15.1s. Estimated remaining: 2h 46m 32s. Rate: 9.5 symbols/min [2025-09-16T18:23:50.718967] [INFO ] [MAIN ] EXK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:23:50.719162] [DEBUG] [LOCAL ] EXK: No existing data file found in primary storage. [2025-09-16T18:23:50.719199] [WARN ] [MAIN ] EXK: No existing data found, performing full refresh instead. [2025-09-16T18:23:50.719321] [INFO ] [MAIN ] EXK: Wiping existing primary data before full refresh. [2025-09-16T18:23:50.719359] [INFO ] [LOCAL ] EXK: Starting deletion of primary data. [2025-09-16T18:23:50.719400] [DEBUG] [LOCAL ] EXK: No primary data file found to delete. [2025-09-16T18:23:50.719433] [INFO ] [MAIN ] EXK: Starting full primary refresh. [2025-09-16T18:23:50.781902] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-08. [2025-09-16T18:23:51.092021] [INFO ] [REMOTE ] EXK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:23:51.112090] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T18:23:51.574904] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-02. [2025-09-16T18:23:52.374929] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-06. [2025-09-16T18:23:52.588708] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-09-16T18:23:52.988300] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-09-16T18:23:53.174952] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-03. [2025-09-16T18:23:53.305323] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-09-16T18:23:53.974904] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-04. [2025-09-16T18:23:54.547323] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T18:23:54.774923] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-04. [2025-09-16T18:23:55.574946] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-09. [2025-09-16T18:23:56.257493] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T18:23:56.374900] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-05. [2025-09-16T18:23:57.174934] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-07. [2025-09-16T18:23:57.738246] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-09-16T18:23:57.974946] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-06. [2025-09-16T18:24:03.648359] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-09 (8.07 seconds). [2025-09-16T18:24:03.654066] [INFO ] [REMOTE ] EVTL: Full fetch complete (39307 records). [2025-09-16T18:24:03.654189] [DEBUG] [VALIDATOR ] EVTL: Validating primary data. [2025-09-16T18:24:04.055994] [DEBUG] [VALIDATOR ] EVTL: All expected trading days are present. [2025-09-16T18:24:04.056072] [DEBUG] [VALIDATOR ] EVTL: Primary data validation passed. [2025-09-16T18:24:04.056810] [INFO ] [MAIN ] EVTL: Fetched primary data is valid. [2025-09-16T18:24:04.056864] [DEBUG] [LOCAL ] EVTL: Saving primary data. [2025-09-16T18:24:04.057812] [DEBUG] [LOCAL ] EVTL: Trimmed 39307 records to 30107 (keeping 150 days of data) [2025-09-16T18:24:04.225392] [DEBUG] [LOCAL ] EVTL: Primary data saved (30107 records). [2025-09-16T18:24:04.225538] [INFO ] [MAIN ] EVTL: Full history saved. [2025-09-16T18:24:04.225624] [INFO ] [MAIN ] Update Progress: 794/2369 symbols completed. EVTL took 36.4s. Estimated remaining: 2h 46m 40s. Rate: 9.5 symbols/min [2025-09-16T18:24:04.394397] [INFO ] [MAIN ] EXP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:04.414941] [DEBUG] [LOCAL ] EXP: Loaded 21794 existing records from primary storage. [2025-09-16T18:24:04.415076] [INFO ] [MAIN ] EXP: Loaded 21794 existing records. [2025-09-16T18:24:04.594902] [WARN ] [MAIN ] EXP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:04.595200] [INFO ] [MAIN ] EXP: Wiping existing primary data before full refresh. [2025-09-16T18:24:04.595260] [INFO ] [LOCAL ] EXP: Starting deletion of primary data. [2025-09-16T18:24:04.595986] [DEBUG] [LOCAL ] EXP: Deleted primary data file. [2025-09-16T18:24:04.596037] [INFO ] [MAIN ] EXP: Starting full primary refresh. [2025-09-16T18:24:04.945700] [INFO ] [REMOTE ] EXP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:04.945994] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-02. [2025-09-16T18:24:05.983258] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T18:24:05.989210] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-03. [2025-09-16T18:24:06.798990] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-07 (9.62 seconds). [2025-09-16T18:24:06.808456] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-08. [2025-09-16T18:24:06.971260] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-04 (13.00 seconds). [2025-09-16T18:24:06.982400] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-05. [2025-09-16T18:24:07.828179] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-09-16T18:24:07.838109] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-09. [2025-09-16T18:24:08.677360] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-09-16T18:24:08.689489] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-06. [2025-09-16T18:24:10.129636] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T18:24:10.140333] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-07. [2025-09-16T18:24:10.721564] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-09 (2.88 seconds). [2025-09-16T18:24:10.727295] [INFO ] [REMOTE ] EXG: Full fetch complete (37741 records). [2025-09-16T18:24:10.727405] [DEBUG] [VALIDATOR ] EXG: Validating primary data. [2025-09-16T18:24:11.123470] [DEBUG] [VALIDATOR ] EXG: All expected trading days are present. [2025-09-16T18:24:11.125874] [DEBUG] [VALIDATOR ] EXG: Primary data validation passed. [2025-09-16T18:24:11.126871] [INFO ] [MAIN ] EXG: Fetched primary data is valid. [2025-09-16T18:24:11.127120] [DEBUG] [LOCAL ] EXG: Saving primary data. [2025-09-16T18:24:11.128832] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-03 (5.14 seconds). [2025-09-16T18:24:11.136773] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-04. [2025-09-16T18:24:11.137132] [DEBUG] [LOCAL ] EXG: Trimmed 37741 records to 25052 (keeping 150 days of data) [2025-09-16T18:24:11.286494] [DEBUG] [LOCAL ] EXG: Primary data saved (25052 records). [2025-09-16T18:24:11.286652] [INFO ] [MAIN ] EXG: Full history saved. [2025-09-16T18:24:11.286733] [INFO ] [MAIN ] Update Progress: 795/2369 symbols completed. EXG took 29.2s. Estimated remaining: 2h 46m 35s. Rate: 9.4 symbols/min [2025-09-16T18:24:11.454569] [INFO ] [MAIN ] EXPD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:11.494044] [DEBUG] [LOCAL ] EXPD: Loaded 35289 existing records from primary storage. [2025-09-16T18:24:11.494193] [INFO ] [MAIN ] EXPD: Loaded 35289 existing records. [2025-09-16T18:24:11.674845] [WARN ] [MAIN ] EXPD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:11.675142] [INFO ] [MAIN ] EXPD: Wiping existing primary data before full refresh. [2025-09-16T18:24:11.675192] [INFO ] [LOCAL ] EXPD: Starting deletion of primary data. [2025-09-16T18:24:11.675999] [DEBUG] [LOCAL ] EXPD: Deleted primary data file. [2025-09-16T18:24:11.676058] [INFO ] [MAIN ] EXPD: Starting full primary refresh. [2025-09-16T18:24:12.009327] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-06 (14.03 seconds). [2025-09-16T18:24:12.080867] [INFO ] [REMOTE ] EXPD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:12.082118] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-02. [2025-09-16T18:24:12.082545] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-07. [2025-09-16T18:24:12.119569] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-09-16T18:24:12.132071] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-08. [2025-09-16T18:24:12.400931] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-09-16T18:24:12.408010] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-05. [2025-09-16T18:24:13.247633] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T18:24:13.259437] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-09. [2025-09-16T18:24:13.388717] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-09-16T18:24:13.398379] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-06. [2025-09-16T18:24:13.498131] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T18:24:13.507456] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-03. [2025-09-16T18:24:13.810326] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T18:24:13.821870] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-08. [2025-09-16T18:24:14.396862] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T18:24:14.403760] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-07. [2025-09-16T18:24:14.513330] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-09-16T18:24:14.519879] [INFO ] [REMOTE ] EW: Full fetch complete (58063 records). [2025-09-16T18:24:14.520005] [DEBUG] [VALIDATOR ] EW: Validating primary data. [2025-09-16T18:24:14.935626] [DEBUG] [VALIDATOR ] EW: All expected trading days are present. [2025-09-16T18:24:14.935867] [DEBUG] [VALIDATOR ] EW: Primary data validation passed. [2025-09-16T18:24:14.936827] [INFO ] [MAIN ] EW: Fetched primary data is valid. [2025-09-16T18:24:14.937016] [DEBUG] [LOCAL ] EW: Saving primary data. [2025-09-16T18:24:14.938664] [DEBUG] [LOCAL ] EW: Trimmed 58063 records to 39779 (keeping 150 days of data) [2025-09-16T18:24:15.061661] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T18:24:15.075943] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-04. [2025-09-16T18:24:15.223506] [DEBUG] [LOCAL ] EW: Primary data saved (39779 records). [2025-09-16T18:24:15.223692] [INFO ] [MAIN ] EW: Full history saved. [2025-09-16T18:24:15.224774] [INFO ] [MAIN ] Update Progress: 796/2369 symbols completed. EW took 38.4s. Estimated remaining: 2h 46m 24s. Rate: 9.5 symbols/min [2025-09-16T18:24:15.401872] [INFO ] [MAIN ] EXR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:15.439476] [DEBUG] [LOCAL ] EXR: Loaded 33469 existing records from primary storage. [2025-09-16T18:24:15.439818] [INFO ] [MAIN ] EXR: Loaded 33469 existing records. [2025-09-16T18:24:15.575904] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T18:24:15.613788] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-09. [2025-09-16T18:24:15.664894] [WARN ] [MAIN ] EXR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:15.665123] [INFO ] [MAIN ] EXR: Wiping existing primary data before full refresh. [2025-09-16T18:24:15.665187] [INFO ] [LOCAL ] EXR: Starting deletion of primary data. [2025-09-16T18:24:15.665990] [DEBUG] [LOCAL ] EXR: Deleted primary data file. [2025-09-16T18:24:15.666049] [INFO ] [MAIN ] EXR: Starting full primary refresh. [2025-09-16T18:24:16.019207] [INFO ] [REMOTE ] EXR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:16.019532] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-02. [2025-09-16T18:24:16.497319] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-09-16T18:24:16.505138] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-08. [2025-09-16T18:24:17.019670] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T18:24:17.025500] [INFO ] [REMOTE ] EXK: Full fetch complete (57224 records). [2025-09-16T18:24:17.025600] [DEBUG] [VALIDATOR ] EXK: Validating primary data. [2025-09-16T18:24:17.026306] [FATAL] [VALIDATOR ] EXK: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:24:17.026383] [FATAL] [MAIN ] EXK: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:24:17.026420] [WARN ] [MAIN ] EXK: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:24:17.026458] [INFO ] [LOCAL ] EXK: Starting deletion of primary data. [2025-09-16T18:24:17.026515] [DEBUG] [LOCAL ] EXK: No primary data file found to delete. [2025-09-16T18:24:17.026551] [INFO ] [LOCAL ] EXK: Starting deletion of derivative data. [2025-09-16T18:24:17.026590] [DEBUG] [LOCAL ] EXK: No derivative data file found to delete. [2025-09-16T18:24:17.026688] [INFO ] [MAIN ] Update Progress: 797/2369 symbols completed. EXK took 26.5s. Estimated remaining: 2h 46m 8s. Rate: 9.5 symbols/min [2025-09-16T18:24:17.094465] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-09-16T18:24:17.104923] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-05. [2025-09-16T18:24:17.455336] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T18:24:17.465398] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-03. [2025-09-16T18:24:17.584389] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T18:24:17.591045] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-09. [2025-09-16T18:24:18.826226] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-09-16T18:24:18.837210] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-06. [2025-09-16T18:24:18.846747] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T18:24:18.850481] [INFO ] [REMOTE ] EXP: Full fetch complete (32680 records). [2025-09-16T18:24:18.850630] [DEBUG] [VALIDATOR ] EXP: Validating primary data. [2025-09-16T18:24:18.851228] [FATAL] [VALIDATOR ] EXP: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:24:18.851359] [FATAL] [MAIN ] EXP: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:24:18.851406] [WARN ] [MAIN ] EXP: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:24:18.860986] [INFO ] [LOCAL ] EXP: Starting deletion of primary data. [2025-09-16T18:24:18.877215] [DEBUG] [LOCAL ] EXP: No primary data file found to delete. [2025-09-16T18:24:18.877307] [INFO ] [LOCAL ] EXP: Starting deletion of derivative data. [2025-09-16T18:24:18.879031] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-09-16T18:24:18.879074] [DEBUG] [LOCAL ] EXP: Deleted derivative data file. [2025-09-16T18:24:18.888391] [INFO ] [MAIN ] Update Progress: 798/2369 symbols completed. EXP took 14.7s. Estimated remaining: 2h 45m 53s. Rate: 9.5 symbols/min [2025-09-16T18:24:18.889284] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-04. [2025-09-16T18:24:20.418066] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T18:24:20.428087] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-07. [2025-09-16T18:24:20.474412] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-09-16T18:24:20.484798] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-05. [2025-09-16T18:24:22.113887] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-09-16T18:24:22.123562] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-06. [2025-09-16T18:24:22.203894] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-09-16T18:24:22.214724] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-08. [2025-09-16T18:24:23.193010] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T18:24:23.202401] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-07. [2025-09-16T18:24:23.941169] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-09-16T18:24:23.951510] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-09. [2025-09-16T18:24:24.907762] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T18:24:24.917808] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-08. [2025-09-16T18:24:25.387718] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T18:24:25.393701] [INFO ] [REMOTE ] EXPD: Full fetch complete (52515 records). [2025-09-16T18:24:25.393880] [DEBUG] [VALIDATOR ] EXPD: Validating primary data. [2025-09-16T18:24:25.788966] [DEBUG] [VALIDATOR ] EXPD: All expected trading days are present. [2025-09-16T18:24:25.789119] [DEBUG] [VALIDATOR ] EXPD: Primary data validation passed. [2025-09-16T18:24:25.789864] [INFO ] [MAIN ] EXPD: Fetched primary data is valid. [2025-09-16T18:24:25.789912] [DEBUG] [LOCAL ] EXPD: Saving primary data. [2025-09-16T18:24:25.791147] [DEBUG] [LOCAL ] EXPD: Trimmed 52515 records to 35628 (keeping 150 days of data) [2025-09-16T18:24:26.013657] [DEBUG] [LOCAL ] EXPD: Primary data saved (35628 records). [2025-09-16T18:24:26.013813] [INFO ] [MAIN ] EXPD: Full history saved. [2025-09-16T18:24:26.013919] [INFO ] [MAIN ] Update Progress: 799/2369 symbols completed. EXPD took 14.7s. Estimated remaining: 2h 45m 48s. Rate: 9.5 symbols/min [2025-09-16T18:24:26.500320] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T18:24:26.510626] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-09. [2025-09-16T18:24:28.046022] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T18:24:28.052007] [INFO ] [REMOTE ] EXR: Full fetch complete (49591 records). [2025-09-16T18:24:28.052187] [DEBUG] [VALIDATOR ] EXR: Validating primary data. [2025-09-16T18:24:28.428086] [DEBUG] [VALIDATOR ] EXR: All expected trading days are present. [2025-09-16T18:24:28.428164] [DEBUG] [VALIDATOR ] EXR: Primary data validation passed. [2025-09-16T18:24:28.428934] [INFO ] [MAIN ] EXR: Fetched primary data is valid. [2025-09-16T18:24:28.428981] [DEBUG] [LOCAL ] EXR: Saving primary data. [2025-09-16T18:24:28.430025] [DEBUG] [LOCAL ] EXR: Trimmed 49591 records to 33814 (keeping 150 days of data) [2025-09-16T18:24:28.642986] [DEBUG] [LOCAL ] EXR: Primary data saved (33814 records). [2025-09-16T18:24:28.643136] [INFO ] [MAIN ] EXR: Full history saved. [2025-09-16T18:24:28.647097] [INFO ] [MAIN ] Update Progress: 800/2369 symbols completed. EXR took 13.4s. Estimated remaining: 2h 45m 35s. Rate: 9.5 symbols/min [2025-09-16T18:24:29.244148] [INFO ] [MAIN ] FBK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:29.274124] [INFO ] [MAIN ] F: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:29.327390] [INFO ] [MAIN ] FAF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:29.325350] [DEBUG] [LOCAL ] FBK: No existing data file found in primary storage. [2025-09-16T18:24:29.335706] [WARN ] [MAIN ] FBK: No existing data found, performing full refresh instead. [2025-09-16T18:24:29.354261] [INFO ] [MAIN ] FBIN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:29.354873] [INFO ] [MAIN ] FBK: Wiping existing primary data before full refresh. [2025-09-16T18:24:29.355213] [INFO ] [LOCAL ] FBK: Starting deletion of primary data. [2025-09-16T18:24:29.355269] [DEBUG] [LOCAL ] FBK: No primary data file found to delete. [2025-09-16T18:24:29.355304] [INFO ] [MAIN ] FBK: Starting full primary refresh. [2025-09-16T18:24:29.677663] [DEBUG] [LOCAL ] F: Loaded 39780 existing records from primary storage. [2025-09-16T18:24:29.676681] [DEBUG] [LOCAL ] FAF: Loaded 31072 existing records from primary storage. [2025-09-16T18:24:29.698721] [INFO ] [MAIN ] F: Loaded 39780 existing records. [2025-09-16T18:24:29.698666] [INFO ] [MAIN ] FAF: Loaded 31072 existing records. [2025-09-16T18:24:29.709667] [DEBUG] [LOCAL ] FBIN: Loaded 37347 existing records from primary storage. [2025-09-16T18:24:29.806567] [INFO ] [MAIN ] FBIN: Loaded 37347 existing records. [2025-09-16T18:24:30.072481] [INFO ] [REMOTE ] FBK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:30.117427] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-02. [2025-09-16T18:24:30.225607] [WARN ] [MAIN ] F: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:30.360340] [INFO ] [MAIN ] F: Wiping existing primary data before full refresh. [2025-09-16T18:24:30.365993] [WARN ] [MAIN ] FAF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:30.369884] [INFO ] [LOCAL ] F: Starting deletion of primary data. [2025-09-16T18:24:30.375323] [INFO ] [MAIN ] FAF: Wiping existing primary data before full refresh. [2025-09-16T18:24:30.375238] [WARN ] [MAIN ] FBIN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:30.375458] [INFO ] [LOCAL ] FAF: Starting deletion of primary data. [2025-09-16T18:24:30.375828] [INFO ] [MAIN ] FBIN: Wiping existing primary data before full refresh. [2025-09-16T18:24:30.376082] [INFO ] [LOCAL ] FBIN: Starting deletion of primary data. [2025-09-16T18:24:30.376873] [DEBUG] [LOCAL ] FAF: Deleted primary data file. [2025-09-16T18:24:30.376950] [INFO ] [MAIN ] FAF: Starting full primary refresh. [2025-09-16T18:24:30.388599] [DEBUG] [LOCAL ] F: Deleted primary data file. [2025-09-16T18:24:30.388802] [DEBUG] [LOCAL ] FBIN: Deleted primary data file. [2025-09-16T18:24:30.441368] [INFO ] [MAIN ] FBIN: Starting full primary refresh. [2025-09-16T18:24:30.446867] [INFO ] [MAIN ] F: Starting full primary refresh. [2025-09-16T18:24:31.332892] [INFO ] [REMOTE ] FAF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:31.350288] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-02. [2025-09-16T18:24:31.507966] [INFO ] [REMOTE ] F: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:31.530128] [INFO ] [REMOTE ] F: Fetching primary data for 2025-02. [2025-09-16T18:24:31.568029] [INFO ] [REMOTE ] FBIN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:31.575566] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-02. [2025-09-16T18:24:31.584407] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T18:24:31.588806] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-03. [2025-09-16T18:24:32.573896] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-09-16T18:24:32.578432] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-04. [2025-09-16T18:24:32.914811] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T18:24:32.921563] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-03. [2025-09-16T18:24:33.307806] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T18:24:33.317696] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-03. [2025-09-16T18:24:33.666531] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-02 (2.14 seconds). [2025-09-16T18:24:33.676659] [INFO ] [REMOTE ] F: Fetching primary data for 2025-03. [2025-09-16T18:24:33.980535] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T18:24:33.987180] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-05. [2025-09-16T18:24:34.098609] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-09-16T18:24:34.107279] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-04. [2025-09-16T18:24:35.081709] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T18:24:35.092210] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-04. [2025-09-16T18:24:35.105530] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T18:24:35.580000] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-06. [2025-09-16T18:24:35.594507] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T18:24:35.707658] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-09-16T18:24:36.374864] [INFO ] [REMOTE ] F: Fetching primary data for 2025-04. [2025-09-16T18:24:36.549487] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T18:24:36.777132] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T18:24:37.174898] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-05. [2025-09-16T18:24:37.974909] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-05. [2025-09-16T18:24:38.697469] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-04 (2.32 seconds). [2025-09-16T18:24:38.746851] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T18:24:38.774863] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-06. [2025-09-16T18:24:39.574923] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-07. [2025-09-16T18:24:39.691990] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-09-16T18:24:39.885934] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-09-16T18:24:40.374915] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-06. [2025-09-16T18:24:41.057635] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T18:24:41.174881] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-08. [2025-09-16T18:24:41.587020] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T18:24:41.974930] [INFO ] [REMOTE ] F: Fetching primary data for 2025-05. [2025-09-16T18:24:42.229802] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T18:24:42.774915] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-07. [2025-09-16T18:24:43.574905] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-09. [2025-09-16T18:24:44.312904] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-05 (2.34 seconds). [2025-09-16T18:24:44.374903] [INFO ] [REMOTE ] F: Fetching primary data for 2025-06. [2025-09-16T18:24:44.450626] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-09-16T18:24:44.594135] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T18:24:44.597423] [INFO ] [REMOTE ] FBK: Full fetch complete (26441 records). [2025-09-16T18:24:44.597556] [DEBUG] [VALIDATOR ] FBK: Validating primary data. [2025-09-16T18:24:44.967698] [DEBUG] [VALIDATOR ] FBK: All expected trading days are present. [2025-09-16T18:24:44.967813] [DEBUG] [VALIDATOR ] FBK: Primary data validation passed. [2025-09-16T18:24:44.968455] [INFO ] [MAIN ] FBK: Fetched primary data is valid. [2025-09-16T18:24:44.968531] [DEBUG] [LOCAL ] FBK: Saving primary data. [2025-09-16T18:24:44.970109] [DEBUG] [LOCAL ] FBK: Trimmed 26441 records to 19358 (keeping 150 days of data) [2025-09-16T18:24:45.092486] [DEBUG] [LOCAL ] FBK: Primary data saved (19358 records). [2025-09-16T18:24:45.092630] [INFO ] [MAIN ] FBK: Full history saved. [2025-09-16T18:24:45.092728] [INFO ] [MAIN ] Update Progress: 801/2369 symbols completed. FBK took 16.4s. Estimated remaining: 2h 45m 48s. Rate: 9.5 symbols/min [2025-09-16T18:24:45.184066] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-08. [2025-09-16T18:24:45.271602] [INFO ] [MAIN ] FBP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:45.309124] [DEBUG] [LOCAL ] FBP: Loaded 33317 existing records from primary storage. [2025-09-16T18:24:45.309313] [INFO ] [MAIN ] FBP: Loaded 33317 existing records. [2025-09-16T18:24:45.486919] [WARN ] [MAIN ] FBP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:45.487177] [INFO ] [MAIN ] FBP: Wiping existing primary data before full refresh. [2025-09-16T18:24:45.487224] [INFO ] [LOCAL ] FBP: Starting deletion of primary data. [2025-09-16T18:24:45.488036] [DEBUG] [LOCAL ] FBP: Deleted primary data file. [2025-09-16T18:24:45.488121] [INFO ] [MAIN ] FBP: Starting full primary refresh. [2025-09-16T18:24:45.840069] [INFO ] [REMOTE ] FBP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:45.974926] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-02. [2025-09-16T18:24:46.566387] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T18:24:46.603349] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-06 (2.23 seconds). [2025-09-16T18:24:46.774904] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-09. [2025-09-16T18:24:47.495935] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T18:24:47.574922] [INFO ] [REMOTE ] F: Fetching primary data for 2025-07. [2025-09-16T18:24:48.046057] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T18:24:48.051715] [INFO ] [REMOTE ] FAF: Full fetch complete (44947 records). [2025-09-16T18:24:48.051838] [DEBUG] [VALIDATOR ] FAF: Validating primary data. [2025-09-16T18:24:48.052482] [FATAL] [VALIDATOR ] FAF: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:24:48.052558] [FATAL] [MAIN ] FAF: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:24:48.052610] [WARN ] [MAIN ] FAF: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:24:48.052677] [INFO ] [LOCAL ] FAF: Starting deletion of primary data. [2025-09-16T18:24:48.052737] [DEBUG] [LOCAL ] FAF: No primary data file found to delete. [2025-09-16T18:24:48.052772] [INFO ] [LOCAL ] FAF: Starting deletion of derivative data. [2025-09-16T18:24:48.053082] [DEBUG] [LOCAL ] FAF: Deleted derivative data file. [2025-09-16T18:24:48.053178] [INFO ] [MAIN ] Update Progress: 802/2369 symbols completed. FAF took 19.4s. Estimated remaining: 2h 45m 35s. Rate: 9.5 symbols/min [2025-09-16T18:24:48.227262] [INFO ] [MAIN ] FBRT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:48.257303] [DEBUG] [LOCAL ] FBRT: Loaded 24991 existing records from primary storage. [2025-09-16T18:24:48.257442] [INFO ] [MAIN ] FBRT: Loaded 24991 existing records. [2025-09-16T18:24:48.380061] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-07. [2025-09-16T18:24:48.434231] [WARN ] [MAIN ] FBRT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:48.434494] [INFO ] [MAIN ] FBRT: Wiping existing primary data before full refresh. [2025-09-16T18:24:48.434538] [INFO ] [LOCAL ] FBRT: Starting deletion of primary data. [2025-09-16T18:24:48.435180] [DEBUG] [LOCAL ] FBRT: Deleted primary data file. [2025-09-16T18:24:48.435229] [INFO ] [MAIN ] FBRT: Starting full primary refresh. [2025-09-16T18:24:48.793881] [INFO ] [REMOTE ] FBRT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:49.174889] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-03. [2025-09-16T18:24:49.972780] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T18:24:49.983335] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-02. [2025-09-16T18:24:50.278372] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-07 (2.70 seconds). [2025-09-16T18:24:50.774922] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-08. [2025-09-16T18:24:50.957413] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T18:24:51.025293] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T18:24:51.574926] [INFO ] [REMOTE ] F: Fetching primary data for 2025-08. [2025-09-16T18:24:52.374948] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-03. [2025-09-16T18:24:52.642238] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-09-16T18:24:53.174890] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-09. [2025-09-16T18:24:53.509503] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-09-16T18:24:53.974952] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-04. [2025-09-16T18:24:54.064311] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-08 (2.49 seconds). [2025-09-16T18:24:54.774918] [INFO ] [REMOTE ] F: Fetching primary data for 2025-09. [2025-09-16T18:24:54.865934] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-09-16T18:24:54.873134] [INFO ] [REMOTE ] FBIN: Full fetch complete (54879 records). [2025-09-16T18:24:54.873433] [DEBUG] [VALIDATOR ] FBIN: Validating primary data. [2025-09-16T18:24:55.295664] [DEBUG] [VALIDATOR ] FBIN: All expected trading days are present. [2025-09-16T18:24:55.295775] [DEBUG] [VALIDATOR ] FBIN: Primary data validation passed. [2025-09-16T18:24:55.296764] [INFO ] [MAIN ] FBIN: Fetched primary data is valid. [2025-09-16T18:24:55.296821] [DEBUG] [LOCAL ] FBIN: Saving primary data. [2025-09-16T18:24:55.298407] [DEBUG] [LOCAL ] FBIN: Trimmed 54879 records to 37713 (keeping 150 days of data) [2025-09-16T18:24:55.559812] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T18:24:55.577669] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-05. [2025-09-16T18:24:55.593563] [DEBUG] [LOCAL ] FBIN: Primary data saved (37713 records). [2025-09-16T18:24:55.593830] [INFO ] [MAIN ] FBIN: Full history saved. [2025-09-16T18:24:55.593989] [INFO ] [MAIN ] Update Progress: 803/2369 symbols completed. FBIN took 26.9s. Estimated remaining: 2h 45m 31s. Rate: 9.5 symbols/min [2025-09-16T18:24:55.798846] [INFO ] [MAIN ] FC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:55.814222] [DEBUG] [LOCAL ] FC: Loaded 10433 existing records from primary storage. [2025-09-16T18:24:55.814460] [INFO ] [MAIN ] FC: Loaded 10433 existing records. [2025-09-16T18:24:56.021856] [WARN ] [MAIN ] FC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:56.022252] [INFO ] [MAIN ] FC: Wiping existing primary data before full refresh. [2025-09-16T18:24:56.022305] [INFO ] [LOCAL ] FC: Starting deletion of primary data. [2025-09-16T18:24:56.022806] [DEBUG] [LOCAL ] FC: Deleted primary data file. [2025-09-16T18:24:56.022858] [INFO ] [MAIN ] FC: Starting full primary refresh. [2025-09-16T18:24:56.380142] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-04. [2025-09-16T18:24:56.416625] [INFO ] [REMOTE ] FC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:56.761054] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-09-16T18:24:56.959106] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-09-16T18:24:56.968621] [INFO ] [REMOTE ] F: Full fetch complete (58500 records). [2025-09-16T18:24:56.968807] [DEBUG] [VALIDATOR ] F: Validating primary data. [2025-09-16T18:24:57.181069] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-06. [2025-09-16T18:24:57.362627] [DEBUG] [VALIDATOR ] F: All expected trading days are present. [2025-09-16T18:24:57.362730] [DEBUG] [VALIDATOR ] F: Primary data validation passed. [2025-09-16T18:24:57.363522] [INFO ] [MAIN ] F: Fetched primary data is valid. [2025-09-16T18:24:57.363566] [DEBUG] [LOCAL ] F: Saving primary data. [2025-09-16T18:24:57.365149] [DEBUG] [LOCAL ] F: Trimmed 58500 records to 40170 (keeping 150 days of data) [2025-09-16T18:24:57.620465] [DEBUG] [LOCAL ] F: Primary data saved (40170 records). [2025-09-16T18:24:57.620594] [INFO ] [MAIN ] F: Full history saved. [2025-09-16T18:24:57.620723] [INFO ] [MAIN ] Update Progress: 804/2369 symbols completed. F took 29.0s. Estimated remaining: 2h 45m 17s. Rate: 9.5 symbols/min [2025-09-16T18:24:57.794621] [INFO ] [MAIN ] FCF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:24:57.827938] [DEBUG] [LOCAL ] FCF: Loaded 28033 existing records from primary storage. [2025-09-16T18:24:57.828158] [INFO ] [MAIN ] FCF: Loaded 28033 existing records. [2025-09-16T18:24:57.980072] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-02. [2025-09-16T18:24:58.006987] [WARN ] [MAIN ] FCF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:24:58.007209] [INFO ] [MAIN ] FCF: Wiping existing primary data before full refresh. [2025-09-16T18:24:58.007265] [INFO ] [LOCAL ] FCF: Starting deletion of primary data. [2025-09-16T18:24:58.007907] [DEBUG] [LOCAL ] FCF: Deleted primary data file. [2025-09-16T18:24:58.007955] [INFO ] [MAIN ] FCF: Starting full primary refresh. [2025-09-16T18:24:58.359524] [INFO ] [REMOTE ] FCF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:24:58.381055] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T18:24:58.416521] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-09-16T18:24:58.774900] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-07. [2025-09-16T18:24:59.107946] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-09-16T18:24:59.574924] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-02. [2025-09-16T18:25:00.374980] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-05. [2025-09-16T18:25:00.583490] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T18:25:00.966665] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T18:25:01.174978] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-08. [2025-09-16T18:25:01.524669] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T18:25:01.974991] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-03. [2025-09-16T18:25:02.592172] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T18:25:02.774947] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-03. [2025-09-16T18:25:03.128803] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T18:25:03.497558] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-09-16T18:25:03.574914] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-04. [2025-09-16T18:25:04.374912] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-06. [2025-09-16T18:25:05.174862] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-09. [2025-09-16T18:25:05.252262] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-09-16T18:25:05.591784] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-09-16T18:25:05.974921] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-05. [2025-09-16T18:25:06.000310] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-09-16T18:25:06.004238] [INFO ] [REMOTE ] FBRT: Full fetch complete (37205 records). [2025-09-16T18:25:06.004341] [DEBUG] [VALIDATOR ] FBRT: Validating primary data. [2025-09-16T18:25:06.378382] [DEBUG] [VALIDATOR ] FBRT: All expected trading days are present. [2025-09-16T18:25:06.378459] [DEBUG] [VALIDATOR ] FBRT: Primary data validation passed. [2025-09-16T18:25:06.379093] [INFO ] [MAIN ] FBRT: Fetched primary data is valid. [2025-09-16T18:25:06.379139] [DEBUG] [LOCAL ] FBRT: Saving primary data. [2025-09-16T18:25:06.379944] [DEBUG] [LOCAL ] FBRT: Trimmed 37205 records to 25233 (keeping 150 days of data) [2025-09-16T18:25:06.541264] [DEBUG] [LOCAL ] FBRT: Primary data saved (25233 records). [2025-09-16T18:25:06.541397] [INFO ] [MAIN ] FBRT: Full history saved. [2025-09-16T18:25:06.541481] [INFO ] [MAIN ] Update Progress: 805/2369 symbols completed. FBRT took 18.5s. Estimated remaining: 2h 45m 15s. Rate: 9.5 symbols/min [2025-09-16T18:25:06.712175] [INFO ] [MAIN ] FCN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:06.712355] [DEBUG] [LOCAL ] FCN: No existing data file found in primary storage. [2025-09-16T18:25:06.712395] [WARN ] [MAIN ] FCN: No existing data found, performing full refresh instead. [2025-09-16T18:25:06.712516] [INFO ] [MAIN ] FCN: Wiping existing primary data before full refresh. [2025-09-16T18:25:06.712556] [INFO ] [LOCAL ] FCN: Starting deletion of primary data. [2025-09-16T18:25:06.712598] [DEBUG] [LOCAL ] FCN: No primary data file found to delete. [2025-09-16T18:25:06.712631] [INFO ] [MAIN ] FCN: Starting full primary refresh. [2025-09-16T18:25:06.780016] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-04. [2025-09-16T18:25:07.071392] [INFO ] [REMOTE ] FCN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:25:07.094719] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T18:25:07.537321] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-09-16T18:25:07.574878] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-05. [2025-09-16T18:25:08.264191] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-09-16T18:25:08.374920] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-06. [2025-09-16T18:25:08.986194] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-09-16T18:25:09.174862] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-07. [2025-09-16T18:25:09.974905] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-06. [2025-09-16T18:25:10.125900] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-09-16T18:25:10.774910] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-08. [2025-09-16T18:25:11.012519] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-09-16T18:25:11.371501] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T18:25:11.574903] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-02. [2025-09-16T18:25:12.374925] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-07. [2025-09-16T18:25:12.500076] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-09-16T18:25:13.174931] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-09. [2025-09-16T18:25:13.685558] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T18:25:13.816892] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-09-16T18:25:13.819221] [INFO ] [REMOTE ] FC: Full fetch complete (16654 records). [2025-09-16T18:25:13.819308] [DEBUG] [VALIDATOR ] FC: Validating primary data. [2025-09-16T18:25:13.819861] [FATAL] [VALIDATOR ] FC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:25:13.819948] [FATAL] [MAIN ] FC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:25:13.819990] [WARN ] [MAIN ] FC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:25:13.820034] [INFO ] [LOCAL ] FC: Starting deletion of primary data. [2025-09-16T18:25:13.820103] [DEBUG] [LOCAL ] FC: No primary data file found to delete. [2025-09-16T18:25:13.820150] [INFO ] [LOCAL ] FC: Starting deletion of derivative data. [2025-09-16T18:25:13.820632] [DEBUG] [LOCAL ] FC: Deleted derivative data file. [2025-09-16T18:25:13.820994] [INFO ] [MAIN ] Update Progress: 806/2369 symbols completed. FC took 18.2s. Estimated remaining: 2h 45m 11s. Rate: 9.5 symbols/min [2025-09-16T18:25:13.975113] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-07. [2025-09-16T18:25:13.993890] [INFO ] [MAIN ] FCPT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:14.020404] [DEBUG] [LOCAL ] FCPT: Loaded 26579 existing records from primary storage. [2025-09-16T18:25:14.020572] [INFO ] [MAIN ] FCPT: Loaded 26579 existing records. [2025-09-16T18:25:14.206828] [WARN ] [MAIN ] FCPT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:25:14.207125] [INFO ] [MAIN ] FCPT: Wiping existing primary data before full refresh. [2025-09-16T18:25:14.207190] [INFO ] [LOCAL ] FCPT: Starting deletion of primary data. [2025-09-16T18:25:14.208070] [DEBUG] [LOCAL ] FCPT: Deleted primary data file. [2025-09-16T18:25:14.208148] [INFO ] [MAIN ] FCPT: Starting full primary refresh. [2025-09-16T18:25:14.601215] [INFO ] [REMOTE ] FCPT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:25:14.774927] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-08. [2025-09-16T18:25:15.274542] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-09-16T18:25:15.574940] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-03. [2025-09-16T18:25:15.940978] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T18:25:16.374895] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-09. [2025-09-16T18:25:16.694881] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-09-16T18:25:17.174881] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-04. [2025-09-16T18:25:17.404375] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-09-16T18:25:17.409156] [INFO ] [REMOTE ] FBP: Full fetch complete (49924 records). [2025-09-16T18:25:17.409255] [DEBUG] [VALIDATOR ] FBP: Validating primary data. [2025-09-16T18:25:17.409925] [FATAL] [VALIDATOR ] FBP: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:25:17.410004] [FATAL] [MAIN ] FBP: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:25:17.410053] [WARN ] [MAIN ] FBP: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:25:17.410099] [INFO ] [LOCAL ] FBP: Starting deletion of primary data. [2025-09-16T18:25:17.410179] [DEBUG] [LOCAL ] FBP: No primary data file found to delete. [2025-09-16T18:25:17.410217] [INFO ] [LOCAL ] FBP: Starting deletion of derivative data. [2025-09-16T18:25:17.410448] [DEBUG] [LOCAL ] FBP: Deleted derivative data file. [2025-09-16T18:25:17.411748] [INFO ] [MAIN ] Update Progress: 807/2369 symbols completed. FBP took 32.3s. Estimated remaining: 2h 44m 59s. Rate: 9.5 symbols/min [2025-09-16T18:25:17.580926] [INFO ] [MAIN ] FCRX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:17.581104] [DEBUG] [LOCAL ] FCRX: No existing data file found in primary storage. [2025-09-16T18:25:17.581143] [WARN ] [MAIN ] FCRX: No existing data found, performing full refresh instead. [2025-09-16T18:25:17.581264] [INFO ] [MAIN ] FCRX: Wiping existing primary data before full refresh. [2025-09-16T18:25:17.581302] [INFO ] [LOCAL ] FCRX: Starting deletion of primary data. [2025-09-16T18:25:17.581346] [DEBUG] [LOCAL ] FCRX: No primary data file found to delete. [2025-09-16T18:25:17.581407] [INFO ] [MAIN ] FCRX: Starting full primary refresh. [2025-09-16T18:25:17.935313] [INFO ] [REMOTE ] FCRX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:25:17.974860] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-02. [2025-09-16T18:25:18.338716] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-09-16T18:25:18.434249] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-09-16T18:25:18.774901] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-02. [2025-09-16T18:25:19.574899] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-03. [2025-09-16T18:25:20.148908] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-09-16T18:25:20.170787] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-09-16T18:25:20.374893] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-05. [2025-09-16T18:25:21.174924] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-04. [2025-09-16T18:25:21.226211] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-09-16T18:25:21.448919] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-04 (0.27 seconds). [2025-09-16T18:25:21.974970] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-06. [2025-09-16T18:25:22.774856] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-03. [2025-09-16T18:25:23.257374] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T18:25:23.574919] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-07. [2025-09-16T18:25:23.923513] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T18:25:24.368887] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T18:25:24.374901] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-05. [2025-09-16T18:25:24.729620] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-09-16T18:25:25.174895] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-06. [2025-09-16T18:25:25.436967] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-06 (0.26 seconds). [2025-09-16T18:25:25.974906] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-08. [2025-09-16T18:25:26.774866] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-08. [2025-09-16T18:25:26.808419] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-09-16T18:25:27.574896] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-04. [2025-09-16T18:25:27.918036] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T18:25:28.374905] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-07. [2025-09-16T18:25:28.837715] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-09-16T18:25:28.890895] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T18:25:29.174915] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-09. [2025-09-16T18:25:29.930048] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-09-16T18:25:29.932764] [INFO ] [REMOTE ] FCN: Full fetch complete (28426 records). [2025-09-16T18:25:29.932841] [DEBUG] [VALIDATOR ] FCN: Validating primary data. [2025-09-16T18:25:29.933342] [FATAL] [VALIDATOR ] FCN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:25:29.933420] [FATAL] [MAIN ] FCN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:25:29.933457] [WARN ] [MAIN ] FCN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:25:29.933495] [INFO ] [LOCAL ] FCN: Starting deletion of primary data. [2025-09-16T18:25:29.933585] [DEBUG] [LOCAL ] FCN: No primary data file found to delete. [2025-09-16T18:25:29.933625] [INFO ] [LOCAL ] FCN: Starting deletion of derivative data. [2025-09-16T18:25:29.933691] [DEBUG] [LOCAL ] FCN: No derivative data file found to delete. [2025-09-16T18:25:29.933748] [INFO ] [MAIN ] Update Progress: 808/2369 symbols completed. FCN took 23.4s. Estimated remaining: 2h 45m 5s. Rate: 9.5 symbols/min [2025-09-16T18:25:29.979991] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-08. [2025-09-16T18:25:30.114766] [INFO ] [MAIN ] FCT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:30.125316] [DEBUG] [LOCAL ] FCT: Loaded 10373 existing records from primary storage. [2025-09-16T18:25:30.125433] [INFO ] [MAIN ] FCT: Loaded 10373 existing records. [2025-09-16T18:25:30.306403] [WARN ] [MAIN ] FCT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:25:30.306682] [INFO ] [MAIN ] FCT: Wiping existing primary data before full refresh. [2025-09-16T18:25:30.306730] [INFO ] [LOCAL ] FCT: Starting deletion of primary data. [2025-09-16T18:25:30.307090] [DEBUG] [LOCAL ] FCT: Deleted primary data file. [2025-09-16T18:25:30.307135] [INFO ] [MAIN ] FCT: Starting full primary refresh. [2025-09-16T18:25:30.415294] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-09-16T18:25:30.668749] [INFO ] [REMOTE ] FCT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:25:30.774874] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-02. [2025-09-16T18:25:31.574916] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-05. [2025-09-16T18:25:31.649514] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T18:25:32.374903] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-09. [2025-09-16T18:25:32.679087] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-09-16T18:25:33.174899] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-09. [2025-09-16T18:25:33.363611] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T18:25:33.368210] [INFO ] [REMOTE ] FCF: Full fetch complete (43355 records). [2025-09-16T18:25:33.368309] [DEBUG] [VALIDATOR ] FCF: Validating primary data. [2025-09-16T18:25:33.682705] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-09-16T18:25:33.699475] [INFO ] [REMOTE ] FCRX: Full fetch complete (1250 records). [2025-09-16T18:25:33.704806] [DEBUG] [VALIDATOR ] FCRX: Validating primary data. [2025-09-16T18:25:33.737666] [FATAL] [VALIDATOR ] FCRX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:25:33.755562] [FATAL] [MAIN ] FCRX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:25:33.767045] [WARN ] [MAIN ] FCRX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:25:33.769378] [DEBUG] [VALIDATOR ] FCF: All expected trading days are present. [2025-09-16T18:25:33.769502] [DEBUG] [VALIDATOR ] FCF: Primary data validation passed. [2025-09-16T18:25:33.769465] [INFO ] [LOCAL ] FCRX: Starting deletion of primary data. [2025-09-16T18:25:33.770270] [INFO ] [MAIN ] FCF: Fetched primary data is valid. [2025-09-16T18:25:33.770331] [DEBUG] [LOCAL ] FCF: Saving primary data. [2025-09-16T18:25:33.770413] [DEBUG] [LOCAL ] FCRX: No primary data file found to delete. [2025-09-16T18:25:33.770573] [INFO ] [LOCAL ] FCRX: Starting deletion of derivative data. [2025-09-16T18:25:33.770625] [DEBUG] [LOCAL ] FCRX: No derivative data file found to delete. [2025-09-16T18:25:33.770715] [INFO ] [MAIN ] Update Progress: 809/2369 symbols completed. FCRX took 16.4s. Estimated remaining: 2h 44m 54s. Rate: 9.5 symbols/min [2025-09-16T18:25:33.870885] [DEBUG] [LOCAL ] FCF: Trimmed 43355 records to 28310 (keeping 150 days of data) [2025-09-16T18:25:33.975897] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-06. [2025-09-16T18:25:34.031938] [INFO ] [MAIN ] FCX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:34.033696] [DEBUG] [LOCAL ] FCX: No existing data file found in primary storage. [2025-09-16T18:25:34.048095] [WARN ] [MAIN ] FCX: No existing data found, performing full refresh instead. [2025-09-16T18:25:34.052302] [INFO ] [MAIN ] FCX: Wiping existing primary data before full refresh. [2025-09-16T18:25:34.053499] [INFO ] [LOCAL ] FCX: Starting deletion of primary data. [2025-09-16T18:25:34.053988] [DEBUG] [LOCAL ] FCX: No primary data file found to delete. [2025-09-16T18:25:34.055778] [INFO ] [MAIN ] FCX: Starting full primary refresh. [2025-09-16T18:25:34.523305] [INFO ] [REMOTE ] FCX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:25:34.535920] [DEBUG] [LOCAL ] FCF: Primary data saved (28310 records). [2025-09-16T18:25:34.536008] [INFO ] [MAIN ] FCF: Full history saved. [2025-09-16T18:25:34.536091] [INFO ] [MAIN ] Update Progress: 810/2369 symbols completed. FCF took 36.9s. Estimated remaining: 2h 44m 36s. Rate: 9.5 symbols/min [2025-09-16T18:25:34.706627] [INFO ] [MAIN ] FDP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:34.706829] [DEBUG] [LOCAL ] FDP: No existing data file found in primary storage. [2025-09-16T18:25:34.706889] [WARN ] [MAIN ] FDP: No existing data found, performing full refresh instead. [2025-09-16T18:25:34.707011] [INFO ] [MAIN ] FDP: Wiping existing primary data before full refresh. [2025-09-16T18:25:34.707051] [INFO ] [LOCAL ] FDP: Starting deletion of primary data. [2025-09-16T18:25:34.707094] [DEBUG] [LOCAL ] FDP: No primary data file found to delete. [2025-09-16T18:25:34.707129] [INFO ] [MAIN ] FDP: Starting full primary refresh. [2025-09-16T18:25:34.783435] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-02. [2025-09-16T18:25:35.076247] [INFO ] [REMOTE ] FDP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:25:35.361972] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T18:25:35.574896] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-02. [2025-09-16T18:25:36.374948] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-03. [2025-09-16T18:25:36.559833] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T18:25:36.901613] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-09-16T18:25:36.961833] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-09-16T18:25:37.174884] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-04. [2025-09-16T18:25:37.974919] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-07. [2025-09-16T18:25:38.146301] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-09-16T18:25:38.774934] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-03. [2025-09-16T18:25:39.444035] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T18:25:39.574945] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-08. [2025-09-16T18:25:40.374867] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-03. [2025-09-16T18:25:40.763474] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-09-16T18:25:40.925597] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-09-16T18:25:41.174928] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-09. [2025-09-16T18:25:41.361759] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-09-16T18:25:41.974924] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-04. [2025-09-16T18:25:42.403730] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-09-16T18:25:42.408186] [INFO ] [REMOTE ] FCPT: Full fetch complete (40630 records). [2025-09-16T18:25:42.408314] [DEBUG] [VALIDATOR ] FCPT: Validating primary data. [2025-09-16T18:25:42.408972] [FATAL] [VALIDATOR ] FCPT: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:25:42.409077] [FATAL] [MAIN ] FCPT: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:25:42.409120] [WARN ] [MAIN ] FCPT: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:25:42.409168] [INFO ] [LOCAL ] FCPT: Starting deletion of primary data. [2025-09-16T18:25:42.409226] [DEBUG] [LOCAL ] FCPT: No primary data file found to delete. [2025-09-16T18:25:42.409264] [INFO ] [LOCAL ] FCPT: Starting deletion of derivative data. [2025-09-16T18:25:42.409511] [DEBUG] [LOCAL ] FCPT: Deleted derivative data file. [2025-09-16T18:25:42.409593] [INFO ] [MAIN ] Update Progress: 811/2369 symbols completed. FCPT took 28.6s. Estimated remaining: 2h 44m 33s. Rate: 9.5 symbols/min [2025-09-16T18:25:42.585954] [INFO ] [MAIN ] FDS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:42.604328] [DEBUG] [LOCAL ] FDS: Loaded 18187 existing records from primary storage. [2025-09-16T18:25:42.604472] [INFO ] [MAIN ] FDS: Loaded 18187 existing records. [2025-09-16T18:25:42.780069] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-05. [2025-09-16T18:25:42.784400] [WARN ] [MAIN ] FDS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:25:42.785389] [INFO ] [MAIN ] FDS: Wiping existing primary data before full refresh. [2025-09-16T18:25:42.785438] [INFO ] [LOCAL ] FDS: Starting deletion of primary data. [2025-09-16T18:25:42.786005] [DEBUG] [LOCAL ] FDS: Deleted primary data file. [2025-09-16T18:25:42.786055] [INFO ] [MAIN ] FDS: Starting full primary refresh. [2025-09-16T18:25:43.155414] [INFO ] [REMOTE ] FDS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:25:43.574897] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-02. [2025-09-16T18:25:43.690574] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T18:25:44.327732] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-04 (2.35 seconds). [2025-09-16T18:25:44.374879] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-05. [2025-09-16T18:25:44.592278] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T18:25:45.174905] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-04. [2025-09-16T18:25:45.974901] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-03. [2025-09-16T18:25:46.413575] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-09-16T18:25:46.446831] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-09-16T18:25:46.774916] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-06. [2025-09-16T18:25:47.299973] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T18:25:47.574947] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-05. [2025-09-16T18:25:48.374918] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-06. [2025-09-16T18:25:48.691664] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-09-16T18:25:48.937354] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-09-16T18:25:49.174908] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-04. [2025-09-16T18:25:49.355134] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T18:25:49.974942] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-07. [2025-09-16T18:25:50.394800] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-09-16T18:25:50.774909] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-06. [2025-09-16T18:25:51.574888] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-07. [2025-09-16T18:25:51.611766] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-09-16T18:25:52.225635] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-09-16T18:25:52.374941] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-08. [2025-09-16T18:25:52.492915] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-09-16T18:25:53.174968] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-08. [2025-09-16T18:25:53.974941] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-07. [2025-09-16T18:25:54.210337] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-09-16T18:25:54.471658] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-08 (2.10 seconds). [2025-09-16T18:25:54.774922] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-09. [2025-09-16T18:25:55.365918] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-09-16T18:25:55.574930] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-08. [2025-09-16T18:25:56.374934] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-09. [2025-09-16T18:25:56.614386] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-09-16T18:25:56.620834] [INFO ] [REMOTE ] FCX: Full fetch complete (58379 records). [2025-09-16T18:25:56.620962] [DEBUG] [VALIDATOR ] FCX: Validating primary data. [2025-09-16T18:25:57.003274] [DEBUG] [VALIDATOR ] FCX: All expected trading days are present. [2025-09-16T18:25:57.019588] [DEBUG] [VALIDATOR ] FCX: Primary data validation passed. [2025-09-16T18:25:57.020698] [INFO ] [MAIN ] FCX: Fetched primary data is valid. [2025-09-16T18:25:57.020882] [DEBUG] [LOCAL ] FCX: Saving primary data. [2025-09-16T18:25:57.022572] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T18:25:57.029793] [DEBUG] [LOCAL ] FCX: Trimmed 58379 records to 40052 (keeping 150 days of data) [2025-09-16T18:25:57.217157] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-09. [2025-09-16T18:25:57.278058] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-09-16T18:25:57.282792] [INFO ] [REMOTE ] FCT: Full fetch complete (15503 records). [2025-09-16T18:25:57.283911] [DEBUG] [VALIDATOR ] FCT: Validating primary data. [2025-09-16T18:25:57.653692] [DEBUG] [LOCAL ] FCX: Primary data saved (40052 records). [2025-09-16T18:25:57.677434] [INFO ] [MAIN ] FCX: Full history saved. [2025-09-16T18:25:57.677607] [INFO ] [MAIN ] Update Progress: 812/2369 symbols completed. FCX took 23.9s. Estimated remaining: 2h 44m 44s. Rate: 9.5 symbols/min [2025-09-16T18:25:57.678094] [DEBUG] [VALIDATOR ] FCT: All expected trading days are present. [2025-09-16T18:25:57.710327] [DEBUG] [VALIDATOR ] FCT: Primary data validation passed. [2025-09-16T18:25:57.743098] [INFO ] [MAIN ] FCT: Fetched primary data is valid. [2025-09-16T18:25:57.754498] [DEBUG] [LOCAL ] FCT: Saving primary data. [2025-09-16T18:25:57.781556] [DEBUG] [LOCAL ] FCT: Trimmed 15503 records to 10467 (keeping 150 days of data) [2025-09-16T18:25:57.898561] [INFO ] [MAIN ] FDX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:57.918284] [DEBUG] [LOCAL ] FCT: Primary data saved (10467 records). [2025-09-16T18:25:57.918510] [INFO ] [MAIN ] FCT: Full history saved. [2025-09-16T18:25:57.918609] [INFO ] [MAIN ] Update Progress: 813/2369 symbols completed. FCT took 28.0s. Estimated remaining: 2h 44m 26s. Rate: 9.5 symbols/min [2025-09-16T18:25:57.976369] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-05. [2025-09-16T18:25:58.114665] [INFO ] [MAIN ] FE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:58.128098] [DEBUG] [LOCAL ] FDX: Loaded 37922 existing records from primary storage. [2025-09-16T18:25:58.128246] [INFO ] [MAIN ] FDX: Loaded 37922 existing records. [2025-09-16T18:25:58.315149] [WARN ] [MAIN ] FDX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:25:58.315385] [INFO ] [MAIN ] FDX: Wiping existing primary data before full refresh. [2025-09-16T18:25:58.315430] [INFO ] [LOCAL ] FDX: Starting deletion of primary data. [2025-09-16T18:25:58.316459] [DEBUG] [LOCAL ] FDX: Deleted primary data file. [2025-09-16T18:25:58.316512] [INFO ] [MAIN ] FDX: Starting full primary refresh. [2025-09-16T18:25:58.446375] [DEBUG] [LOCAL ] FE: Loaded 39450 existing records from primary storage. [2025-09-16T18:25:58.479748] [INFO ] [MAIN ] FE: Loaded 39450 existing records. [2025-09-16T18:25:58.594619] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T18:25:58.703721] [INFO ] [REMOTE ] FDP: Full fetch complete (33185 records). [2025-09-16T18:25:58.724699] [DEBUG] [VALIDATOR ] FDP: Validating primary data. [2025-09-16T18:25:58.915147] [WARN ] [MAIN ] FE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:25:58.959283] [INFO ] [REMOTE ] FDX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:25:58.980611] [INFO ] [MAIN ] FE: Wiping existing primary data before full refresh. [2025-09-16T18:25:58.995921] [INFO ] [LOCAL ] FE: Starting deletion of primary data. [2025-09-16T18:25:58.999744] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-02. [2025-09-16T18:25:59.007020] [DEBUG] [LOCAL ] FE: Deleted primary data file. [2025-09-16T18:25:59.007161] [INFO ] [MAIN ] FE: Starting full primary refresh. [2025-09-16T18:25:59.503128] [DEBUG] [VALIDATOR ] FDP: All expected trading days are present. [2025-09-16T18:25:59.515170] [DEBUG] [VALIDATOR ] FDP: Primary data validation passed. [2025-09-16T18:25:59.541874] [INFO ] [MAIN ] FDP: Fetched primary data is valid. [2025-09-16T18:25:59.552696] [DEBUG] [LOCAL ] FDP: Saving primary data. [2025-09-16T18:25:59.564971] [DEBUG] [LOCAL ] FDP: Trimmed 33185 records to 22537 (keeping 150 days of data) [2025-09-16T18:25:59.704806] [INFO ] [REMOTE ] FE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:25:59.731373] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-02. [2025-09-16T18:25:59.750988] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-09-16T18:25:59.802560] [DEBUG] [LOCAL ] FDP: Primary data saved (22537 records). [2025-09-16T18:25:59.802744] [INFO ] [MAIN ] FDP: Full history saved. [2025-09-16T18:25:59.802833] [INFO ] [MAIN ] Update Progress: 814/2369 symbols completed. FDP took 25.3s. Estimated remaining: 2h 44m 11s. Rate: 9.5 symbols/min [2025-09-16T18:25:59.975717] [INFO ] [MAIN ] FENG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:25:59.975908] [DEBUG] [LOCAL ] FENG: No existing data file found in primary storage. [2025-09-16T18:25:59.975948] [WARN ] [MAIN ] FENG: No existing data found, performing full refresh instead. [2025-09-16T18:25:59.976071] [INFO ] [MAIN ] FENG: Wiping existing primary data before full refresh. [2025-09-16T18:25:59.976115] [INFO ] [LOCAL ] FENG: Starting deletion of primary data. [2025-09-16T18:25:59.976159] [DEBUG] [LOCAL ] FENG: No primary data file found to delete. [2025-09-16T18:25:59.976193] [INFO ] [MAIN ] FENG: Starting full primary refresh. [2025-09-16T18:26:00.337825] [INFO ] [REMOTE ] FENG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:26:00.374860] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-02. [2025-09-16T18:26:00.487151] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T18:26:01.005761] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-09-16T18:26:01.174901] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-03. [2025-09-16T18:26:01.366927] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:26:01.828562] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-09-16T18:26:01.974948] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-04. [2025-09-16T18:26:02.451931] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-09-16T18:26:02.774942] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-06. [2025-09-16T18:26:03.574953] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-03. [2025-09-16T18:26:04.114061] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-09-16T18:26:04.374914] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-03. [2025-09-16T18:26:05.174934] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-07. [2025-09-16T18:26:05.550689] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-09-16T18:26:05.974933] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-05. [2025-09-16T18:26:06.206824] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T18:26:06.323441] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-09-16T18:26:06.342228] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T18:26:06.774901] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-04. [2025-09-16T18:26:07.574898] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-04. [2025-09-16T18:26:08.374906] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-08. [2025-09-16T18:26:08.975490] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-04 (2.20 seconds). [2025-09-16T18:26:09.174893] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-05. [2025-09-16T18:26:09.338618] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-09-16T18:26:09.447102] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-09-16T18:26:09.974901] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-05. [2025-09-16T18:26:10.774914] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-06. [2025-09-16T18:26:11.251426] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-09-16T18:26:11.542158] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T18:26:11.574847] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-06. [2025-09-16T18:26:11.696565] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-05 (2.52 seconds). [2025-09-16T18:26:12.374903] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-06. [2025-09-16T18:26:13.174935] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-09. [2025-09-16T18:26:13.553701] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-09-16T18:26:13.984076] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-07. [2025-09-16T18:26:14.010414] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T18:26:14.774893] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-07. [2025-09-16T18:26:14.809992] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T18:26:14.813899] [INFO ] [REMOTE ] FDS: Full fetch complete (27170 records). [2025-09-16T18:26:14.814010] [DEBUG] [VALIDATOR ] FDS: Validating primary data. [2025-09-16T18:26:15.197093] [DEBUG] [VALIDATOR ] FDS: All expected trading days are present. [2025-09-16T18:26:15.197174] [DEBUG] [VALIDATOR ] FDS: Primary data validation passed. [2025-09-16T18:26:15.197762] [INFO ] [MAIN ] FDS: Fetched primary data is valid. [2025-09-16T18:26:15.197811] [DEBUG] [LOCAL ] FDS: Saving primary data. [2025-09-16T18:26:15.198504] [DEBUG] [LOCAL ] FDS: Trimmed 27170 records to 18492 (keeping 150 days of data) [2025-09-16T18:26:15.321461] [DEBUG] [LOCAL ] FDS: Primary data saved (18492 records). [2025-09-16T18:26:15.321620] [INFO ] [MAIN ] FDS: Full history saved. [2025-09-16T18:26:15.321731] [INFO ] [MAIN ] Update Progress: 815/2369 symbols completed. FDS took 32.9s. Estimated remaining: 2h 44m 22s. Rate: 9.5 symbols/min [2025-09-16T18:26:15.509869] [INFO ] [MAIN ] FERG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:26:15.510094] [DEBUG] [LOCAL ] FERG: No existing data file found in primary storage. [2025-09-16T18:26:15.510133] [WARN ] [MAIN ] FERG: No existing data found, performing full refresh instead. [2025-09-16T18:26:15.510262] [INFO ] [MAIN ] FERG: Wiping existing primary data before full refresh. [2025-09-16T18:26:15.510304] [INFO ] [LOCAL ] FERG: Starting deletion of primary data. [2025-09-16T18:26:15.510347] [DEBUG] [LOCAL ] FERG: No primary data file found to delete. [2025-09-16T18:26:15.510404] [INFO ] [MAIN ] FERG: Starting full primary refresh. [2025-09-16T18:26:15.578169] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-07. [2025-09-16T18:26:15.897072] [INFO ] [REMOTE ] FERG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:26:15.900433] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-09-16T18:26:15.931622] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-09-16T18:26:16.348215] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T18:26:16.374873] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-08. [2025-09-16T18:26:17.174909] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-08. [2025-09-16T18:26:17.459628] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-08 (0.28 seconds). [2025-09-16T18:26:17.717131] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T18:26:17.974906] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-09. [2025-09-16T18:26:18.774913] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-02. [2025-09-16T18:26:19.240765] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T18:26:19.246982] [INFO ] [REMOTE ] FE: Full fetch complete (58096 records). [2025-09-16T18:26:19.247084] [DEBUG] [VALIDATOR ] FE: Validating primary data. [2025-09-16T18:26:19.581253] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-09. [2025-09-16T18:26:19.686170] [DEBUG] [VALIDATOR ] FE: All expected trading days are present. [2025-09-16T18:26:19.686267] [DEBUG] [VALIDATOR ] FE: Primary data validation passed. [2025-09-16T18:26:19.687106] [INFO ] [MAIN ] FE: Fetched primary data is valid. [2025-09-16T18:26:19.687195] [DEBUG] [LOCAL ] FE: Saving primary data. [2025-09-16T18:26:19.688658] [DEBUG] [LOCAL ] FE: Trimmed 58096 records to 39837 (keeping 150 days of data) [2025-09-16T18:26:19.938743] [DEBUG] [LOCAL ] FE: Primary data saved (39837 records). [2025-09-16T18:26:19.938903] [INFO ] [MAIN ] FE: Full history saved. [2025-09-16T18:26:19.938992] [INFO ] [MAIN ] Update Progress: 816/2369 symbols completed. FE took 22.0s. Estimated remaining: 2h 44m 13s. Rate: 9.5 symbols/min [2025-09-16T18:26:20.125317] [INFO ] [MAIN ] FET: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:26:20.135890] [DEBUG] [LOCAL ] FET: Loaded 8868 existing records from primary storage. [2025-09-16T18:26:20.136067] [INFO ] [MAIN ] FET: Loaded 8868 existing records. [2025-09-16T18:26:20.252970] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-09-16T18:26:20.332056] [WARN ] [MAIN ] FET: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:26:20.326986] [INFO ] [REMOTE ] FENG: Full fetch complete (1671 records). [2025-09-16T18:26:20.332419] [DEBUG] [VALIDATOR ] FENG: Validating primary data. [2025-09-16T18:26:20.376694] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-08. [2025-09-16T18:26:20.376593] [INFO ] [MAIN ] FET: Wiping existing primary data before full refresh. [2025-09-16T18:26:20.420681] [INFO ] [LOCAL ] FET: Starting deletion of primary data. [2025-09-16T18:26:20.443575] [DEBUG] [LOCAL ] FET: Deleted primary data file. [2025-09-16T18:26:20.469446] [INFO ] [MAIN ] FET: Starting full primary refresh. [2025-09-16T18:26:20.534697] [WARN ] [VALIDATOR ] FENG: Validation failed - insufficient historical data. Have 149 unique days, need 150 days [2025-09-16T18:26:20.569122] [FATAL] [MAIN ] FENG: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:26:20.595897] [WARN ] [MAIN ] FENG: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:26:20.621440] [INFO ] [LOCAL ] FENG: Starting deletion of primary data. [2025-09-16T18:26:20.640209] [DEBUG] [LOCAL ] FENG: No primary data file found to delete. [2025-09-16T18:26:20.655733] [INFO ] [LOCAL ] FENG: Starting deletion of derivative data. [2025-09-16T18:26:20.692847] [DEBUG] [LOCAL ] FENG: No derivative data file found to delete. [2025-09-16T18:26:20.727329] [INFO ] [MAIN ] Update Progress: 817/2369 symbols completed. FENG took 20.9s. Estimated remaining: 2h 43m 56s. Rate: 9.5 symbols/min [2025-09-16T18:26:20.963575] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-02 (2.19 seconds). [2025-09-16T18:26:21.111707] [INFO ] [MAIN ] FF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:26:21.131249] [INFO ] [REMOTE ] FET: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:26:21.149024] [DEBUG] [LOCAL ] FF: Loaded 17610 existing records from primary storage. [2025-09-16T18:26:21.149176] [INFO ] [MAIN ] FF: Loaded 17610 existing records. [2025-09-16T18:26:21.180252] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-03. [2025-09-16T18:26:21.333572] [WARN ] [MAIN ] FF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:26:21.333854] [INFO ] [MAIN ] FF: Wiping existing primary data before full refresh. [2025-09-16T18:26:21.333904] [INFO ] [LOCAL ] FF: Starting deletion of primary data. [2025-09-16T18:26:21.334504] [DEBUG] [LOCAL ] FF: Deleted primary data file. [2025-09-16T18:26:21.334567] [INFO ] [MAIN ] FF: Starting full primary refresh. [2025-09-16T18:26:21.690330] [INFO ] [REMOTE ] FF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:26:21.974862] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-02. [2025-09-16T18:26:22.022665] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T18:26:22.774882] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-09. [2025-09-16T18:26:22.975012] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-09-16T18:26:23.109246] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-09-16T18:26:23.574918] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-03. [2025-09-16T18:26:24.314350] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T18:26:24.321055] [INFO ] [REMOTE ] FDX: Full fetch complete (55865 records). [2025-09-16T18:26:24.321377] [DEBUG] [VALIDATOR ] FDX: Validating primary data. [2025-09-16T18:26:24.386957] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-02. [2025-09-16T18:26:24.713567] [DEBUG] [VALIDATOR ] FDX: All expected trading days are present. [2025-09-16T18:26:24.713681] [DEBUG] [VALIDATOR ] FDX: Primary data validation passed. [2025-09-16T18:26:24.714624] [INFO ] [MAIN ] FDX: Fetched primary data is valid. [2025-09-16T18:26:24.714718] [DEBUG] [LOCAL ] FDX: Saving primary data. [2025-09-16T18:26:24.716078] [DEBUG] [LOCAL ] FDX: Trimmed 55865 records to 38297 (keeping 150 days of data) [2025-09-16T18:26:24.960571] [DEBUG] [LOCAL ] FDX: Primary data saved (38297 records). [2025-09-16T18:26:24.960732] [INFO ] [MAIN ] FDX: Full history saved. [2025-09-16T18:26:24.961478] [INFO ] [MAIN ] Update Progress: 818/2369 symbols completed. FDX took 27.3s. Estimated remaining: 2h 43m 45s. Rate: 9.5 symbols/min [2025-09-16T18:26:25.132988] [INFO ] [MAIN ] FFA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:26:25.139414] [DEBUG] [LOCAL ] FFA: Loaded 3377 existing records from primary storage. [2025-09-16T18:26:25.139545] [INFO ] [MAIN ] FFA: Loaded 3377 existing records. [2025-09-16T18:26:25.182943] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-04. [2025-09-16T18:26:25.322661] [WARN ] [MAIN ] FFA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:26:25.322961] [INFO ] [MAIN ] FFA: Wiping existing primary data before full refresh. [2025-09-16T18:26:25.323118] [INFO ] [LOCAL ] FFA: Starting deletion of primary data. [2025-09-16T18:26:25.328533] [DEBUG] [LOCAL ] FFA: Deleted primary data file. [2025-09-16T18:26:25.336326] [INFO ] [MAIN ] FFA: Starting full primary refresh. [2025-09-16T18:26:25.406938] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T18:26:25.696129] [INFO ] [REMOTE ] FFA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:26:25.974910] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-02. [2025-09-16T18:26:26.598304] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-09-16T18:26:26.614897] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-03 (3.04 seconds). [2025-09-16T18:26:26.774925] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-03. [2025-09-16T18:26:27.574909] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-04. [2025-09-16T18:26:28.197014] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-09-16T18:26:28.374901] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-05. [2025-09-16T18:26:29.174913] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-03. [2025-09-16T18:26:30.055131] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-09-16T18:26:30.061938] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-04. [2025-09-16T18:26:30.270171] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-03 (3.50 seconds). [2025-09-16T18:26:30.606055] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-05 (2.23 seconds). [2025-09-16T18:26:30.774915] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-04. [2025-09-16T18:26:31.348982] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-09-16T18:26:31.372206] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T18:26:31.574898] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-05. [2025-09-16T18:26:32.374921] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-05. [2025-09-16T18:26:32.832794] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-09-16T18:26:33.174904] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-06. [2025-09-16T18:26:33.974906] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-06. [2025-09-16T18:26:34.096201] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-09-16T18:26:34.587549] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-09-16T18:26:34.774897] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-07. [2025-09-16T18:26:35.527351] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-09-16T18:26:35.559138] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-05 (3.18 seconds). [2025-09-16T18:26:35.574838] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-06. [2025-09-16T18:26:35.993332] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-09-16T18:26:36.374858] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-07. [2025-09-16T18:26:37.174925] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-08. [2025-09-16T18:26:37.295155] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-09-16T18:26:37.974958] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-08. [2025-09-16T18:26:38.092148] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-04 (12.91 seconds). [2025-09-16T18:26:38.169233] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T18:26:38.766920] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T18:26:38.774841] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-09. [2025-09-16T18:26:39.574935] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-09. [2025-09-16T18:26:39.782484] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T18:26:39.785339] [INFO ] [REMOTE ] FF: Full fetch complete (27944 records). [2025-09-16T18:26:39.785416] [DEBUG] [VALIDATOR ] FF: Validating primary data. [2025-09-16T18:26:39.785948] [FATAL] [VALIDATOR ] FF: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:26:39.786026] [FATAL] [MAIN ] FF: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:26:39.786066] [WARN ] [MAIN ] FF: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:26:39.786115] [INFO ] [LOCAL ] FF: Starting deletion of primary data. [2025-09-16T18:26:39.786180] [DEBUG] [LOCAL ] FF: No primary data file found to delete. [2025-09-16T18:26:39.786228] [INFO ] [LOCAL ] FF: Starting deletion of derivative data. [2025-09-16T18:26:39.786471] [DEBUG] [LOCAL ] FF: Deleted derivative data file. [2025-09-16T18:26:39.786560] [INFO ] [MAIN ] Update Progress: 819/2369 symbols completed. FF took 19.1s. Estimated remaining: 2h 43m 55s. Rate: 9.5 symbols/min [2025-09-16T18:26:39.957334] [INFO ] [MAIN ] FFC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:26:39.970750] [DEBUG] [LOCAL ] FFC: Loaded 11958 existing records from primary storage. [2025-09-16T18:26:39.970873] [INFO ] [MAIN ] FFC: Loaded 11958 existing records. [2025-09-16T18:26:40.178720] [WARN ] [MAIN ] FFC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:26:40.179004] [INFO ] [MAIN ] FFC: Wiping existing primary data before full refresh. [2025-09-16T18:26:40.179051] [INFO ] [LOCAL ] FFC: Starting deletion of primary data. [2025-09-16T18:26:40.179446] [DEBUG] [LOCAL ] FFC: Deleted primary data file. [2025-09-16T18:26:40.179491] [INFO ] [MAIN ] FFC: Starting full primary refresh. [2025-09-16T18:26:40.387015] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-05. [2025-09-16T18:26:40.536563] [INFO ] [REMOTE ] FFC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:26:40.638785] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-09-16T18:26:40.641099] [INFO ] [REMOTE ] FET: Full fetch complete (13641 records). [2025-09-16T18:26:40.641171] [DEBUG] [VALIDATOR ] FET: Validating primary data. [2025-09-16T18:26:40.999671] [DEBUG] [VALIDATOR ] FET: All expected trading days are present. [2025-09-16T18:26:40.999754] [DEBUG] [VALIDATOR ] FET: Primary data validation passed. [2025-09-16T18:26:41.000063] [INFO ] [MAIN ] FET: Fetched primary data is valid. [2025-09-16T18:26:41.000111] [DEBUG] [LOCAL ] FET: Saving primary data. [2025-09-16T18:26:41.000636] [DEBUG] [LOCAL ] FET: Trimmed 13641 records to 9003 (keeping 150 days of data) [2025-09-16T18:26:41.056129] [DEBUG] [LOCAL ] FET: Primary data saved (9003 records). [2025-09-16T18:26:41.056299] [INFO ] [MAIN ] FET: Full history saved. [2025-09-16T18:26:41.056390] [INFO ] [MAIN ] Update Progress: 820/2369 symbols completed. FET took 21.1s. Estimated remaining: 2h 43m 39s. Rate: 9.5 symbols/min [2025-09-16T18:26:41.180018] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-07. [2025-09-16T18:26:41.228108] [INFO ] [MAIN ] FFWM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:26:41.252335] [DEBUG] [LOCAL ] FFWM: Loaded 26071 existing records from primary storage. [2025-09-16T18:26:41.252465] [INFO ] [MAIN ] FFWM: Loaded 26071 existing records. [2025-09-16T18:26:41.430883] [WARN ] [MAIN ] FFWM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:26:41.431228] [INFO ] [MAIN ] FFWM: Wiping existing primary data before full refresh. [2025-09-16T18:26:41.431279] [INFO ] [LOCAL ] FFWM: Starting deletion of primary data. [2025-09-16T18:26:41.431874] [DEBUG] [LOCAL ] FFWM: Deleted primary data file. [2025-09-16T18:26:41.431928] [INFO ] [MAIN ] FFWM: Starting full primary refresh. [2025-09-16T18:26:41.785570] [INFO ] [REMOTE ] FFWM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:26:41.887492] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-09-16T18:26:41.974931] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-02. [2025-09-16T18:26:42.230551] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T18:26:42.727746] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T18:26:42.774846] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-03. [2025-09-16T18:26:43.574893] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-02. [2025-09-16T18:26:43.765128] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-09-16T18:26:44.374906] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-04. [2025-09-16T18:26:44.939526] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T18:26:45.174925] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-03. [2025-09-16T18:26:45.207132] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-09-16T18:26:45.974919] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-06. [2025-09-16T18:26:46.393581] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-09-16T18:26:46.774967] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-04. [2025-09-16T18:26:47.574897] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-08. [2025-09-16T18:26:47.735025] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T18:26:48.194314] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T18:26:48.207444] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-09-16T18:26:48.374913] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-05. [2025-09-16T18:26:49.174911] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-07. [2025-09-16T18:26:49.440940] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-09-16T18:26:49.974939] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-09. [2025-09-16T18:26:50.555133] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-09-16T18:26:50.556257] [INFO ] [REMOTE ] FFA: Full fetch complete (5970 records). [2025-09-16T18:26:50.556354] [DEBUG] [VALIDATOR ] FFA: Validating primary data. [2025-09-16T18:26:50.556740] [FATAL] [VALIDATOR ] FFA: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:26:50.556807] [FATAL] [MAIN ] FFA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:26:50.556851] [WARN ] [MAIN ] FFA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:26:50.556892] [INFO ] [LOCAL ] FFA: Starting deletion of primary data. [2025-09-16T18:26:50.556948] [DEBUG] [LOCAL ] FFA: No primary data file found to delete. [2025-09-16T18:26:50.556982] [INFO ] [LOCAL ] FFA: Starting deletion of derivative data. [2025-09-16T18:26:50.557224] [DEBUG] [LOCAL ] FFA: Deleted derivative data file. [2025-09-16T18:26:50.557301] [INFO ] [MAIN ] Update Progress: 821/2369 symbols completed. FFA took 25.6s. Estimated remaining: 2h 43m 39s. Rate: 9.5 symbols/min [2025-09-16T18:26:50.729879] [INFO ] [MAIN ] FG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:26:50.758834] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-09-16T18:26:50.779138] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-05. [2025-09-16T18:26:50.779497] [DEBUG] [LOCAL ] FG: Loaded 15163 existing records from primary storage. [2025-09-16T18:26:50.780344] [INFO ] [MAIN ] FG: Loaded 15163 existing records. [2025-09-16T18:26:50.966022] [WARN ] [MAIN ] FG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:26:50.966301] [INFO ] [MAIN ] FG: Wiping existing primary data before full refresh. [2025-09-16T18:26:50.966351] [INFO ] [LOCAL ] FG: Starting deletion of primary data. [2025-09-16T18:26:50.966774] [DEBUG] [LOCAL ] FG: Deleted primary data file. [2025-09-16T18:26:50.966824] [INFO ] [MAIN ] FG: Starting full primary refresh. [2025-09-16T18:26:51.331947] [INFO ] [REMOTE ] FG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:26:51.574904] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-08. [2025-09-16T18:26:52.301466] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T18:26:52.374866] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-06. [2025-09-16T18:26:53.174904] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-02. [2025-09-16T18:26:53.334081] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-09-16T18:26:53.441614] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T18:26:53.980588] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-06. [2025-09-16T18:26:53.983675] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-09-16T18:26:54.774871] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-03. [2025-09-16T18:26:55.081147] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-09-16T18:26:55.574881] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-07. [2025-09-16T18:26:55.874911] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T18:26:56.374904] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-04. [2025-09-16T18:26:57.176382] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-07. [2025-09-16T18:26:57.178695] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T18:26:57.683467] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T18:26:57.945144] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T18:26:57.974851] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-09. [2025-09-16T18:26:58.774894] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-08. [2025-09-16T18:26:59.574929] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-05. [2025-09-16T18:26:59.605438] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-09-16T18:26:59.611762] [INFO ] [REMOTE ] FERG: Full fetch complete (52858 records). [2025-09-16T18:26:59.611906] [DEBUG] [VALIDATOR ] FERG: Validating primary data. [2025-09-16T18:26:59.612602] [FATAL] [VALIDATOR ] FERG: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:26:59.612705] [FATAL] [MAIN ] FERG: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:26:59.612750] [WARN ] [MAIN ] FERG: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:26:59.612827] [INFO ] [LOCAL ] FERG: Starting deletion of primary data. [2025-09-16T18:26:59.612895] [DEBUG] [LOCAL ] FERG: No primary data file found to delete. [2025-09-16T18:26:59.612932] [INFO ] [LOCAL ] FERG: Starting deletion of derivative data. [2025-09-16T18:26:59.613017] [DEBUG] [LOCAL ] FERG: No derivative data file found to delete. [2025-09-16T18:26:59.613080] [INFO ] [MAIN ] Update Progress: 822/2369 symbols completed. FERG took 44.3s. Estimated remaining: 2h 43m 37s. Rate: 9.5 symbols/min [2025-09-16T18:26:59.792550] [INFO ] [MAIN ] FGN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:26:59.815935] [DEBUG] [LOCAL ] FGN: Loaded 2723 existing records from primary storage. [2025-09-16T18:26:59.816437] [INFO ] [MAIN ] FGN: Loaded 2723 existing records. [2025-09-16T18:26:59.851104] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T18:27:00.013163] [WARN ] [MAIN ] FGN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:00.013430] [INFO ] [MAIN ] FGN: Wiping existing primary data before full refresh. [2025-09-16T18:27:00.013479] [INFO ] [LOCAL ] FGN: Starting deletion of primary data. [2025-09-16T18:27:00.013698] [DEBUG] [LOCAL ] FGN: Deleted primary data file. [2025-09-16T18:27:00.013746] [INFO ] [MAIN ] FGN: Starting full primary refresh. [2025-09-16T18:27:00.381783] [INFO ] [REMOTE ] FGN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:00.381867] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-08. [2025-09-16T18:27:00.800867] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-09-16T18:27:01.005007] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-09-16T18:27:01.174914] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-09. [2025-09-16T18:27:01.974901] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-09. [2025-09-16T18:27:02.614537] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T18:27:02.618832] [INFO ] [REMOTE ] FFWM: Full fetch complete (39173 records). [2025-09-16T18:27:02.618943] [DEBUG] [VALIDATOR ] FFWM: Validating primary data. [2025-09-16T18:27:02.620621] [FATAL] [VALIDATOR ] FFWM: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:27:02.620736] [FATAL] [MAIN ] FFWM: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:27:02.620810] [WARN ] [MAIN ] FFWM: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:27:02.620913] [INFO ] [LOCAL ] FFWM: Starting deletion of primary data. [2025-09-16T18:27:02.620975] [DEBUG] [LOCAL ] FFWM: No primary data file found to delete. [2025-09-16T18:27:02.621012] [INFO ] [LOCAL ] FFWM: Starting deletion of derivative data. [2025-09-16T18:27:02.621287] [DEBUG] [LOCAL ] FFWM: Deleted derivative data file. [2025-09-16T18:27:02.621386] [INFO ] [MAIN ] Update Progress: 823/2369 symbols completed. FFWM took 21.6s. Estimated remaining: 2h 43m 25s. Rate: 9.5 symbols/min [2025-09-16T18:27:02.802765] [INFO ] [MAIN ] FGSN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:02.798503] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-02. [2025-09-16T18:27:02.809009] [DEBUG] [LOCAL ] FGSN: Loaded 3496 existing records from primary storage. [2025-09-16T18:27:02.809109] [INFO ] [MAIN ] FGSN: Loaded 3496 existing records. [2025-09-16T18:27:02.959796] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T18:27:02.980376] [INFO ] [REMOTE ] FFC: Full fetch complete (17923 records). [2025-09-16T18:27:02.996589] [DEBUG] [VALIDATOR ] FFC: Validating primary data. [2025-09-16T18:27:03.000230] [WARN ] [MAIN ] FGSN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:03.000714] [INFO ] [MAIN ] FGSN: Wiping existing primary data before full refresh. [2025-09-16T18:27:03.001163] [FATAL] [VALIDATOR ] FFC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:27:03.001227] [INFO ] [LOCAL ] FGSN: Starting deletion of primary data. [2025-09-16T18:27:03.001412] [FATAL] [MAIN ] FFC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:27:03.001528] [WARN ] [MAIN ] FFC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:27:03.001575] [INFO ] [LOCAL ] FFC: Starting deletion of primary data. [2025-09-16T18:27:03.001657] [DEBUG] [LOCAL ] FFC: No primary data file found to delete. [2025-09-16T18:27:03.001699] [INFO ] [LOCAL ] FFC: Starting deletion of derivative data. [2025-09-16T18:27:03.001753] [DEBUG] [LOCAL ] FGSN: Deleted primary data file. [2025-09-16T18:27:03.001847] [INFO ] [MAIN ] FGSN: Starting full primary refresh. [2025-09-16T18:27:03.033484] [DEBUG] [LOCAL ] FFC: Deleted derivative data file. [2025-09-16T18:27:03.072133] [INFO ] [MAIN ] Update Progress: 824/2369 symbols completed. FFC took 23.3s. Estimated remaining: 2h 43m 7s. Rate: 9.5 symbols/min [2025-09-16T18:27:03.417203] [INFO ] [MAIN ] FHI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:03.433598] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-09-16T18:27:03.576821] [INFO ] [REMOTE ] FGSN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:03.577004] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-06. [2025-09-16T18:27:03.589710] [DEBUG] [LOCAL ] FHI: Loaded 29782 existing records from primary storage. [2025-09-16T18:27:03.589835] [INFO ] [MAIN ] FHI: Loaded 29782 existing records. [2025-09-16T18:27:03.766406] [WARN ] [MAIN ] FHI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:03.766698] [INFO ] [MAIN ] FHI: Wiping existing primary data before full refresh. [2025-09-16T18:27:03.766747] [INFO ] [LOCAL ] FHI: Starting deletion of primary data. [2025-09-16T18:27:03.767460] [DEBUG] [LOCAL ] FHI: Deleted primary data file. [2025-09-16T18:27:03.767508] [INFO ] [MAIN ] FHI: Starting full primary refresh. [2025-09-16T18:27:04.121697] [INFO ] [REMOTE ] FHI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:04.374873] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-03. [2025-09-16T18:27:04.768402] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-09-16T18:27:04.953839] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T18:27:05.174958] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-02. [2025-09-16T18:27:05.899747] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-09-16T18:27:05.975123] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-07. [2025-09-16T18:27:06.775037] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-03. [2025-09-16T18:27:06.840331] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T18:27:07.316663] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-09-16T18:27:07.574929] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-04. [2025-09-16T18:27:08.061947] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-09-16T18:27:08.374932] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-05. [2025-09-16T18:27:08.925288] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-09-16T18:27:09.174943] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-02. [2025-09-16T18:27:09.974920] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-06. [2025-09-16T18:27:10.563296] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T18:27:10.749272] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T18:27:10.774851] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-03. [2025-09-16T18:27:11.574915] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-04. [2025-09-16T18:27:12.041632] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-09-16T18:27:12.247753] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-09-16T18:27:12.374898] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-04. [2025-09-16T18:27:13.174938] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-05. [2025-09-16T18:27:13.952590] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-09-16T18:27:13.954138] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T18:27:13.974870] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-06. [2025-09-16T18:27:14.620657] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-09-16T18:27:14.774896] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-07. [2025-09-16T18:27:15.355144] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-09-16T18:27:15.574908] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-05. [2025-09-16T18:27:16.374940] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-08. [2025-09-16T18:27:17.051627] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-09-16T18:27:17.174871] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-07. [2025-09-16T18:27:17.205250] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-09-16T18:27:17.732599] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-09-16T18:27:17.974965] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-06. [2025-09-16T18:27:18.774909] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-08. [2025-09-16T18:27:19.354878] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-09-16T18:27:19.456717] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T18:27:19.574884] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-07. [2025-09-16T18:27:20.374908] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-09. [2025-09-16T18:27:20.797883] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-09-16T18:27:20.956994] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-09-16T18:27:20.958833] [INFO ] [REMOTE ] FGSN: Full fetch complete (5529 records). [2025-09-16T18:27:20.959049] [DEBUG] [VALIDATOR ] FGSN: Validating primary data. [2025-09-16T18:27:21.180169] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-08. [2025-09-16T18:27:21.358005] [DEBUG] [VALIDATOR ] FGSN: All expected trading days are present. [2025-09-16T18:27:21.358120] [DEBUG] [VALIDATOR ] FGSN: Primary data validation passed. [2025-09-16T18:27:21.358361] [INFO ] [MAIN ] FGSN: Fetched primary data is valid. [2025-09-16T18:27:21.358411] [DEBUG] [LOCAL ] FGSN: Saving primary data. [2025-09-16T18:27:21.359375] [DEBUG] [LOCAL ] FGSN: Trimmed 5529 records to 3526 (keeping 150 days of data) [2025-09-16T18:27:21.382294] [DEBUG] [LOCAL ] FGSN: Primary data saved (3526 records). [2025-09-16T18:27:21.382389] [INFO ] [MAIN ] FGSN: Full history saved. [2025-09-16T18:27:21.382482] [INFO ] [MAIN ] Update Progress: 825/2369 symbols completed. FGSN took 18.8s. Estimated remaining: 2h 43m 24s. Rate: 9.4 symbols/min [2025-09-16T18:27:21.560684] [INFO ] [MAIN ] FHN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:21.608807] [DEBUG] [LOCAL ] FHN: Loaded 39364 existing records from primary storage. [2025-09-16T18:27:21.609012] [INFO ] [MAIN ] FHN: Loaded 39364 existing records. [2025-09-16T18:27:21.801555] [WARN ] [MAIN ] FHN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:21.801864] [INFO ] [MAIN ] FHN: Wiping existing primary data before full refresh. [2025-09-16T18:27:21.801920] [INFO ] [LOCAL ] FHN: Starting deletion of primary data. [2025-09-16T18:27:21.803266] [DEBUG] [LOCAL ] FHN: Deleted primary data file. [2025-09-16T18:27:21.803331] [INFO ] [MAIN ] FHN: Starting full primary refresh. [2025-09-16T18:27:21.979950] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-09. [2025-09-16T18:27:22.161091] [INFO ] [REMOTE ] FHN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:22.658984] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-09-16T18:27:22.660248] [INFO ] [REMOTE ] FGN: Full fetch complete (4341 records). [2025-09-16T18:27:22.660327] [DEBUG] [VALIDATOR ] FGN: Validating primary data. [2025-09-16T18:27:22.779991] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-08. [2025-09-16T18:27:23.019730] [DEBUG] [VALIDATOR ] FGN: All expected trading days are present. [2025-09-16T18:27:23.019811] [DEBUG] [VALIDATOR ] FGN: Primary data validation passed. [2025-09-16T18:27:23.019955] [INFO ] [MAIN ] FGN: Fetched primary data is valid. [2025-09-16T18:27:23.019994] [DEBUG] [LOCAL ] FGN: Saving primary data. [2025-09-16T18:27:23.020344] [DEBUG] [LOCAL ] FGN: Trimmed 4341 records to 2748 (keeping 150 days of data) [2025-09-16T18:27:23.037411] [DEBUG] [LOCAL ] FGN: Primary data saved (2748 records). [2025-09-16T18:27:23.037470] [INFO ] [MAIN ] FGN: Full history saved. [2025-09-16T18:27:23.037564] [INFO ] [MAIN ] Update Progress: 826/2369 symbols completed. FGN took 23.4s. Estimated remaining: 2h 43m 8s. Rate: 9.5 symbols/min [2025-09-16T18:27:23.213842] [INFO ] [MAIN ] FI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:23.260212] [DEBUG] [LOCAL ] FI: Loaded 39603 existing records from primary storage. [2025-09-16T18:27:23.260351] [INFO ] [MAIN ] FI: Loaded 39603 existing records. [2025-09-16T18:27:23.433917] [WARN ] [MAIN ] FI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:23.434233] [INFO ] [MAIN ] FI: Wiping existing primary data before full refresh. [2025-09-16T18:27:23.434281] [INFO ] [LOCAL ] FI: Starting deletion of primary data. [2025-09-16T18:27:23.435138] [DEBUG] [LOCAL ] FI: Deleted primary data file. [2025-09-16T18:27:23.435188] [INFO ] [MAIN ] FI: Starting full primary refresh. [2025-09-16T18:27:23.577764] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-02. [2025-09-16T18:27:23.851427] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T18:27:23.855206] [INFO ] [REMOTE ] FI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:24.374902] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-09. [2025-09-16T18:27:25.156889] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-09-16T18:27:25.159556] [INFO ] [REMOTE ] FG: Full fetch complete (23260 records). [2025-09-16T18:27:25.159633] [DEBUG] [VALIDATOR ] FG: Validating primary data. [2025-09-16T18:27:25.193140] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-02. [2025-09-16T18:27:25.531190] [DEBUG] [VALIDATOR ] FG: All expected trading days are present. [2025-09-16T18:27:25.531547] [DEBUG] [VALIDATOR ] FG: Primary data validation passed. [2025-09-16T18:27:25.532047] [INFO ] [MAIN ] FG: Fetched primary data is valid. [2025-09-16T18:27:25.532181] [DEBUG] [LOCAL ] FG: Saving primary data. [2025-09-16T18:27:25.533289] [DEBUG] [LOCAL ] FG: Trimmed 23260 records to 15335 (keeping 150 days of data) [2025-09-16T18:27:25.634964] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-09-16T18:27:25.667871] [DEBUG] [LOCAL ] FG: Primary data saved (15335 records). [2025-09-16T18:27:25.667983] [INFO ] [MAIN ] FG: Full history saved. [2025-09-16T18:27:25.668091] [INFO ] [MAIN ] Update Progress: 827/2369 symbols completed. FG took 35.1s. Estimated remaining: 2h 42m 55s. Rate: 9.5 symbols/min [2025-09-16T18:27:25.839952] [INFO ] [MAIN ] FICO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:25.857431] [DEBUG] [LOCAL ] FICO: Loaded 16098 existing records from primary storage. [2025-09-16T18:27:25.857577] [INFO ] [MAIN ] FICO: Loaded 16098 existing records. [2025-09-16T18:27:25.979965] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-03. [2025-09-16T18:27:26.038192] [WARN ] [MAIN ] FICO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:26.038419] [INFO ] [MAIN ] FICO: Wiping existing primary data before full refresh. [2025-09-16T18:27:26.038466] [INFO ] [LOCAL ] FICO: Starting deletion of primary data. [2025-09-16T18:27:26.039052] [DEBUG] [LOCAL ] FICO: Deleted primary data file. [2025-09-16T18:27:26.039108] [INFO ] [MAIN ] FICO: Starting full primary refresh. [2025-09-16T18:27:26.413123] [INFO ] [REMOTE ] FICO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:26.433841] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-08 (5.25 seconds). [2025-09-16T18:27:26.709834] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T18:27:26.774851] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-03. [2025-09-16T18:27:27.529040] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T18:27:27.574871] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-09. [2025-09-16T18:27:28.256899] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T18:27:28.374909] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-04. [2025-09-16T18:27:29.047154] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T18:27:29.051868] [INFO ] [REMOTE ] FHI: Full fetch complete (43948 records). [2025-09-16T18:27:29.051954] [DEBUG] [VALIDATOR ] FHI: Validating primary data. [2025-09-16T18:27:29.180076] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-02. [2025-09-16T18:27:29.434896] [DEBUG] [VALIDATOR ] FHI: All expected trading days are present. [2025-09-16T18:27:29.434985] [DEBUG] [VALIDATOR ] FHI: Primary data validation passed. [2025-09-16T18:27:29.435660] [INFO ] [MAIN ] FHI: Fetched primary data is valid. [2025-09-16T18:27:29.435707] [DEBUG] [LOCAL ] FHI: Saving primary data. [2025-09-16T18:27:29.436868] [DEBUG] [LOCAL ] FHI: Trimmed 43948 records to 30062 (keeping 150 days of data) [2025-09-16T18:27:29.635377] [DEBUG] [LOCAL ] FHI: Primary data saved (30062 records). [2025-09-16T18:27:29.635525] [INFO ] [MAIN ] FHI: Full history saved. [2025-09-16T18:27:29.635632] [INFO ] [MAIN ] Update Progress: 828/2369 symbols completed. FHI took 26.6s. Estimated remaining: 2h 42m 44s. Rate: 9.5 symbols/min [2025-09-16T18:27:29.810629] [INFO ] [MAIN ] FIG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:29.810812] [DEBUG] [LOCAL ] FIG: No existing data file found in primary storage. [2025-09-16T18:27:29.810850] [WARN ] [MAIN ] FIG: No existing data found, performing full refresh instead. [2025-09-16T18:27:29.810971] [INFO ] [MAIN ] FIG: Wiping existing primary data before full refresh. [2025-09-16T18:27:29.811033] [INFO ] [LOCAL ] FIG: Starting deletion of primary data. [2025-09-16T18:27:29.811134] [DEBUG] [LOCAL ] FIG: No primary data file found to delete. [2025-09-16T18:27:29.811180] [INFO ] [MAIN ] FIG: Starting full primary refresh. [2025-09-16T18:27:29.975373] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-04. [2025-09-16T18:27:30.174986] [INFO ] [REMOTE ] FIG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:30.374503] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-09-16T18:27:30.484089] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-09-16T18:27:30.774924] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-02. [2025-09-16T18:27:31.176702] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-09-16T18:27:31.574933] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-05. [2025-09-16T18:27:31.698722] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T18:27:32.374898] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-05. [2025-09-16T18:27:33.174914] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-03. [2025-09-16T18:27:33.386527] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T18:27:33.482759] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-09-16T18:27:33.974899] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-06. [2025-09-16T18:27:34.026387] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T18:27:34.774904] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-06. [2025-09-16T18:27:35.574891] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-04. [2025-09-16T18:27:35.878882] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-09-16T18:27:35.909555] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-09-16T18:27:36.360620] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T18:27:36.374846] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-07. [2025-09-16T18:27:37.174960] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-05. [2025-09-16T18:27:37.458307] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-09-16T18:27:37.812391] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T18:27:37.974940] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-06. [2025-09-16T18:27:38.174121] [DEBUG] [REMOTE ] FIG: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:27:38.174487] [WARN ] [REMOTE ] FIG: No data returned for month 2025-06 [2025-09-16T18:27:38.774937] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-07. [2025-09-16T18:27:39.574936] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-03. [2025-09-16T18:27:40.374892] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-08. [2025-09-16T18:27:40.606801] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-09-16T18:27:40.944016] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-09-16T18:27:41.174923] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-04. [2025-09-16T18:27:41.868496] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T18:27:41.974911] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-09. [2025-09-16T18:27:42.286610] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-09-16T18:27:42.774935] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-08. [2025-09-16T18:27:43.574882] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-07. [2025-09-16T18:27:43.653234] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T18:27:43.659822] [INFO ] [REMOTE ] FHN: Full fetch complete (58023 records). [2025-09-16T18:27:43.659959] [DEBUG] [VALIDATOR ] FHN: Validating primary data. [2025-09-16T18:27:43.962249] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-09-16T18:27:44.051386] [DEBUG] [VALIDATOR ] FHN: All expected trading days are present. [2025-09-16T18:27:44.051473] [DEBUG] [VALIDATOR ] FHN: Primary data validation passed. [2025-09-16T18:27:44.052235] [INFO ] [MAIN ] FHN: Fetched primary data is valid. [2025-09-16T18:27:44.052280] [DEBUG] [LOCAL ] FHN: Saving primary data. [2025-09-16T18:27:44.053380] [DEBUG] [LOCAL ] FHN: Trimmed 58023 records to 39754 (keeping 150 days of data) [2025-09-16T18:27:44.303287] [DEBUG] [LOCAL ] FHN: Primary data saved (39754 records). [2025-09-16T18:27:44.303431] [INFO ] [MAIN ] FHN: Full history saved. [2025-09-16T18:27:44.303517] [INFO ] [MAIN ] Update Progress: 829/2369 symbols completed. FHN took 22.9s. Estimated remaining: 2h 42m 54s. Rate: 9.5 symbols/min [2025-09-16T18:27:44.384072] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-08. [2025-09-16T18:27:44.477441] [INFO ] [MAIN ] FIGS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:44.511741] [DEBUG] [LOCAL ] FIGS: Loaded 36014 existing records from primary storage. [2025-09-16T18:27:44.511964] [INFO ] [MAIN ] FIGS: Loaded 36014 existing records. [2025-09-16T18:27:44.702129] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T18:27:44.732661] [WARN ] [MAIN ] FIGS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:44.733268] [INFO ] [MAIN ] FIGS: Wiping existing primary data before full refresh. [2025-09-16T18:27:44.733319] [INFO ] [LOCAL ] FIGS: Starting deletion of primary data. [2025-09-16T18:27:44.734141] [DEBUG] [LOCAL ] FIGS: Deleted primary data file. [2025-09-16T18:27:44.734197] [INFO ] [MAIN ] FIGS: Starting full primary refresh. [2025-09-16T18:27:45.097582] [INFO ] [REMOTE ] FIGS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:45.174881] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-02. [2025-09-16T18:27:45.974900] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-05. [2025-09-16T18:27:46.774939] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-09. [2025-09-16T18:27:46.841347] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T18:27:47.068380] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-08 (2.68 seconds). [2025-09-16T18:27:47.197196] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T18:27:47.574893] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-03. [2025-09-16T18:27:48.374889] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-09. [2025-09-16T18:27:48.511231] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-09-16T18:27:48.517435] [INFO ] [REMOTE ] FI: Full fetch complete (58122 records). [2025-09-16T18:27:48.517524] [DEBUG] [VALIDATOR ] FI: Validating primary data. [2025-09-16T18:27:48.899871] [DEBUG] [VALIDATOR ] FI: All expected trading days are present. [2025-09-16T18:27:48.899953] [DEBUG] [VALIDATOR ] FI: Primary data validation passed. [2025-09-16T18:27:48.900721] [INFO ] [MAIN ] FI: Fetched primary data is valid. [2025-09-16T18:27:48.900766] [DEBUG] [LOCAL ] FI: Saving primary data. [2025-09-16T18:27:48.901935] [DEBUG] [LOCAL ] FI: Trimmed 58122 records to 39987 (keeping 150 days of data) [2025-09-16T18:27:49.147111] [DEBUG] [LOCAL ] FI: Primary data saved (39987 records). [2025-09-16T18:27:49.147280] [INFO ] [MAIN ] FI: Full history saved. [2025-09-16T18:27:49.147364] [INFO ] [MAIN ] Update Progress: 830/2369 symbols completed. FI took 26.1s. Estimated remaining: 2h 42m 44s. Rate: 9.5 symbols/min [2025-09-16T18:27:49.180039] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-06. [2025-09-16T18:27:49.325844] [INFO ] [MAIN ] FIHL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:49.358724] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T18:27:49.390026] [DEBUG] [LOCAL ] FIHL: Loaded 28558 existing records from primary storage. [2025-09-16T18:27:49.390156] [INFO ] [MAIN ] FIHL: Loaded 28558 existing records. [2025-09-16T18:27:49.606684] [WARN ] [MAIN ] FIHL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:49.606967] [INFO ] [MAIN ] FIHL: Wiping existing primary data before full refresh. [2025-09-16T18:27:49.607015] [INFO ] [LOCAL ] FIHL: Starting deletion of primary data. [2025-09-16T18:27:49.607715] [DEBUG] [LOCAL ] FIHL: Deleted primary data file. [2025-09-16T18:27:49.607772] [INFO ] [MAIN ] FIHL: Starting full primary refresh. [2025-09-16T18:27:49.969064] [INFO ] [REMOTE ] FIHL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:49.974855] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-02. [2025-09-16T18:27:50.480958] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T18:27:50.774913] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-07. [2025-09-16T18:27:50.983478] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-09 (2.61 seconds). [2025-09-16T18:27:50.989741] [INFO ] [REMOTE ] FIG: Full fetch complete (12943 records). [2025-09-16T18:27:50.989832] [DEBUG] [VALIDATOR ] FIG: Validating primary data. [2025-09-16T18:27:51.167715] [WARN ] [VALIDATOR ] FIG: Validation failed - insufficient historical data. Have 102 unique days, need 150 days [2025-09-16T18:27:51.168021] [FATAL] [MAIN ] FIG: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:27:51.168074] [WARN ] [MAIN ] FIG: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:27:51.168118] [INFO ] [LOCAL ] FIG: Starting deletion of primary data. [2025-09-16T18:27:51.168178] [DEBUG] [LOCAL ] FIG: No primary data file found to delete. [2025-09-16T18:27:51.168215] [INFO ] [LOCAL ] FIG: Starting deletion of derivative data. [2025-09-16T18:27:51.168266] [DEBUG] [LOCAL ] FIG: No derivative data file found to delete. [2025-09-16T18:27:51.168328] [INFO ] [MAIN ] Update Progress: 831/2369 symbols completed. FIG took 21.5s. Estimated remaining: 2h 42m 30s. Rate: 9.5 symbols/min [2025-09-16T18:27:51.341865] [INFO ] [MAIN ] FINS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:51.350246] [DEBUG] [LOCAL ] FINS: Loaded 6401 existing records from primary storage. [2025-09-16T18:27:51.350403] [INFO ] [MAIN ] FINS: Loaded 6401 existing records. [2025-09-16T18:27:51.474118] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-09-16T18:27:51.550457] [WARN ] [MAIN ] FINS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:51.550688] [INFO ] [MAIN ] FINS: Wiping existing primary data before full refresh. [2025-09-16T18:27:51.550737] [INFO ] [LOCAL ] FINS: Starting deletion of primary data. [2025-09-16T18:27:51.551034] [DEBUG] [LOCAL ] FINS: Deleted primary data file. [2025-09-16T18:27:51.551078] [INFO ] [MAIN ] FINS: Starting full primary refresh. [2025-09-16T18:27:51.581443] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-04. [2025-09-16T18:27:51.910419] [INFO ] [REMOTE ] FINS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:52.338945] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-09-16T18:27:52.374923] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-08. [2025-09-16T18:27:53.099385] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T18:27:53.174943] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-05. [2025-09-16T18:27:53.716007] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T18:27:53.974908] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-09. [2025-09-16T18:27:54.787721] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-02. [2025-09-16T18:27:54.790268] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T18:27:55.156539] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T18:27:55.159491] [INFO ] [REMOTE ] FICO: Full fetch complete (21700 records). [2025-09-16T18:27:55.159584] [DEBUG] [VALIDATOR ] FICO: Validating primary data. [2025-09-16T18:27:55.539118] [DEBUG] [VALIDATOR ] FICO: All expected trading days are present. [2025-09-16T18:27:55.539199] [DEBUG] [VALIDATOR ] FICO: Primary data validation passed. [2025-09-16T18:27:55.539715] [INFO ] [MAIN ] FICO: Fetched primary data is valid. [2025-09-16T18:27:55.539761] [DEBUG] [LOCAL ] FICO: Saving primary data. [2025-09-16T18:27:55.540378] [DEBUG] [LOCAL ] FICO: Trimmed 21700 records to 16209 (keeping 150 days of data) [2025-09-16T18:27:55.589583] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-06. [2025-09-16T18:27:55.637789] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-09-16T18:27:55.661016] [DEBUG] [LOCAL ] FICO: Primary data saved (16209 records). [2025-09-16T18:27:55.661207] [INFO ] [MAIN ] FICO: Full history saved. [2025-09-16T18:27:55.661311] [INFO ] [MAIN ] Update Progress: 832/2369 symbols completed. FICO took 30.0s. Estimated remaining: 2h 42m 20s. Rate: 9.5 symbols/min [2025-09-16T18:27:55.833707] [INFO ] [MAIN ] FINV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:27:55.875442] [DEBUG] [LOCAL ] FINV: Loaded 32958 existing records from primary storage. [2025-09-16T18:27:55.875606] [INFO ] [MAIN ] FINV: Loaded 32958 existing records. [2025-09-16T18:27:56.058202] [WARN ] [MAIN ] FINV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:27:56.058463] [INFO ] [MAIN ] FINV: Wiping existing primary data before full refresh. [2025-09-16T18:27:56.058519] [INFO ] [LOCAL ] FINV: Starting deletion of primary data. [2025-09-16T18:27:56.059267] [DEBUG] [LOCAL ] FINV: Deleted primary data file. [2025-09-16T18:27:56.059319] [INFO ] [MAIN ] FINV: Starting full primary refresh. [2025-09-16T18:27:56.382133] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-03. [2025-09-16T18:27:56.421454] [INFO ] [REMOTE ] FINV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:27:57.174920] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-02. [2025-09-16T18:27:57.280969] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-09-16T18:27:57.316538] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-09-16T18:27:57.974916] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-03. [2025-09-16T18:27:58.775007] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-07. [2025-09-16T18:27:58.917826] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T18:27:59.559781] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-09-16T18:27:59.574852] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-04. [2025-09-16T18:28:00.374900] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-03. [2025-09-16T18:28:00.483027] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T18:28:01.103298] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-09-16T18:28:01.174858] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-05. [2025-09-16T18:28:01.788255] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-09-16T18:28:01.974940] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-04. [2025-09-16T18:28:02.774937] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-08. [2025-09-16T18:28:02.794830] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-09-16T18:28:03.070678] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-09-16T18:28:03.574946] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-05. [2025-09-16T18:28:04.361373] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-09-16T18:28:04.374840] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-06. [2025-09-16T18:28:04.548074] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T18:28:05.158424] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T18:28:05.174857] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-07. [2025-09-16T18:28:05.974938] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-09. [2025-09-16T18:28:05.991685] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-09-16T18:28:06.774920] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-06. [2025-09-16T18:28:07.574915] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-08. [2025-09-16T18:28:07.670864] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T18:28:07.676622] [INFO ] [REMOTE ] FIGS: Full fetch complete (53284 records). [2025-09-16T18:28:07.676741] [DEBUG] [VALIDATOR ] FIGS: Validating primary data. [2025-09-16T18:28:08.047815] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-09-16T18:28:08.065611] [DEBUG] [VALIDATOR ] FIGS: All expected trading days are present. [2025-09-16T18:28:08.065813] [DEBUG] [VALIDATOR ] FIGS: Primary data validation passed. [2025-09-16T18:28:08.066888] [INFO ] [MAIN ] FIGS: Fetched primary data is valid. [2025-09-16T18:28:08.066947] [DEBUG] [LOCAL ] FIGS: Saving primary data. [2025-09-16T18:28:08.068305] [DEBUG] [LOCAL ] FIGS: Trimmed 53284 records to 36336 (keeping 150 days of data) [2025-09-16T18:28:08.213288] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T18:28:08.314835] [DEBUG] [LOCAL ] FIGS: Primary data saved (36336 records). [2025-09-16T18:28:08.314995] [INFO ] [MAIN ] FIGS: Full history saved. [2025-09-16T18:28:08.315090] [INFO ] [MAIN ] Update Progress: 833/2369 symbols completed. FIGS took 24.0s. Estimated remaining: 2h 42m 26s. Rate: 9.5 symbols/min [2025-09-16T18:28:08.380168] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-07. [2025-09-16T18:28:08.489177] [INFO ] [MAIN ] FIS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:28:08.525179] [DEBUG] [LOCAL ] FIS: Loaded 39350 existing records from primary storage. [2025-09-16T18:28:08.525338] [INFO ] [MAIN ] FIS: Loaded 39350 existing records. [2025-09-16T18:28:08.705512] [WARN ] [MAIN ] FIS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:28:08.705777] [INFO ] [MAIN ] FIS: Wiping existing primary data before full refresh. [2025-09-16T18:28:08.705823] [INFO ] [LOCAL ] FIS: Starting deletion of primary data. [2025-09-16T18:28:08.706747] [DEBUG] [LOCAL ] FIS: Deleted primary data file. [2025-09-16T18:28:08.706799] [INFO ] [MAIN ] FIS: Starting full primary refresh. [2025-09-16T18:28:09.068844] [INFO ] [REMOTE ] FIS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:28:09.174923] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-09. [2025-09-16T18:28:09.769267] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-09-16T18:28:09.770736] [INFO ] [REMOTE ] FINS: Full fetch complete (8767 records). [2025-09-16T18:28:09.770819] [DEBUG] [VALIDATOR ] FINS: Validating primary data. [2025-09-16T18:28:09.983229] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-04. [2025-09-16T18:28:10.136773] [DEBUG] [VALIDATOR ] FINS: All expected trading days are present. [2025-09-16T18:28:10.136890] [DEBUG] [VALIDATOR ] FINS: Primary data validation passed. [2025-09-16T18:28:10.137151] [INFO ] [MAIN ] FINS: Fetched primary data is valid. [2025-09-16T18:28:10.137202] [DEBUG] [LOCAL ] FINS: Saving primary data. [2025-09-16T18:28:10.137633] [DEBUG] [LOCAL ] FINS: Trimmed 8767 records to 6435 (keeping 150 days of data) [2025-09-16T18:28:10.189520] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T18:28:10.206865] [DEBUG] [LOCAL ] FINS: Primary data saved (6435 records). [2025-09-16T18:28:10.206951] [INFO ] [MAIN ] FINS: Full history saved. [2025-09-16T18:28:10.207028] [INFO ] [MAIN ] Update Progress: 834/2369 symbols completed. FINS took 19.0s. Estimated remaining: 2h 42m 11s. Rate: 9.5 symbols/min [2025-09-16T18:28:10.380933] [INFO ] [MAIN ] FIX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:28:10.400710] [DEBUG] [LOCAL ] FIX: Loaded 21071 existing records from primary storage. [2025-09-16T18:28:10.400873] [INFO ] [MAIN ] FIX: Loaded 21071 existing records. [2025-09-16T18:28:10.574946] [WARN ] [MAIN ] FIX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:28:10.575234] [INFO ] [MAIN ] FIX: Wiping existing primary data before full refresh. [2025-09-16T18:28:10.575288] [INFO ] [LOCAL ] FIX: Starting deletion of primary data. [2025-09-16T18:28:10.575930] [DEBUG] [LOCAL ] FIX: Deleted primary data file. [2025-09-16T18:28:10.575981] [INFO ] [MAIN ] FIX: Starting full primary refresh. [2025-09-16T18:28:10.780625] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-08. [2025-09-16T18:28:10.929099] [INFO ] [REMOTE ] FIX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:28:11.574869] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-02. [2025-09-16T18:28:11.839368] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-09-16T18:28:12.324937] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T18:28:12.374878] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-09. [2025-09-16T18:28:12.821554] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T18:28:13.175001] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-03. [2025-09-16T18:28:13.615082] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T18:28:13.620034] [INFO ] [REMOTE ] FIHL: Full fetch complete (43922 records). [2025-09-16T18:28:13.620116] [DEBUG] [VALIDATOR ] FIHL: Validating primary data. [2025-09-16T18:28:13.981759] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-05. [2025-09-16T18:28:13.996825] [DEBUG] [VALIDATOR ] FIHL: All expected trading days are present. [2025-09-16T18:28:13.997821] [DEBUG] [VALIDATOR ] FIHL: Primary data validation passed. [2025-09-16T18:28:13.998428] [INFO ] [MAIN ] FIHL: Fetched primary data is valid. [2025-09-16T18:28:13.998470] [DEBUG] [LOCAL ] FIHL: Saving primary data. [2025-09-16T18:28:13.999443] [DEBUG] [LOCAL ] FIHL: Trimmed 43922 records to 28860 (keeping 150 days of data) [2025-09-16T18:28:14.171088] [DEBUG] [LOCAL ] FIHL: Primary data saved (28860 records). [2025-09-16T18:28:14.171242] [INFO ] [MAIN ] FIHL: Full history saved. [2025-09-16T18:28:14.171358] [INFO ] [MAIN ] Update Progress: 835/2369 symbols completed. FIHL took 25.0s. Estimated remaining: 2h 42m 0s. Rate: 9.5 symbols/min [2025-09-16T18:28:14.342726] [INFO ] [MAIN ] FIXT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:28:14.342865] [DEBUG] [LOCAL ] FIXT: No existing data file found in primary storage. [2025-09-16T18:28:14.342901] [WARN ] [MAIN ] FIXT: No existing data found, performing full refresh instead. [2025-09-16T18:28:14.343015] [INFO ] [MAIN ] FIXT: Wiping existing primary data before full refresh. [2025-09-16T18:28:14.343052] [INFO ] [LOCAL ] FIXT: Starting deletion of primary data. [2025-09-16T18:28:14.343093] [DEBUG] [LOCAL ] FIXT: No primary data file found to delete. [2025-09-16T18:28:14.343124] [INFO ] [MAIN ] FIXT: Starting full primary refresh. [2025-09-16T18:28:14.692251] [INFO ] [REMOTE ] FIXT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:28:14.747788] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-09-16T18:28:14.774836] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-04. [2025-09-16T18:28:15.528513] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-09-16T18:28:15.574891] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-02. [2025-09-16T18:28:15.766493] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-09-16T18:28:16.374941] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-02. [2025-09-16T18:28:16.641324] [DEBUG] [REMOTE ] FIXT: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:28:16.641689] [WARN ] [REMOTE ] FIXT: No data returned for month 2025-02 [2025-09-16T18:28:17.174913] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-05. [2025-09-16T18:28:17.335224] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-09-16T18:28:17.974916] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-03. [2025-09-16T18:28:18.314573] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T18:28:18.774904] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-06. [2025-09-16T18:28:19.483544] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-09-16T18:28:19.574860] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-04. [2025-09-16T18:28:20.013997] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T18:28:20.374943] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-07. [2025-09-16T18:28:21.174944] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-03. [2025-09-16T18:28:21.547895] [DEBUG] [REMOTE ] FIXT: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:28:21.551249] [WARN ] [REMOTE ] FIXT: No data returned for month 2025-03 [2025-09-16T18:28:21.553839] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-09-16T18:28:21.746939] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T18:28:21.974908] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-05. [2025-09-16T18:28:22.774899] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-04. [2025-09-16T18:28:22.979754] [DEBUG] [REMOTE ] FIXT: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:28:22.980254] [WARN ] [REMOTE ] FIXT: No data returned for month 2025-04 [2025-09-16T18:28:23.574899] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-05. [2025-09-16T18:28:23.808522] [DEBUG] [REMOTE ] FIXT: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:28:23.808854] [WARN ] [REMOTE ] FIXT: No data returned for month 2025-05 [2025-09-16T18:28:24.374906] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-06. [2025-09-16T18:28:25.017034] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-05 (3.04 seconds). [2025-09-16T18:28:25.100871] [INFO ] [REMOTE ] FIXT: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-09-16T18:28:25.174926] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-06. [2025-09-16T18:28:25.974916] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-08. [2025-09-16T18:28:26.774907] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-06. [2025-09-16T18:28:26.894358] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-09-16T18:28:27.574917] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-07. [2025-09-16T18:28:27.597240] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-09-16T18:28:28.041264] [INFO ] [REMOTE ] FIXT: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-09-16T18:28:28.222926] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T18:28:28.374870] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-07. [2025-09-16T18:28:29.174890] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-08. [2025-09-16T18:28:29.638187] [INFO ] [REMOTE ] FIXT: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-09-16T18:28:29.974895] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-09. [2025-09-16T18:28:30.159369] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-09-16T18:28:30.774904] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-09. [2025-09-16T18:28:31.357143] [INFO ] [REMOTE ] FIXT: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-09-16T18:28:31.358082] [INFO ] [REMOTE ] FIXT: Full fetch complete (1654 records). [2025-09-16T18:28:31.358152] [DEBUG] [VALIDATOR ] FIXT: Validating primary data. [2025-09-16T18:28:31.536258] [WARN ] [VALIDATOR ] FIXT: Validation failed - insufficient historical data. Have 64 unique days, need 150 days [2025-09-16T18:28:31.536548] [FATAL] [MAIN ] FIXT: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:28:31.536693] [WARN ] [MAIN ] FIXT: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:28:31.536854] [INFO ] [LOCAL ] FIXT: Starting deletion of primary data. [2025-09-16T18:28:31.545305] [DEBUG] [LOCAL ] FIXT: No primary data file found to delete. [2025-09-16T18:28:31.545411] [INFO ] [LOCAL ] FIXT: Starting deletion of derivative data. [2025-09-16T18:28:31.546234] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T18:28:31.545938] [DEBUG] [LOCAL ] FIXT: No derivative data file found to delete. [2025-09-16T18:28:31.549495] [INFO ] [MAIN ] Update Progress: 836/2369 symbols completed. FIXT took 17.4s. Estimated remaining: 2h 42m 14s. Rate: 9.4 symbols/min [2025-09-16T18:28:31.549968] [INFO ] [REMOTE ] FIX: Full fetch complete (34784 records). [2025-09-16T18:28:31.590355] [DEBUG] [VALIDATOR ] FIX: Validating primary data. [2025-09-16T18:28:31.585139] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-08. [2025-09-16T18:28:31.852552] [INFO ] [MAIN ] FLC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:28:32.012216] [DEBUG] [LOCAL ] FLC: Loaded 3882 existing records from primary storage. [2025-09-16T18:28:32.014529] [INFO ] [MAIN ] FLC: Loaded 3882 existing records. [2025-09-16T18:28:32.244344] [DEBUG] [VALIDATOR ] FIX: All expected trading days are present. [2025-09-16T18:28:32.278315] [DEBUG] [VALIDATOR ] FIX: Primary data validation passed. [2025-09-16T18:28:32.282526] [WARN ] [MAIN ] FLC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:28:32.283405] [INFO ] [MAIN ] FIX: Fetched primary data is valid. [2025-09-16T18:28:32.283728] [INFO ] [MAIN ] FLC: Wiping existing primary data before full refresh. [2025-09-16T18:28:32.283626] [DEBUG] [LOCAL ] FIX: Saving primary data. [2025-09-16T18:28:32.284009] [INFO ] [LOCAL ] FLC: Starting deletion of primary data. [2025-09-16T18:28:32.284449] [DEBUG] [LOCAL ] FLC: Deleted primary data file. [2025-09-16T18:28:32.284572] [INFO ] [MAIN ] FLC: Starting full primary refresh. [2025-09-16T18:28:32.290096] [DEBUG] [LOCAL ] FIX: Trimmed 34784 records to 21252 (keeping 150 days of data) [2025-09-16T18:28:32.381614] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-07. [2025-09-16T18:28:32.790215] [INFO ] [REMOTE ] FLC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:28:32.815099] [DEBUG] [LOCAL ] FIX: Primary data saved (21252 records). [2025-09-16T18:28:32.815257] [INFO ] [MAIN ] FIX: Full history saved. [2025-09-16T18:28:32.815357] [INFO ] [MAIN ] Update Progress: 837/2369 symbols completed. FIX took 22.6s. Estimated remaining: 2h 41m 59s. Rate: 9.5 symbols/min [2025-09-16T18:28:33.174906] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-02. [2025-09-16T18:28:33.468911] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-09-16T18:28:33.841658] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-09-16T18:28:33.977975] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-03. [2025-09-16T18:28:33.983839] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T18:28:34.681521] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-09-16T18:28:34.774899] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-04. [2025-09-16T18:28:35.532815] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-09-16T18:28:35.574866] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-09. [2025-09-16T18:28:36.374894] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-05. [2025-09-16T18:28:36.858075] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-09-16T18:28:37.174915] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-08. [2025-09-16T18:28:37.213536] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T18:28:37.219725] [INFO ] [REMOTE ] FINV: Full fetch complete (48867 records). [2025-09-16T18:28:37.219834] [DEBUG] [VALIDATOR ] FINV: Validating primary data. [2025-09-16T18:28:37.597739] [DEBUG] [VALIDATOR ] FINV: All expected trading days are present. [2025-09-16T18:28:37.597828] [DEBUG] [VALIDATOR ] FINV: Primary data validation passed. [2025-09-16T18:28:37.598519] [INFO ] [MAIN ] FINV: Fetched primary data is valid. [2025-09-16T18:28:37.598563] [DEBUG] [LOCAL ] FINV: Saving primary data. [2025-09-16T18:28:37.599633] [DEBUG] [LOCAL ] FINV: Trimmed 48867 records to 33313 (keeping 150 days of data) [2025-09-16T18:28:37.788982] [DEBUG] [LOCAL ] FINV: Primary data saved (33313 records). [2025-09-16T18:28:37.789125] [INFO ] [MAIN ] FINV: Full history saved. [2025-09-16T18:28:37.790522] [INFO ] [MAIN ] Update Progress: 838/2369 symbols completed. FINV took 42.1s. Estimated remaining: 2h 41m 50s. Rate: 9.5 symbols/min [2025-09-16T18:28:37.974929] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-06. [2025-09-16T18:28:38.563357] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T18:28:38.774951] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-07. [2025-09-16T18:28:39.083714] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-09-16T18:28:39.391739] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-09-16T18:28:39.574960] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-08. [2025-09-16T18:28:40.342869] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-09-16T18:28:40.374850] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-09. [2025-09-16T18:28:41.047599] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-09-16T18:28:41.049062] [INFO ] [REMOTE ] FLC: Full fetch complete (6118 records). [2025-09-16T18:28:41.049148] [DEBUG] [VALIDATOR ] FLC: Validating primary data. [2025-09-16T18:28:41.179974] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-09. [2025-09-16T18:28:41.419210] [DEBUG] [VALIDATOR ] FLC: All expected trading days are present. [2025-09-16T18:28:41.419291] [DEBUG] [VALIDATOR ] FLC: Primary data validation passed. [2025-09-16T18:28:41.419468] [INFO ] [MAIN ] FLC: Fetched primary data is valid. [2025-09-16T18:28:41.419510] [DEBUG] [LOCAL ] FLC: Saving primary data. [2025-09-16T18:28:41.419929] [DEBUG] [LOCAL ] FLC: Trimmed 6118 records to 3942 (keeping 150 days of data) [2025-09-16T18:28:41.446410] [DEBUG] [LOCAL ] FLC: Primary data saved (3942 records). [2025-09-16T18:28:41.446571] [INFO ] [MAIN ] FLC: Full history saved. [2025-09-16T18:28:41.446704] [INFO ] [MAIN ] Update Progress: 839/2369 symbols completed. FLC took 9.9s. Estimated remaining: 2h 41m 39s. Rate: 9.5 symbols/min [2025-09-16T18:28:43.089718] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-09-16T18:28:43.096147] [INFO ] [REMOTE ] FIS: Full fetch complete (58049 records). [2025-09-16T18:28:43.096286] [DEBUG] [VALIDATOR ] FIS: Validating primary data. [2025-09-16T18:28:43.498146] [DEBUG] [VALIDATOR ] FIS: All expected trading days are present. [2025-09-16T18:28:43.498233] [DEBUG] [VALIDATOR ] FIS: Primary data validation passed. [2025-09-16T18:28:43.499100] [INFO ] [MAIN ] FIS: Fetched primary data is valid. [2025-09-16T18:28:43.499155] [DEBUG] [LOCAL ] FIS: Saving primary data. [2025-09-16T18:28:43.500410] [DEBUG] [LOCAL ] FIS: Trimmed 58049 records to 39736 (keeping 150 days of data) [2025-09-16T18:28:43.754895] [DEBUG] [LOCAL ] FIS: Primary data saved (39736 records). [2025-09-16T18:28:43.755084] [INFO ] [MAIN ] FIS: Full history saved. [2025-09-16T18:28:43.755184] [INFO ] [MAIN ] Update Progress: 840/2369 symbols completed. FIS took 35.4s. Estimated remaining: 2h 41m 25s. Rate: 9.5 symbols/min [2025-09-16T18:28:44.294287] [INFO ] [MAIN ] FLNG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:28:44.469314] [INFO ] [MAIN ] FLOC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:28:44.471280] [INFO ] [MAIN ] FLG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:28:44.496312] [INFO ] [MAIN ] FLO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:28:44.531234] [DEBUG] [LOCAL ] FLOC: Loaded 20511 existing records from primary storage. [2025-09-16T18:28:44.532271] [DEBUG] [LOCAL ] FLNG: Loaded 24804 existing records from primary storage. [2025-09-16T18:28:44.541994] [INFO ] [MAIN ] FLNG: Loaded 24804 existing records. [2025-09-16T18:28:44.541878] [INFO ] [MAIN ] FLOC: Loaded 20511 existing records. [2025-09-16T18:28:44.780423] [DEBUG] [LOCAL ] FLG: Loaded 39349 existing records from primary storage. [2025-09-16T18:28:44.791541] [INFO ] [MAIN ] FLG: Loaded 39349 existing records. [2025-09-16T18:28:44.926902] [DEBUG] [LOCAL ] FLO: Loaded 38409 existing records from primary storage. [2025-09-16T18:28:44.995859] [INFO ] [MAIN ] FLO: Loaded 38409 existing records. [2025-09-16T18:28:45.014710] [WARN ] [MAIN ] FLOC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:28:44.995990] [WARN ] [MAIN ] FLNG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:28:45.070202] [INFO ] [MAIN ] FLOC: Wiping existing primary data before full refresh. [2025-09-16T18:28:45.085670] [INFO ] [MAIN ] FLNG: Wiping existing primary data before full refresh. [2025-09-16T18:28:45.095836] [INFO ] [LOCAL ] FLNG: Starting deletion of primary data. [2025-09-16T18:28:45.092210] [INFO ] [LOCAL ] FLOC: Starting deletion of primary data. [2025-09-16T18:28:45.101772] [DEBUG] [LOCAL ] FLNG: Deleted primary data file. [2025-09-16T18:28:45.110369] [DEBUG] [LOCAL ] FLOC: Deleted primary data file. [2025-09-16T18:28:45.135111] [INFO ] [MAIN ] FLNG: Starting full primary refresh. [2025-09-16T18:28:45.129480] [INFO ] [MAIN ] FLOC: Starting full primary refresh. [2025-09-16T18:28:45.430479] [WARN ] [MAIN ] FLG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:28:45.518157] [WARN ] [MAIN ] FLO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:28:45.533740] [INFO ] [MAIN ] FLG: Wiping existing primary data before full refresh. [2025-09-16T18:28:45.550355] [INFO ] [LOCAL ] FLG: Starting deletion of primary data. [2025-09-16T18:28:45.606679] [DEBUG] [LOCAL ] FLG: Deleted primary data file. [2025-09-16T18:28:45.574393] [INFO ] [MAIN ] FLO: Wiping existing primary data before full refresh. [2025-09-16T18:28:45.617127] [INFO ] [MAIN ] FLG: Starting full primary refresh. [2025-09-16T18:28:45.617220] [INFO ] [LOCAL ] FLO: Starting deletion of primary data. [2025-09-16T18:28:45.712870] [DEBUG] [LOCAL ] FLO: Deleted primary data file. [2025-09-16T18:28:45.782357] [INFO ] [MAIN ] FLO: Starting full primary refresh. [2025-09-16T18:28:46.283368] [INFO ] [REMOTE ] FLNG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:28:46.309024] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-02. [2025-09-16T18:28:46.506248] [INFO ] [REMOTE ] FLOC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:28:46.524857] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-02. [2025-09-16T18:28:46.677432] [INFO ] [REMOTE ] FLG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:28:46.688182] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-02. [2025-09-16T18:28:46.765052] [INFO ] [REMOTE ] FLO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:28:46.765345] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-02. [2025-09-16T18:28:48.090005] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T18:28:48.095930] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-03. [2025-09-16T18:28:48.212866] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-09-16T18:28:48.220475] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-03. [2025-09-16T18:28:48.325950] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:28:48.335932] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-03. [2025-09-16T18:28:48.496837] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T18:28:48.506616] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-03. [2025-09-16T18:28:49.250072] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T18:28:49.257449] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-04. [2025-09-16T18:28:49.429895] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-09-16T18:28:49.437717] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-04. [2025-09-16T18:28:50.159397] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-09-16T18:28:50.199513] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-04. [2025-09-16T18:28:50.202525] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-09-16T18:28:50.510502] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-09-16T18:28:50.554304] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T18:28:50.774915] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-05. [2025-09-16T18:28:51.574960] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-05. [2025-09-16T18:28:51.632062] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T18:28:52.161883] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T18:28:52.374967] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-06. [2025-09-16T18:28:52.752245] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-09-16T18:28:53.174901] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-05. [2025-09-16T18:28:53.522797] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-09-16T18:28:53.974959] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-04. [2025-09-16T18:28:54.774900] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-07. [2025-09-16T18:28:54.995342] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-09-16T18:28:55.463382] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T18:28:55.574868] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-06. [2025-09-16T18:28:55.752407] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-09-16T18:28:56.374901] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-06. [2025-09-16T18:28:57.174891] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-08. [2025-09-16T18:28:57.211446] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T18:28:57.753308] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T18:28:57.974916] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-07. [2025-09-16T18:28:58.163605] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T18:28:58.774913] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-05. [2025-09-16T18:28:59.574939] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-09. [2025-09-16T18:28:59.790451] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T18:29:00.102948] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T18:29:00.374925] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-06. [2025-09-16T18:29:01.051264] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T18:29:01.054722] [INFO ] [REMOTE ] FLOC: Full fetch complete (31627 records). [2025-09-16T18:29:01.054835] [DEBUG] [VALIDATOR ] FLOC: Validating primary data. [2025-09-16T18:29:01.179969] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-07. [2025-09-16T18:29:01.432275] [DEBUG] [VALIDATOR ] FLOC: All expected trading days are present. [2025-09-16T18:29:01.432356] [DEBUG] [VALIDATOR ] FLOC: Primary data validation passed. [2025-09-16T18:29:01.432931] [INFO ] [MAIN ] FLOC: Fetched primary data is valid. [2025-09-16T18:29:01.433014] [DEBUG] [LOCAL ] FLOC: Saving primary data. [2025-09-16T18:29:01.433826] [DEBUG] [LOCAL ] FLOC: Trimmed 31627 records to 20716 (keeping 150 days of data) [2025-09-16T18:29:01.582967] [DEBUG] [LOCAL ] FLOC: Primary data saved (20716 records). [2025-09-16T18:29:01.583106] [INFO ] [MAIN ] FLOC: Full history saved. [2025-09-16T18:29:01.583296] [INFO ] [MAIN ] Update Progress: 841/2369 symbols completed. FLOC took 17.8s. Estimated remaining: 2h 41m 39s. Rate: 9.5 symbols/min [2025-09-16T18:29:01.796353] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T18:29:01.798595] [INFO ] [MAIN ] FLR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:29:01.809054] [DEBUG] [LOCAL ] FLR: No existing data file found in primary storage. [2025-09-16T18:29:01.809278] [WARN ] [MAIN ] FLR: No existing data found, performing full refresh instead. [2025-09-16T18:29:01.809839] [INFO ] [MAIN ] FLR: Wiping existing primary data before full refresh. [2025-09-16T18:29:01.809882] [INFO ] [LOCAL ] FLR: Starting deletion of primary data. [2025-09-16T18:29:01.809937] [DEBUG] [LOCAL ] FLR: No primary data file found to delete. [2025-09-16T18:29:01.810347] [INFO ] [MAIN ] FLR: Starting full primary refresh. [2025-09-16T18:29:01.983431] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-07. [2025-09-16T18:29:02.168444] [INFO ] [REMOTE ] FLR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:29:02.748878] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T18:29:02.774849] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-08. [2025-09-16T18:29:03.379678] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-09-16T18:29:03.574877] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-08. [2025-09-16T18:29:03.816189] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-09-16T18:29:04.374902] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-02. [2025-09-16T18:29:05.174885] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-08. [2025-09-16T18:29:05.477387] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-09-16T18:29:05.974888] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-09. [2025-09-16T18:29:06.112122] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T18:29:06.558504] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T18:29:06.774878] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-09. [2025-09-16T18:29:07.574899] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-03. [2025-09-16T18:29:07.688349] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T18:29:07.694742] [INFO ] [REMOTE ] FLG: Full fetch complete (58009 records). [2025-09-16T18:29:07.694858] [DEBUG] [VALIDATOR ] FLG: Validating primary data. [2025-09-16T18:29:08.097358] [DEBUG] [VALIDATOR ] FLG: All expected trading days are present. [2025-09-16T18:29:08.097476] [DEBUG] [VALIDATOR ] FLG: Primary data validation passed. [2025-09-16T18:29:08.098347] [INFO ] [MAIN ] FLG: Fetched primary data is valid. [2025-09-16T18:29:08.098408] [DEBUG] [LOCAL ] FLG: Saving primary data. [2025-09-16T18:29:08.099846] [DEBUG] [LOCAL ] FLG: Trimmed 58009 records to 39734 (keeping 150 days of data) [2025-09-16T18:29:08.242891] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T18:29:08.251422] [INFO ] [REMOTE ] FLO: Full fetch complete (56357 records). [2025-09-16T18:29:08.252008] [DEBUG] [VALIDATOR ] FLO: Validating primary data. [2025-09-16T18:29:08.425988] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-09. [2025-09-16T18:29:08.760274] [DEBUG] [VALIDATOR ] FLO: All expected trading days are present. [2025-09-16T18:29:08.761395] [DEBUG] [VALIDATOR ] FLO: Primary data validation passed. [2025-09-16T18:29:08.764243] [INFO ] [MAIN ] FLO: Fetched primary data is valid. [2025-09-16T18:29:08.764936] [DEBUG] [LOCAL ] FLO: Saving primary data. [2025-09-16T18:29:08.775880] [DEBUG] [LOCAL ] FLO: Trimmed 56357 records to 38799 (keeping 150 days of data) [2025-09-16T18:29:08.784675] [DEBUG] [LOCAL ] FLG: Primary data saved (39734 records). [2025-09-16T18:29:08.784821] [INFO ] [MAIN ] FLG: Full history saved. [2025-09-16T18:29:08.784925] [INFO ] [MAIN ] Update Progress: 842/2369 symbols completed. FLG took 25.0s. Estimated remaining: 2h 41m 35s. Rate: 9.5 symbols/min [2025-09-16T18:29:09.055203] [INFO ] [MAIN ] FLS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:29:09.110634] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-09-16T18:29:09.136486] [DEBUG] [LOCAL ] FLS: Loaded 36835 existing records from primary storage. [2025-09-16T18:29:09.137106] [INFO ] [MAIN ] FLS: Loaded 36835 existing records. [2025-09-16T18:29:09.181114] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-04. [2025-09-16T18:29:09.326874] [WARN ] [MAIN ] FLS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:29:09.327137] [INFO ] [MAIN ] FLS: Wiping existing primary data before full refresh. [2025-09-16T18:29:09.327185] [INFO ] [LOCAL ] FLS: Starting deletion of primary data. [2025-09-16T18:29:09.377866] [DEBUG] [LOCAL ] FLS: Deleted primary data file. [2025-09-16T18:29:09.386075] [INFO ] [MAIN ] FLS: Starting full primary refresh. [2025-09-16T18:29:09.782767] [INFO ] [REMOTE ] FLS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:29:09.805555] [DEBUG] [LOCAL ] FLO: Primary data saved (38799 records). [2025-09-16T18:29:09.805683] [INFO ] [MAIN ] FLO: Full history saved. [2025-09-16T18:29:09.807223] [INFO ] [MAIN ] Update Progress: 843/2369 symbols completed. FLO took 26.0s. Estimated remaining: 2h 41m 19s. Rate: 9.5 symbols/min [2025-09-16T18:29:09.976430] [INFO ] [MAIN ] FLUT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:29:09.975119] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-02. [2025-09-16T18:29:10.016676] [DEBUG] [LOCAL ] FLUT: Loaded 37330 existing records from primary storage. [2025-09-16T18:29:10.016846] [INFO ] [MAIN ] FLUT: Loaded 37330 existing records. [2025-09-16T18:29:10.201666] [WARN ] [MAIN ] FLUT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:29:10.202163] [INFO ] [MAIN ] FLUT: Wiping existing primary data before full refresh. [2025-09-16T18:29:10.202221] [INFO ] [LOCAL ] FLUT: Starting deletion of primary data. [2025-09-16T18:29:10.214900] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-09-16T18:29:10.214941] [DEBUG] [LOCAL ] FLUT: Deleted primary data file. [2025-09-16T18:29:10.219427] [INFO ] [MAIN ] FLUT: Starting full primary refresh. [2025-09-16T18:29:10.286503] [INFO ] [REMOTE ] FLNG: Full fetch complete (37755 records). [2025-09-16T18:29:10.303708] [DEBUG] [VALIDATOR ] FLNG: Validating primary data. [2025-09-16T18:29:10.892912] [INFO ] [REMOTE ] FLUT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:29:10.893289] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-02. [2025-09-16T18:29:10.980180] [DEBUG] [VALIDATOR ] FLNG: All expected trading days are present. [2025-09-16T18:29:10.980386] [DEBUG] [VALIDATOR ] FLNG: Primary data validation passed. [2025-09-16T18:29:10.981558] [INFO ] [MAIN ] FLNG: Fetched primary data is valid. [2025-09-16T18:29:10.981803] [DEBUG] [LOCAL ] FLNG: Saving primary data. [2025-09-16T18:29:10.983408] [DEBUG] [LOCAL ] FLNG: Trimmed 37755 records to 25121 (keeping 150 days of data) [2025-09-16T18:29:11.115087] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-09-16T18:29:11.170214] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-09-16T18:29:11.225142] [DEBUG] [LOCAL ] FLNG: Primary data saved (25121 records). [2025-09-16T18:29:11.225283] [INFO ] [MAIN ] FLNG: Full history saved. [2025-09-16T18:29:11.225374] [INFO ] [MAIN ] Update Progress: 844/2369 symbols completed. FLNG took 27.5s. Estimated remaining: 2h 41m 3s. Rate: 9.5 symbols/min [2025-09-16T18:29:11.393347] [INFO ] [MAIN ] FLXR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:29:11.408730] [DEBUG] [LOCAL ] FLXR: Loaded 15177 existing records from primary storage. [2025-09-16T18:29:11.408859] [INFO ] [MAIN ] FLXR: Loaded 15177 existing records. [2025-09-16T18:29:11.581925] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-03. [2025-09-16T18:29:11.586257] [WARN ] [MAIN ] FLXR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:29:11.587192] [INFO ] [MAIN ] FLXR: Wiping existing primary data before full refresh. [2025-09-16T18:29:11.587257] [INFO ] [LOCAL ] FLXR: Starting deletion of primary data. [2025-09-16T18:29:11.587841] [DEBUG] [LOCAL ] FLXR: Deleted primary data file. [2025-09-16T18:29:11.588011] [INFO ] [MAIN ] FLXR: Starting full primary refresh. [2025-09-16T18:29:11.956350] [INFO ] [REMOTE ] FLXR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:29:12.374871] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-02. [2025-09-16T18:29:12.802436] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-09-16T18:29:12.959069] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-09-16T18:29:12.997496] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-09-16T18:29:13.174909] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-03. [2025-09-16T18:29:13.974920] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-05. [2025-09-16T18:29:14.774905] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-03. [2025-09-16T18:29:14.991487] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-09-16T18:29:15.574908] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-04. [2025-09-16T18:29:15.774658] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-09-16T18:29:15.913203] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-09-16T18:29:16.374907] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-06. [2025-09-16T18:29:17.174901] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-04. [2025-09-16T18:29:17.425791] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-09-16T18:29:17.974940] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-05. [2025-09-16T18:29:18.048597] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-09-16T18:29:18.226421] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-09-16T18:29:18.774981] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-05. [2025-09-16T18:29:19.574922] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-04. [2025-09-16T18:29:19.750709] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-09-16T18:29:19.772292] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T18:29:20.374893] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-06. [2025-09-16T18:29:21.066968] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T18:29:21.174876] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-06. [2025-09-16T18:29:21.974887] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-05. [2025-09-16T18:29:22.131514] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T18:29:22.153280] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T18:29:22.774901] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-07. [2025-09-16T18:29:23.409919] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T18:29:23.574892] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-07. [2025-09-16T18:29:24.374865] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-06. [2025-09-16T18:29:24.862922] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-09-16T18:29:25.174906] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-08. [2025-09-16T18:29:25.393659] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T18:29:25.974883] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-08. [2025-09-16T18:29:26.014115] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T18:29:26.774939] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-07. [2025-09-16T18:29:26.942944] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T18:29:27.489677] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T18:29:27.574920] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-09. [2025-09-16T18:29:28.374878] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-07. [2025-09-16T18:29:28.534827] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T18:29:29.053853] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T18:29:29.060223] [INFO ] [REMOTE ] FLR: Full fetch complete (57773 records). [2025-09-16T18:29:29.060463] [DEBUG] [VALIDATOR ] FLR: Validating primary data. [2025-09-16T18:29:29.177738] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-09. [2025-09-16T18:29:29.384177] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-09-16T18:29:29.472798] [DEBUG] [VALIDATOR ] FLR: All expected trading days are present. [2025-09-16T18:29:29.472878] [DEBUG] [VALIDATOR ] FLR: Primary data validation passed. [2025-09-16T18:29:29.473617] [INFO ] [MAIN ] FLR: Fetched primary data is valid. [2025-09-16T18:29:29.473678] [DEBUG] [LOCAL ] FLR: Saving primary data. [2025-09-16T18:29:29.474868] [DEBUG] [LOCAL ] FLR: Trimmed 57773 records to 39691 (keeping 150 days of data) [2025-09-16T18:29:29.708042] [DEBUG] [LOCAL ] FLR: Primary data saved (39691 records). [2025-09-16T18:29:29.708195] [INFO ] [MAIN ] FLR: Full history saved. [2025-09-16T18:29:29.708274] [INFO ] [MAIN ] Update Progress: 845/2369 symbols completed. FLR took 28.1s. Estimated remaining: 2h 41m 19s. Rate: 9.4 symbols/min [2025-09-16T18:29:29.875718] [INFO ] [MAIN ] FMC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:29:29.912853] [DEBUG] [LOCAL ] FMC: Loaded 38090 existing records from primary storage. [2025-09-16T18:29:29.912998] [INFO ] [MAIN ] FMC: Loaded 38090 existing records. [2025-09-16T18:29:29.980004] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-08. [2025-09-16T18:29:30.096637] [WARN ] [MAIN ] FMC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:29:30.096923] [INFO ] [MAIN ] FMC: Wiping existing primary data before full refresh. [2025-09-16T18:29:30.096978] [INFO ] [LOCAL ] FMC: Starting deletion of primary data. [2025-09-16T18:29:30.097799] [DEBUG] [LOCAL ] FMC: Deleted primary data file. [2025-09-16T18:29:30.097851] [INFO ] [MAIN ] FMC: Starting full primary refresh. [2025-09-16T18:29:30.445048] [INFO ] [REMOTE ] FMC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:29:30.774895] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-08. [2025-09-16T18:29:30.998975] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-09-16T18:29:31.005432] [INFO ] [REMOTE ] FLUT: Full fetch complete (55499 records). [2025-09-16T18:29:31.005556] [DEBUG] [VALIDATOR ] FLUT: Validating primary data. [2025-09-16T18:29:31.390755] [DEBUG] [VALIDATOR ] FLUT: All expected trading days are present. [2025-09-16T18:29:31.390846] [DEBUG] [VALIDATOR ] FLUT: Primary data validation passed. [2025-09-16T18:29:31.391621] [INFO ] [MAIN ] FLUT: Fetched primary data is valid. [2025-09-16T18:29:31.391688] [DEBUG] [LOCAL ] FLUT: Saving primary data. [2025-09-16T18:29:31.393544] [DEBUG] [LOCAL ] FLUT: Trimmed 55499 records to 37703 (keeping 150 days of data) [2025-09-16T18:29:31.594840] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-02. [2025-09-16T18:29:31.634981] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T18:29:31.647492] [DEBUG] [LOCAL ] FLUT: Primary data saved (37703 records). [2025-09-16T18:29:31.647616] [INFO ] [MAIN ] FLUT: Full history saved. [2025-09-16T18:29:31.647731] [INFO ] [MAIN ] Update Progress: 846/2369 symbols completed. FLUT took 21.8s. Estimated remaining: 2h 41m 5s. Rate: 9.5 symbols/min [2025-09-16T18:29:31.815985] [INFO ] [MAIN ] FMN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:29:31.821605] [DEBUG] [LOCAL ] FMN: Loaded 3809 existing records from primary storage. [2025-09-16T18:29:31.821720] [INFO ] [MAIN ] FMN: Loaded 3809 existing records. [2025-09-16T18:29:31.998261] [WARN ] [MAIN ] FMN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:29:31.998602] [INFO ] [MAIN ] FMN: Wiping existing primary data before full refresh. [2025-09-16T18:29:31.998672] [INFO ] [LOCAL ] FMN: Starting deletion of primary data. [2025-09-16T18:29:31.998912] [DEBUG] [LOCAL ] FMN: Deleted primary data file. [2025-09-16T18:29:31.998998] [INFO ] [MAIN ] FMN: Starting full primary refresh. [2025-09-16T18:29:32.334390] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-08 (2.35 seconds). [2025-09-16T18:29:32.378488] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-09. [2025-09-16T18:29:32.400487] [INFO ] [REMOTE ] FMN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:29:33.174921] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-02. [2025-09-16T18:29:33.445838] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-09-16T18:29:33.482620] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-09-16T18:29:33.486233] [INFO ] [REMOTE ] FLXR: Full fetch complete (20562 records). [2025-09-16T18:29:33.486329] [DEBUG] [VALIDATOR ] FLXR: Validating primary data. [2025-09-16T18:29:33.867929] [DEBUG] [VALIDATOR ] FLXR: All expected trading days are present. [2025-09-16T18:29:33.868034] [DEBUG] [VALIDATOR ] FLXR: Primary data validation passed. [2025-09-16T18:29:33.868566] [INFO ] [MAIN ] FLXR: Fetched primary data is valid. [2025-09-16T18:29:33.868627] [DEBUG] [LOCAL ] FLXR: Saving primary data. [2025-09-16T18:29:33.869356] [DEBUG] [LOCAL ] FLXR: Trimmed 20562 records to 15343 (keeping 150 days of data) [2025-09-16T18:29:33.971935] [DEBUG] [LOCAL ] FLXR: Primary data saved (15343 records). [2025-09-16T18:29:33.972090] [INFO ] [MAIN ] FLXR: Full history saved. [2025-09-16T18:29:33.972210] [INFO ] [MAIN ] Update Progress: 847/2369 symbols completed. FLXR took 22.7s. Estimated remaining: 2h 40m 51s. Rate: 9.5 symbols/min [2025-09-16T18:29:33.984284] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-09. [2025-09-16T18:29:34.084818] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-09-16T18:29:34.158262] [INFO ] [MAIN ] FMS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:29:34.180001] [DEBUG] [LOCAL ] FMS: Loaded 22098 existing records from primary storage. [2025-09-16T18:29:34.180134] [INFO ] [MAIN ] FMS: Loaded 22098 existing records. [2025-09-16T18:29:34.354329] [WARN ] [MAIN ] FMS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:29:34.354590] [INFO ] [MAIN ] FMS: Wiping existing primary data before full refresh. [2025-09-16T18:29:34.354636] [INFO ] [LOCAL ] FMS: Starting deletion of primary data. [2025-09-16T18:29:34.355221] [DEBUG] [LOCAL ] FMS: Deleted primary data file. [2025-09-16T18:29:34.355279] [INFO ] [MAIN ] FMS: Starting full primary refresh. [2025-09-16T18:29:34.746661] [INFO ] [REMOTE ] FMS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:29:34.774843] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-02. [2025-09-16T18:29:35.574936] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-03. [2025-09-16T18:29:35.822073] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-09-16T18:29:35.828796] [INFO ] [REMOTE ] FLS: Full fetch complete (54352 records). [2025-09-16T18:29:35.828891] [DEBUG] [VALIDATOR ] FLS: Validating primary data. [2025-09-16T18:29:36.224546] [DEBUG] [VALIDATOR ] FLS: All expected trading days are present. [2025-09-16T18:29:36.229478] [DEBUG] [VALIDATOR ] FLS: Primary data validation passed. [2025-09-16T18:29:36.230732] [INFO ] [MAIN ] FLS: Fetched primary data is valid. [2025-09-16T18:29:36.231195] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-09-16T18:29:36.236052] [DEBUG] [LOCAL ] FLS: Saving primary data. [2025-09-16T18:29:36.238122] [DEBUG] [LOCAL ] FLS: Trimmed 54352 records to 37213 (keeping 150 days of data) [2025-09-16T18:29:36.413787] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-03. [2025-09-16T18:29:36.473975] [DEBUG] [LOCAL ] FLS: Primary data saved (37213 records). [2025-09-16T18:29:36.474102] [INFO ] [MAIN ] FLS: Full history saved. [2025-09-16T18:29:36.474204] [INFO ] [MAIN ] Update Progress: 848/2369 symbols completed. FLS took 27.7s. Estimated remaining: 2h 40m 38s. Rate: 9.5 symbols/min [2025-09-16T18:29:36.647808] [INFO ] [MAIN ] FMX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:29:36.648014] [DEBUG] [LOCAL ] FMX: No existing data file found in primary storage. [2025-09-16T18:29:36.648054] [WARN ] [MAIN ] FMX: No existing data found, performing full refresh instead. [2025-09-16T18:29:36.648173] [INFO ] [MAIN ] FMX: Wiping existing primary data before full refresh. [2025-09-16T18:29:36.648210] [INFO ] [LOCAL ] FMX: Starting deletion of primary data. [2025-09-16T18:29:36.648250] [DEBUG] [LOCAL ] FMX: No primary data file found to delete. [2025-09-16T18:29:36.648283] [INFO ] [MAIN ] FMX: Starting full primary refresh. [2025-09-16T18:29:37.006576] [INFO ] [REMOTE ] FMX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:29:37.174891] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-02. [2025-09-16T18:29:37.524132] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-09-16T18:29:37.552550] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T18:29:37.974891] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-03. [2025-09-16T18:29:38.738363] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-09-16T18:29:38.779953] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-04. [2025-09-16T18:29:38.792206] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T18:29:39.307165] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-09-16T18:29:39.574934] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-04. [2025-09-16T18:29:40.374938] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-03. [2025-09-16T18:29:41.174958] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-04. [2025-09-16T18:29:41.359967] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T18:29:41.778189] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T18:29:41.974899] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-04. [2025-09-16T18:29:42.493444] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T18:29:42.774916] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-05. [2025-09-16T18:29:43.055224] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-09-16T18:29:43.324462] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-09-16T18:29:43.574915] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-05. [2025-09-16T18:29:44.374961] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-06. [2025-09-16T18:29:45.068412] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-09-16T18:29:45.174980] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-07. [2025-09-16T18:29:45.533766] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-09-16T18:29:45.791616] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-09-16T18:29:45.974939] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-06. [2025-09-16T18:29:46.774968] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-05. [2025-09-16T18:29:47.574958] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-08. [2025-09-16T18:29:47.706492] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-09-16T18:29:48.020040] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-09-16T18:29:48.374901] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-05. [2025-09-16T18:29:48.437988] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T18:29:49.174912] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-09. [2025-09-16T18:29:49.974762] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T18:29:49.975084] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-07. [2025-09-16T18:29:49.976164] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-09-16T18:29:49.978226] [INFO ] [REMOTE ] FMN: Full fetch complete (6787 records). [2025-09-16T18:29:49.985965] [DEBUG] [VALIDATOR ] FMN: Validating primary data. [2025-09-16T18:29:50.347763] [DEBUG] [VALIDATOR ] FMN: All expected trading days are present. [2025-09-16T18:29:50.347842] [DEBUG] [VALIDATOR ] FMN: Primary data validation passed. [2025-09-16T18:29:50.348027] [INFO ] [MAIN ] FMN: Fetched primary data is valid. [2025-09-16T18:29:50.348065] [DEBUG] [LOCAL ] FMN: Saving primary data. [2025-09-16T18:29:50.348445] [DEBUG] [LOCAL ] FMN: Trimmed 6787 records to 3836 (keeping 150 days of data) [2025-09-16T18:29:50.372320] [DEBUG] [LOCAL ] FMN: Primary data saved (3836 records). [2025-09-16T18:29:50.372408] [INFO ] [MAIN ] FMN: Full history saved. [2025-09-16T18:29:50.372484] [INFO ] [MAIN ] Update Progress: 849/2369 symbols completed. FMN took 18.7s. Estimated remaining: 2h 40m 45s. Rate: 9.5 symbols/min [2025-09-16T18:29:50.552635] [INFO ] [MAIN ] FMY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:29:50.555599] [DEBUG] [LOCAL ] FMY: Loaded 1018 existing records from primary storage. [2025-09-16T18:29:50.555699] [INFO ] [MAIN ] FMY: Loaded 1018 existing records. [2025-09-16T18:29:50.730432] [WARN ] [MAIN ] FMY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:29:50.730755] [INFO ] [MAIN ] FMY: Wiping existing primary data before full refresh. [2025-09-16T18:29:50.730829] [INFO ] [LOCAL ] FMY: Starting deletion of primary data. [2025-09-16T18:29:50.731048] [DEBUG] [LOCAL ] FMY: Deleted primary data file. [2025-09-16T18:29:50.731102] [INFO ] [MAIN ] FMY: Starting full primary refresh. [2025-09-16T18:29:50.786288] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-06. [2025-09-16T18:29:51.097798] [INFO ] [REMOTE ] FMY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:29:51.574948] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-02. [2025-09-16T18:29:51.769260] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-09-16T18:29:51.861305] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T18:29:51.979616] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-09-16T18:29:52.374897] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-07. [2025-09-16T18:29:53.174913] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-03. [2025-09-16T18:29:53.528088] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-09-16T18:29:53.614347] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-09-16T18:29:53.974937] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-06. [2025-09-16T18:29:54.774913] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-08. [2025-09-16T18:29:55.470546] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T18:29:55.574956] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-07. [2025-09-16T18:29:56.019992] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-09-16T18:29:56.374927] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-04. [2025-09-16T18:29:56.974032] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-09-16T18:29:57.020097] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T18:29:57.174924] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-08. [2025-09-16T18:29:57.974929] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-09. [2025-09-16T18:29:58.774914] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-08. [2025-09-16T18:29:59.104982] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T18:29:59.577267] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-09. [2025-09-16T18:29:59.579135] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T18:29:59.583689] [INFO ] [REMOTE ] FMS: Full fetch complete (32959 records). [2025-09-16T18:29:59.583791] [DEBUG] [VALIDATOR ] FMS: Validating primary data. [2025-09-16T18:29:59.960378] [DEBUG] [VALIDATOR ] FMS: All expected trading days are present. [2025-09-16T18:29:59.960468] [DEBUG] [VALIDATOR ] FMS: Primary data validation passed. [2025-09-16T18:29:59.961138] [INFO ] [MAIN ] FMS: Fetched primary data is valid. [2025-09-16T18:29:59.961193] [DEBUG] [LOCAL ] FMS: Saving primary data. [2025-09-16T18:29:59.962189] [DEBUG] [LOCAL ] FMS: Trimmed 32959 records to 22350 (keeping 150 days of data) [2025-09-16T18:30:00.127243] [DEBUG] [LOCAL ] FMS: Primary data saved (22350 records). [2025-09-16T18:30:00.127378] [INFO ] [MAIN ] FMS: Full history saved. [2025-09-16T18:30:00.127481] [INFO ] [MAIN ] Update Progress: 850/2369 symbols completed. FMS took 26.2s. Estimated remaining: 2h 40m 45s. Rate: 9.4 symbols/min [2025-09-16T18:30:00.297473] [INFO ] [MAIN ] FN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:00.297731] [DEBUG] [LOCAL ] FN: No existing data file found in primary storage. [2025-09-16T18:30:00.297772] [WARN ] [MAIN ] FN: No existing data found, performing full refresh instead. [2025-09-16T18:30:00.297903] [INFO ] [MAIN ] FN: Wiping existing primary data before full refresh. [2025-09-16T18:30:00.297981] [INFO ] [LOCAL ] FN: Starting deletion of primary data. [2025-09-16T18:30:00.298044] [DEBUG] [LOCAL ] FN: No primary data file found to delete. [2025-09-16T18:30:00.298091] [INFO ] [MAIN ] FN: Starting full primary refresh. [2025-09-16T18:30:00.377724] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-05. [2025-09-16T18:30:00.659355] [INFO ] [REMOTE ] FN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:00.686240] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-09-16T18:30:00.895674] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-09-16T18:30:01.174892] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-09. [2025-09-16T18:30:01.209908] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-09-16T18:30:01.215968] [INFO ] [REMOTE ] FMC: Full fetch complete (56113 records). [2025-09-16T18:30:01.216081] [DEBUG] [VALIDATOR ] FMC: Validating primary data. [2025-09-16T18:30:01.216779] [FATAL] [VALIDATOR ] FMC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:30:01.216858] [FATAL] [MAIN ] FMC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:30:01.216904] [WARN ] [MAIN ] FMC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:30:01.216965] [INFO ] [LOCAL ] FMC: Starting deletion of primary data. [2025-09-16T18:30:01.217056] [DEBUG] [LOCAL ] FMC: No primary data file found to delete. [2025-09-16T18:30:01.217115] [INFO ] [LOCAL ] FMC: Starting deletion of derivative data. [2025-09-16T18:30:01.217513] [DEBUG] [LOCAL ] FMC: Deleted derivative data file. [2025-09-16T18:30:01.217674] [INFO ] [MAIN ] Update Progress: 851/2369 symbols completed. FMC took 31.5s. Estimated remaining: 2h 40m 29s. Rate: 9.5 symbols/min [2025-09-16T18:30:01.395118] [INFO ] [MAIN ] FNB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:01.445086] [DEBUG] [LOCAL ] FNB: Loaded 38602 existing records from primary storage. [2025-09-16T18:30:01.445249] [INFO ] [MAIN ] FNB: Loaded 38602 existing records. [2025-09-16T18:30:01.640444] [WARN ] [MAIN ] FNB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:01.640712] [INFO ] [MAIN ] FNB: Wiping existing primary data before full refresh. [2025-09-16T18:30:01.640762] [INFO ] [LOCAL ] FNB: Starting deletion of primary data. [2025-09-16T18:30:01.641799] [DEBUG] [LOCAL ] FNB: Deleted primary data file. [2025-09-16T18:30:01.641858] [INFO ] [MAIN ] FNB: Starting full primary refresh. [2025-09-16T18:30:01.983161] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-06. [2025-09-16T18:30:02.023750] [INFO ] [REMOTE ] FNB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:02.490972] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-09-16T18:30:02.774922] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-02. [2025-09-16T18:30:02.789675] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T18:30:02.794614] [INFO ] [REMOTE ] FMX: Full fetch complete (43684 records). [2025-09-16T18:30:02.794714] [DEBUG] [VALIDATOR ] FMX: Validating primary data. [2025-09-16T18:30:02.795349] [FATAL] [VALIDATOR ] FMX: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:30:02.795447] [FATAL] [MAIN ] FMX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:30:02.795486] [WARN ] [MAIN ] FMX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:30:02.795546] [INFO ] [LOCAL ] FMX: Starting deletion of primary data. [2025-09-16T18:30:02.795613] [DEBUG] [LOCAL ] FMX: No primary data file found to delete. [2025-09-16T18:30:02.795680] [INFO ] [LOCAL ] FMX: Starting deletion of derivative data. [2025-09-16T18:30:02.795735] [DEBUG] [LOCAL ] FMX: No derivative data file found to delete. [2025-09-16T18:30:02.795794] [INFO ] [MAIN ] Update Progress: 852/2369 symbols completed. FMX took 26.3s. Estimated remaining: 2h 40m 14s. Rate: 9.5 symbols/min [2025-09-16T18:30:02.972345] [INFO ] [MAIN ] FND: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:03.010245] [DEBUG] [LOCAL ] FND: Loaded 38434 existing records from primary storage. [2025-09-16T18:30:03.010390] [INFO ] [MAIN ] FND: Loaded 38434 existing records. [2025-09-16T18:30:03.190027] [WARN ] [MAIN ] FND: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:03.190283] [INFO ] [MAIN ] FND: Wiping existing primary data before full refresh. [2025-09-16T18:30:03.190330] [INFO ] [LOCAL ] FND: Starting deletion of primary data. [2025-09-16T18:30:03.191236] [DEBUG] [LOCAL ] FND: Deleted primary data file. [2025-09-16T18:30:03.191293] [INFO ] [MAIN ] FND: Starting full primary refresh. [2025-09-16T18:30:03.548154] [INFO ] [REMOTE ] FND: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:03.574898] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-02. [2025-09-16T18:30:04.182078] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-09-16T18:30:04.374923] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-02. [2025-09-16T18:30:05.092397] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T18:30:05.174947] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-07. [2025-09-16T18:30:05.496533] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-09-16T18:30:05.974915] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-03. [2025-09-16T18:30:06.208240] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T18:30:06.774941] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-08. [2025-09-16T18:30:07.243496] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-09-16T18:30:07.507765] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T18:30:07.574929] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-03. [2025-09-16T18:30:08.374922] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-09. [2025-09-16T18:30:08.825202] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-09-16T18:30:08.826057] [INFO ] [REMOTE ] FMY: Full fetch complete (1511 records). [2025-09-16T18:30:08.826128] [DEBUG] [VALIDATOR ] FMY: Validating primary data. [2025-09-16T18:30:09.183511] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-03. [2025-09-16T18:30:09.189378] [DEBUG] [VALIDATOR ] FMY: All expected trading days are present. [2025-09-16T18:30:09.189515] [DEBUG] [VALIDATOR ] FMY: Primary data validation passed. [2025-09-16T18:30:09.189676] [INFO ] [MAIN ] FMY: Fetched primary data is valid. [2025-09-16T18:30:09.189750] [DEBUG] [LOCAL ] FMY: Saving primary data. [2025-09-16T18:30:09.190129] [DEBUG] [LOCAL ] FMY: Trimmed 1511 records to 1027 (keeping 150 days of data) [2025-09-16T18:30:09.197171] [DEBUG] [LOCAL ] FMY: Primary data saved (1027 records). [2025-09-16T18:30:09.197243] [INFO ] [MAIN ] FMY: Full history saved. [2025-09-16T18:30:09.197316] [INFO ] [MAIN ] Update Progress: 853/2369 symbols completed. FMY took 18.8s. Estimated remaining: 2h 40m 8s. Rate: 9.5 symbols/min [2025-09-16T18:30:09.373477] [INFO ] [MAIN ] FNF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:09.403062] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T18:30:09.449151] [DEBUG] [LOCAL ] FNF: Loaded 36151 existing records from primary storage. [2025-09-16T18:30:09.449294] [INFO ] [MAIN ] FNF: Loaded 36151 existing records. [2025-09-16T18:30:09.622814] [WARN ] [MAIN ] FNF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:09.623025] [INFO ] [MAIN ] FNF: Wiping existing primary data before full refresh. [2025-09-16T18:30:09.623069] [INFO ] [LOCAL ] FNF: Starting deletion of primary data. [2025-09-16T18:30:09.623867] [DEBUG] [LOCAL ] FNF: Deleted primary data file. [2025-09-16T18:30:09.623915] [INFO ] [MAIN ] FNF: Starting full primary refresh. [2025-09-16T18:30:09.970450] [INFO ] [REMOTE ] FNF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:09.974853] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-02. [2025-09-16T18:30:10.774909] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-04. [2025-09-16T18:30:11.134947] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-09-16T18:30:11.574907] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-04. [2025-09-16T18:30:11.794139] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-09-16T18:30:12.374881] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-03. [2025-09-16T18:30:12.448269] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T18:30:13.174987] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-05. [2025-09-16T18:30:13.235373] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T18:30:13.974976] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-05. [2025-09-16T18:30:14.205287] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-09-16T18:30:14.653574] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T18:30:14.774919] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-06. [2025-09-16T18:30:15.402891] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-09-16T18:30:15.574901] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-06. [2025-09-16T18:30:16.374900] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-04. [2025-09-16T18:30:16.928023] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T18:30:17.174941] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-04. [2025-09-16T18:30:17.974951] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-07. [2025-09-16T18:30:18.013474] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T18:30:18.095707] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-06 (3.32 seconds). [2025-09-16T18:30:18.329558] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T18:30:18.774917] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-05. [2025-09-16T18:30:19.574915] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-05. [2025-09-16T18:30:19.628745] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T18:30:19.779527] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T18:30:20.374947] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-08. [2025-09-16T18:30:21.111816] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T18:30:21.174927] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-06. [2025-09-16T18:30:21.866214] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T18:30:21.974948] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-07. [2025-09-16T18:30:22.567575] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T18:30:22.774923] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-06. [2025-09-16T18:30:23.525709] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-09-16T18:30:23.574850] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-08. [2025-09-16T18:30:24.374906] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-09. [2025-09-16T18:30:25.049833] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-06 (2.27 seconds). [2025-09-16T18:30:25.174962] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-07. [2025-09-16T18:30:25.240011] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:30:25.979974] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-09. [2025-09-16T18:30:25.994275] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T18:30:26.001531] [INFO ] [REMOTE ] FNB: Full fetch complete (56937 records). [2025-09-16T18:30:26.001739] [DEBUG] [VALIDATOR ] FNB: Validating primary data. [2025-09-16T18:30:26.385135] [DEBUG] [VALIDATOR ] FNB: All expected trading days are present. [2025-09-16T18:30:26.385226] [DEBUG] [VALIDATOR ] FNB: Primary data validation passed. [2025-09-16T18:30:26.386000] [INFO ] [MAIN ] FNB: Fetched primary data is valid. [2025-09-16T18:30:26.386048] [DEBUG] [LOCAL ] FNB: Saving primary data. [2025-09-16T18:30:26.387329] [DEBUG] [LOCAL ] FNB: Trimmed 56937 records to 38986 (keeping 150 days of data) [2025-09-16T18:30:26.633410] [DEBUG] [LOCAL ] FNB: Primary data saved (38986 records). [2025-09-16T18:30:26.633570] [INFO ] [MAIN ] FNB: Full history saved. [2025-09-16T18:30:26.633677] [INFO ] [MAIN ] Update Progress: 854/2369 symbols completed. FNB took 25.4s. Estimated remaining: 2h 40m 21s. Rate: 9.4 symbols/min [2025-09-16T18:30:26.784162] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-07. [2025-09-16T18:30:26.814732] [INFO ] [MAIN ] FNV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:26.849661] [DEBUG] [LOCAL ] FNV: Loaded 33565 existing records from primary storage. [2025-09-16T18:30:26.849825] [INFO ] [MAIN ] FNV: Loaded 33565 existing records. [2025-09-16T18:30:27.041478] [WARN ] [MAIN ] FNV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:27.042020] [INFO ] [MAIN ] FNV: Wiping existing primary data before full refresh. [2025-09-16T18:30:27.042119] [INFO ] [LOCAL ] FNV: Starting deletion of primary data. [2025-09-16T18:30:27.043036] [DEBUG] [LOCAL ] FNV: Deleted primary data file. [2025-09-16T18:30:27.043173] [INFO ] [MAIN ] FNV: Starting full primary refresh. [2025-09-16T18:30:27.208456] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-09-16T18:30:27.456129] [INFO ] [REMOTE ] FNV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:27.574889] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-02. [2025-09-16T18:30:27.718094] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-09-16T18:30:27.724671] [INFO ] [REMOTE ] FND: Full fetch complete (56675 records). [2025-09-16T18:30:27.724800] [DEBUG] [VALIDATOR ] FND: Validating primary data. [2025-09-16T18:30:28.123290] [DEBUG] [VALIDATOR ] FND: All expected trading days are present. [2025-09-16T18:30:28.123440] [DEBUG] [VALIDATOR ] FND: Primary data validation passed. [2025-09-16T18:30:28.124431] [INFO ] [MAIN ] FND: Fetched primary data is valid. [2025-09-16T18:30:28.124594] [DEBUG] [LOCAL ] FND: Saving primary data. [2025-09-16T18:30:28.125986] [DEBUG] [LOCAL ] FND: Trimmed 56675 records to 38807 (keeping 150 days of data) [2025-09-16T18:30:28.245660] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T18:30:28.380135] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-08. [2025-09-16T18:30:28.403738] [DEBUG] [LOCAL ] FND: Primary data saved (38807 records). [2025-09-16T18:30:28.404293] [INFO ] [MAIN ] FND: Full history saved. [2025-09-16T18:30:28.404426] [INFO ] [MAIN ] Update Progress: 855/2369 symbols completed. FND took 25.6s. Estimated remaining: 2h 40m 7s. Rate: 9.5 symbols/min [2025-09-16T18:30:28.575047] [INFO ] [MAIN ] FOA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:28.585180] [DEBUG] [LOCAL ] FOA: Loaded 9145 existing records from primary storage. [2025-09-16T18:30:28.585299] [INFO ] [MAIN ] FOA: Loaded 9145 existing records. [2025-09-16T18:30:28.764236] [WARN ] [MAIN ] FOA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:28.764707] [INFO ] [MAIN ] FOA: Wiping existing primary data before full refresh. [2025-09-16T18:30:28.764776] [INFO ] [LOCAL ] FOA: Starting deletion of primary data. [2025-09-16T18:30:28.765227] [DEBUG] [LOCAL ] FOA: Deleted primary data file. [2025-09-16T18:30:28.765279] [INFO ] [MAIN ] FOA: Starting full primary refresh. [2025-09-16T18:30:29.083480] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-09-16T18:30:29.159617] [INFO ] [REMOTE ] FOA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:29.174851] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-02. [2025-09-16T18:30:29.974945] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-03. [2025-09-16T18:30:30.070099] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-09-16T18:30:30.100116] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-09-16T18:30:30.774909] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-09. [2025-09-16T18:30:31.574879] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-08. [2025-09-16T18:30:31.664153] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-09-16T18:30:32.374924] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-03. [2025-09-16T18:30:32.474930] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T18:30:32.480436] [INFO ] [REMOTE ] FN: Full fetch complete (43431 records). [2025-09-16T18:30:32.480632] [DEBUG] [VALIDATOR ] FN: Validating primary data. [2025-09-16T18:30:32.926623] [DEBUG] [VALIDATOR ] FN: All expected trading days are present. [2025-09-16T18:30:32.926760] [DEBUG] [VALIDATOR ] FN: Primary data validation passed. [2025-09-16T18:30:32.927563] [INFO ] [MAIN ] FN: Fetched primary data is valid. [2025-09-16T18:30:32.927620] [DEBUG] [LOCAL ] FN: Saving primary data. [2025-09-16T18:30:32.929106] [DEBUG] [LOCAL ] FN: Trimmed 43431 records to 28485 (keeping 150 days of data) [2025-09-16T18:30:33.063777] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T18:30:33.091742] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-09-16T18:30:33.178909] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-09. [2025-09-16T18:30:33.191060] [DEBUG] [LOCAL ] FN: Primary data saved (28485 records). [2025-09-16T18:30:33.191228] [INFO ] [MAIN ] FN: Full history saved. [2025-09-16T18:30:33.191334] [INFO ] [MAIN ] Update Progress: 856/2369 symbols completed. FN took 33.1s. Estimated remaining: 2h 39m 58s. Rate: 9.5 symbols/min [2025-09-16T18:30:33.376859] [INFO ] [MAIN ] FOF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:33.383870] [DEBUG] [LOCAL ] FOF: Loaded 5732 existing records from primary storage. [2025-09-16T18:30:33.383997] [INFO ] [MAIN ] FOF: Loaded 5732 existing records. [2025-09-16T18:30:33.559275] [WARN ] [MAIN ] FOF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:33.559522] [INFO ] [MAIN ] FOF: Wiping existing primary data before full refresh. [2025-09-16T18:30:33.559567] [INFO ] [LOCAL ] FOF: Starting deletion of primary data. [2025-09-16T18:30:33.559900] [DEBUG] [LOCAL ] FOF: Deleted primary data file. [2025-09-16T18:30:33.559948] [INFO ] [MAIN ] FOF: Starting full primary refresh. [2025-09-16T18:30:33.915596] [INFO ] [REMOTE ] FOF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:33.974884] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-02. [2025-09-16T18:30:34.683808] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:30:34.689837] [INFO ] [REMOTE ] FNF: Full fetch complete (53558 records). [2025-09-16T18:30:34.689993] [DEBUG] [VALIDATOR ] FNF: Validating primary data. [2025-09-16T18:30:34.782831] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-04. [2025-09-16T18:30:35.023564] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-09-16T18:30:35.087456] [DEBUG] [VALIDATOR ] FNF: All expected trading days are present. [2025-09-16T18:30:35.087537] [DEBUG] [VALIDATOR ] FNF: Primary data validation passed. [2025-09-16T18:30:35.088331] [INFO ] [MAIN ] FNF: Fetched primary data is valid. [2025-09-16T18:30:35.088391] [DEBUG] [LOCAL ] FNF: Saving primary data. [2025-09-16T18:30:35.089514] [DEBUG] [LOCAL ] FNF: Trimmed 53558 records to 36510 (keeping 150 days of data) [2025-09-16T18:30:35.314144] [DEBUG] [LOCAL ] FNF: Primary data saved (36510 records). [2025-09-16T18:30:35.314433] [INFO ] [MAIN ] FNF: Full history saved. [2025-09-16T18:30:35.314528] [INFO ] [MAIN ] Update Progress: 857/2369 symbols completed. FNF took 26.1s. Estimated remaining: 2h 39m 44s. Rate: 9.5 symbols/min [2025-09-16T18:30:35.492017] [INFO ] [MAIN ] FOR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:35.507162] [DEBUG] [LOCAL ] FOR: Loaded 14480 existing records from primary storage. [2025-09-16T18:30:35.507291] [INFO ] [MAIN ] FOR: Loaded 14480 existing records. [2025-09-16T18:30:35.583932] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-04. [2025-09-16T18:30:35.689162] [WARN ] [MAIN ] FOR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:35.689442] [INFO ] [MAIN ] FOR: Wiping existing primary data before full refresh. [2025-09-16T18:30:35.689488] [INFO ] [LOCAL ] FOR: Starting deletion of primary data. [2025-09-16T18:30:35.690128] [DEBUG] [LOCAL ] FOR: Deleted primary data file. [2025-09-16T18:30:35.690179] [INFO ] [MAIN ] FOR: Starting full primary refresh. [2025-09-16T18:30:36.047121] [INFO ] [REMOTE ] FOR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:36.374920] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-02. [2025-09-16T18:30:36.671465] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T18:30:36.782360] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-09-16T18:30:37.174919] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-05. [2025-09-16T18:30:37.220363] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-09-16T18:30:37.974911] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-03. [2025-09-16T18:30:38.774944] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-05. [2025-09-16T18:30:38.966350] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T18:30:38.988939] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-09-16T18:30:39.358087] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-09-16T18:30:39.574903] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-04. [2025-09-16T18:30:40.374901] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-06. [2025-09-16T18:30:40.601381] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T18:30:41.174906] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-03. [2025-09-16T18:30:41.272332] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-09-16T18:30:41.888187] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-09-16T18:30:41.974915] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-06. [2025-09-16T18:30:42.774925] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-05. [2025-09-16T18:30:43.574940] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-04. [2025-09-16T18:30:43.643092] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-09-16T18:30:43.677039] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T18:30:44.119023] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-09-16T18:30:44.374944] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-05. [2025-09-16T18:30:45.174901] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-07. [2025-09-16T18:30:45.252052] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-09-16T18:30:45.974931] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-06. [2025-09-16T18:30:46.065465] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-09-16T18:30:46.560266] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T18:30:46.774920] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-08. [2025-09-16T18:30:47.325983] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-09-16T18:30:47.574948] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-07. [2025-09-16T18:30:48.130971] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-09-16T18:30:48.374951] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-08. [2025-09-16T18:30:49.174925] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-06. [2025-09-16T18:30:49.235387] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T18:30:49.974919] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-09. [2025-09-16T18:30:50.428867] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T18:30:50.774930] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-09. [2025-09-16T18:30:50.958884] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T18:30:50.961198] [INFO ] [REMOTE ] FOA: Full fetch complete (14596 records). [2025-09-16T18:30:50.961363] [DEBUG] [VALIDATOR ] FOA: Validating primary data. [2025-09-16T18:30:50.961970] [FATAL] [VALIDATOR ] FOA: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:30:50.962046] [FATAL] [MAIN ] FOA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:30:50.962082] [WARN ] [MAIN ] FOA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:30:50.962119] [INFO ] [LOCAL ] FOA: Starting deletion of primary data. [2025-09-16T18:30:50.962188] [DEBUG] [LOCAL ] FOA: No primary data file found to delete. [2025-09-16T18:30:50.962247] [INFO ] [LOCAL ] FOA: Starting deletion of derivative data. [2025-09-16T18:30:50.962559] [DEBUG] [LOCAL ] FOA: Deleted derivative data file. [2025-09-16T18:30:50.962636] [INFO ] [MAIN ] Update Progress: 858/2369 symbols completed. FOA took 22.6s. Estimated remaining: 2h 39m 54s. Rate: 9.4 symbols/min [2025-09-16T18:30:51.135497] [INFO ] [MAIN ] FOUR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:51.168560] [DEBUG] [LOCAL ] FOUR: Loaded 36302 existing records from primary storage. [2025-09-16T18:30:51.168723] [INFO ] [MAIN ] FOUR: Loaded 36302 existing records. [2025-09-16T18:30:51.351200] [WARN ] [MAIN ] FOUR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:51.351444] [INFO ] [MAIN ] FOUR: Wiping existing primary data before full refresh. [2025-09-16T18:30:51.351492] [INFO ] [LOCAL ] FOUR: Starting deletion of primary data. [2025-09-16T18:30:51.352243] [DEBUG] [LOCAL ] FOUR: Deleted primary data file. [2025-09-16T18:30:51.352292] [INFO ] [MAIN ] FOUR: Starting full primary refresh. [2025-09-16T18:30:51.577282] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-07. [2025-09-16T18:30:51.660368] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T18:30:51.665543] [INFO ] [REMOTE ] FOF: Full fetch complete (8703 records). [2025-09-16T18:30:51.671149] [DEBUG] [VALIDATOR ] FOF: Validating primary data. [2025-09-16T18:30:51.832142] [INFO ] [REMOTE ] FOUR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:52.126375] [DEBUG] [VALIDATOR ] FOF: All expected trading days are present. [2025-09-16T18:30:52.126455] [DEBUG] [VALIDATOR ] FOF: Primary data validation passed. [2025-09-16T18:30:52.126709] [INFO ] [MAIN ] FOF: Fetched primary data is valid. [2025-09-16T18:30:52.126753] [DEBUG] [LOCAL ] FOF: Saving primary data. [2025-09-16T18:30:52.127182] [DEBUG] [LOCAL ] FOF: Trimmed 8703 records to 5797 (keeping 150 days of data) [2025-09-16T18:30:52.163990] [DEBUG] [LOCAL ] FOF: Primary data saved (5797 records). [2025-09-16T18:30:52.164133] [INFO ] [MAIN ] FOF: Full history saved. [2025-09-16T18:30:52.164221] [INFO ] [MAIN ] Update Progress: 859/2369 symbols completed. FOF took 19.0s. Estimated remaining: 2h 39m 39s. Rate: 9.5 symbols/min [2025-09-16T18:30:52.352505] [INFO ] [MAIN ] FPF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:52.367364] [DEBUG] [LOCAL ] FPF: Loaded 12774 existing records from primary storage. [2025-09-16T18:30:52.367486] [INFO ] [MAIN ] FPF: Loaded 12774 existing records. [2025-09-16T18:30:52.380109] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-07. [2025-09-16T18:30:52.550102] [WARN ] [MAIN ] FPF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:52.550462] [INFO ] [MAIN ] FPF: Wiping existing primary data before full refresh. [2025-09-16T18:30:52.550517] [INFO ] [LOCAL ] FPF: Starting deletion of primary data. [2025-09-16T18:30:52.556230] [DEBUG] [LOCAL ] FPF: Deleted primary data file. [2025-09-16T18:30:52.562984] [INFO ] [MAIN ] FPF: Starting full primary refresh. [2025-09-16T18:30:52.569401] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-09-16T18:30:52.926402] [INFO ] [REMOTE ] FPF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:53.174909] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-08. [2025-09-16T18:30:53.974964] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-02. [2025-09-16T18:30:54.162749] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T18:30:54.224758] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-09-16T18:30:54.774899] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-08. [2025-09-16T18:30:54.908852] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-09-16T18:30:55.574905] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-03. [2025-09-16T18:30:56.374962] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-09. [2025-09-16T18:30:56.537345] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-09-16T18:30:56.664158] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-09-16T18:30:57.174960] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-02. [2025-09-16T18:30:57.345604] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T18:30:57.349326] [INFO ] [REMOTE ] FOR: Full fetch complete (22350 records). [2025-09-16T18:30:57.349464] [DEBUG] [VALIDATOR ] FOR: Validating primary data. [2025-09-16T18:30:57.754792] [DEBUG] [VALIDATOR ] FOR: All expected trading days are present. [2025-09-16T18:30:57.754899] [DEBUG] [VALIDATOR ] FOR: Primary data validation passed. [2025-09-16T18:30:57.755411] [INFO ] [MAIN ] FOR: Fetched primary data is valid. [2025-09-16T18:30:57.755466] [DEBUG] [LOCAL ] FOR: Saving primary data. [2025-09-16T18:30:57.756369] [DEBUG] [LOCAL ] FOR: Trimmed 22350 records to 14590 (keeping 150 days of data) [2025-09-16T18:30:57.853875] [DEBUG] [LOCAL ] FOR: Primary data saved (14590 records). [2025-09-16T18:30:57.854107] [INFO ] [MAIN ] FOR: Full history saved. [2025-09-16T18:30:57.854238] [INFO ] [MAIN ] Update Progress: 860/2369 symbols completed. FOR took 22.5s. Estimated remaining: 2h 39m 31s. Rate: 9.5 symbols/min [2025-09-16T18:30:57.980889] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-04. [2025-09-16T18:30:58.121706] [INFO ] [MAIN ] FPH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:30:58.140137] [DEBUG] [LOCAL ] FPH: Loaded 14437 existing records from primary storage. [2025-09-16T18:30:58.140325] [INFO ] [MAIN ] FPH: Loaded 14437 existing records. [2025-09-16T18:30:58.433549] [WARN ] [MAIN ] FPH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:30:58.433918] [INFO ] [MAIN ] FPH: Wiping existing primary data before full refresh. [2025-09-16T18:30:58.434013] [INFO ] [LOCAL ] FPH: Starting deletion of primary data. [2025-09-16T18:30:58.434491] [DEBUG] [LOCAL ] FPH: Deleted primary data file. [2025-09-16T18:30:58.434556] [INFO ] [MAIN ] FPH: Starting full primary refresh. [2025-09-16T18:30:58.788933] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-09. [2025-09-16T18:30:58.918914] [INFO ] [REMOTE ] FPH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:30:59.210549] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-02 (2.04 seconds). [2025-09-16T18:30:59.258320] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-09-16T18:30:59.574904] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-03. [2025-09-16T18:31:00.374914] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-02. [2025-09-16T18:31:00.585898] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-09-16T18:31:00.590901] [INFO ] [REMOTE ] FNV: Full fetch complete (49997 records). [2025-09-16T18:31:00.591020] [DEBUG] [VALIDATOR ] FNV: Validating primary data. [2025-09-16T18:31:00.592292] [FATAL] [VALIDATOR ] FNV: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:31:00.592558] [FATAL] [MAIN ] FNV: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:31:00.592596] [WARN ] [MAIN ] FNV: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:31:00.592687] [INFO ] [LOCAL ] FNV: Starting deletion of primary data. [2025-09-16T18:31:00.592749] [DEBUG] [LOCAL ] FNV: No primary data file found to delete. [2025-09-16T18:31:00.592786] [INFO ] [LOCAL ] FNV: Starting deletion of derivative data. [2025-09-16T18:31:00.593007] [DEBUG] [LOCAL ] FNV: Deleted derivative data file. [2025-09-16T18:31:00.593086] [INFO ] [MAIN ] Update Progress: 861/2369 symbols completed. FNV took 34.0s. Estimated remaining: 2h 39m 19s. Rate: 9.5 symbols/min [2025-09-16T18:31:00.769782] [INFO ] [MAIN ] FPI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:31:00.769985] [DEBUG] [LOCAL ] FPI: No existing data file found in primary storage. [2025-09-16T18:31:00.770025] [WARN ] [MAIN ] FPI: No existing data found, performing full refresh instead. [2025-09-16T18:31:00.770161] [INFO ] [MAIN ] FPI: Wiping existing primary data before full refresh. [2025-09-16T18:31:00.770200] [INFO ] [LOCAL ] FPI: Starting deletion of primary data. [2025-09-16T18:31:00.770251] [DEBUG] [LOCAL ] FPI: No primary data file found to delete. [2025-09-16T18:31:00.770284] [INFO ] [MAIN ] FPI: Starting full primary refresh. [2025-09-16T18:31:01.127264] [INFO ] [REMOTE ] FPI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:31:01.174884] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-02. [2025-09-16T18:31:01.708055] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-03 (2.13 seconds). [2025-09-16T18:31:01.974953] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-04. [2025-09-16T18:31:02.249715] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-09-16T18:31:02.329224] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-09-16T18:31:02.774900] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-03. [2025-09-16T18:31:03.490496] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T18:31:03.574905] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-03. [2025-09-16T18:31:04.124252] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T18:31:04.374873] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-05. [2025-09-16T18:31:04.710036] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T18:31:05.174903] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-04. [2025-09-16T18:31:05.974912] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-05. [2025-09-16T18:31:06.153198] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-09-16T18:31:06.327813] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T18:31:06.774946] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-06. [2025-09-16T18:31:07.143164] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-09-16T18:31:07.574921] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-05. [2025-09-16T18:31:08.374881] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-04. [2025-09-16T18:31:08.451183] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T18:31:08.777704] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T18:31:09.174944] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-06. [2025-09-16T18:31:09.827244] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T18:31:09.974905] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-05. [2025-09-16T18:31:10.168808] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-09-16T18:31:10.774944] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-07. [2025-09-16T18:31:11.501440] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T18:31:11.574974] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-06. [2025-09-16T18:31:11.649494] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T18:31:12.374970] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-08. [2025-09-16T18:31:12.684118] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-09-16T18:31:13.174937] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-06. [2025-09-16T18:31:13.429425] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T18:31:13.903742] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-09-16T18:31:13.974970] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-07. [2025-09-16T18:31:14.774942] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-09. [2025-09-16T18:31:14.909605] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-09-16T18:31:15.518685] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-09-16T18:31:15.520944] [INFO ] [REMOTE ] FPF: Full fetch complete (19012 records). [2025-09-16T18:31:15.521048] [DEBUG] [VALIDATOR ] FPF: Validating primary data. [2025-09-16T18:31:15.584409] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-08. [2025-09-16T18:31:15.895423] [DEBUG] [VALIDATOR ] FPF: All expected trading days are present. [2025-09-16T18:31:15.895515] [DEBUG] [VALIDATOR ] FPF: Primary data validation passed. [2025-09-16T18:31:15.896110] [INFO ] [MAIN ] FPF: Fetched primary data is valid. [2025-09-16T18:31:15.896154] [DEBUG] [LOCAL ] FPF: Saving primary data. [2025-09-16T18:31:15.896907] [DEBUG] [LOCAL ] FPF: Trimmed 19012 records to 12881 (keeping 150 days of data) [2025-09-16T18:31:15.980025] [DEBUG] [LOCAL ] FPF: Primary data saved (12881 records). [2025-09-16T18:31:15.980155] [INFO ] [MAIN ] FPF: Full history saved. [2025-09-16T18:31:15.980251] [INFO ] [MAIN ] Update Progress: 862/2369 symbols completed. FPF took 23.8s. Estimated remaining: 2h 39m 28s. Rate: 9.5 symbols/min [2025-09-16T18:31:16.158585] [INFO ] [MAIN ] FR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:31:16.194728] [DEBUG] [LOCAL ] FR: Loaded 34195 existing records from primary storage. [2025-09-16T18:31:16.194866] [INFO ] [MAIN ] FR: Loaded 34195 existing records. [2025-09-16T18:31:16.383142] [WARN ] [MAIN ] FR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:31:16.383206] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-07. [2025-09-16T18:31:16.390631] [INFO ] [MAIN ] FR: Wiping existing primary data before full refresh. [2025-09-16T18:31:16.392581] [INFO ] [LOCAL ] FR: Starting deletion of primary data. [2025-09-16T18:31:16.393765] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T18:31:16.398504] [DEBUG] [LOCAL ] FR: Deleted primary data file. [2025-09-16T18:31:16.398561] [INFO ] [MAIN ] FR: Starting full primary refresh. [2025-09-16T18:31:16.751027] [INFO ] [REMOTE ] FR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:31:17.174922] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-02. [2025-09-16T18:31:17.974935] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-07. [2025-09-16T18:31:18.456241] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-09-16T18:31:18.713910] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-09-16T18:31:18.774913] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-09. [2025-09-16T18:31:19.231770] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T18:31:19.559282] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-09-16T18:31:19.562147] [INFO ] [REMOTE ] FPH: Full fetch complete (22086 records). [2025-09-16T18:31:19.562218] [DEBUG] [VALIDATOR ] FPH: Validating primary data. [2025-09-16T18:31:19.580024] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-08. [2025-09-16T18:31:19.949209] [DEBUG] [VALIDATOR ] FPH: All expected trading days are present. [2025-09-16T18:31:19.949289] [DEBUG] [VALIDATOR ] FPH: Primary data validation passed. [2025-09-16T18:31:19.949797] [INFO ] [MAIN ] FPH: Fetched primary data is valid. [2025-09-16T18:31:19.949844] [DEBUG] [LOCAL ] FPH: Saving primary data. [2025-09-16T18:31:19.950413] [DEBUG] [LOCAL ] FPH: Trimmed 22086 records to 14629 (keeping 150 days of data) [2025-09-16T18:31:20.037284] [DEBUG] [LOCAL ] FPH: Primary data saved (14629 records). [2025-09-16T18:31:20.037469] [INFO ] [MAIN ] FPH: Full history saved. [2025-09-16T18:31:20.037569] [INFO ] [MAIN ] Update Progress: 863/2369 symbols completed. FPH took 22.2s. Estimated remaining: 2h 39m 18s. Rate: 9.5 symbols/min [2025-09-16T18:31:20.215694] [INFO ] [MAIN ] FRA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:31:20.227494] [DEBUG] [LOCAL ] FRA: Loaded 9987 existing records from primary storage. [2025-09-16T18:31:20.227617] [INFO ] [MAIN ] FRA: Loaded 9987 existing records. [2025-09-16T18:31:20.380301] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-03. [2025-09-16T18:31:20.420928] [WARN ] [MAIN ] FRA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:31:20.421165] [INFO ] [MAIN ] FRA: Wiping existing primary data before full refresh. [2025-09-16T18:31:20.421224] [INFO ] [LOCAL ] FRA: Starting deletion of primary data. [2025-09-16T18:31:20.421634] [DEBUG] [LOCAL ] FRA: Deleted primary data file. [2025-09-16T18:31:20.421707] [INFO ] [MAIN ] FRA: Starting full primary refresh. [2025-09-16T18:31:20.796746] [INFO ] [REMOTE ] FRA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:31:20.808088] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-09-16T18:31:21.174914] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-02. [2025-09-16T18:31:21.975077] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-08. [2025-09-16T18:31:21.994708] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T18:31:22.774916] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-03. [2025-09-16T18:31:23.574882] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-09. [2025-09-16T18:31:23.766594] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-03 (3.39 seconds). [2025-09-16T18:31:23.980209] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-09-16T18:31:24.374915] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-09. [2025-09-16T18:31:24.650040] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-09-16T18:31:25.174919] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-04. [2025-09-16T18:31:25.892805] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-09 (2.32 seconds). [2025-09-16T18:31:25.897453] [INFO ] [REMOTE ] FPI: Full fetch complete (33901 records). [2025-09-16T18:31:25.897605] [DEBUG] [VALIDATOR ] FPI: Validating primary data. [2025-09-16T18:31:25.991072] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-04. [2025-09-16T18:31:26.293318] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-09-16T18:31:26.312467] [DEBUG] [VALIDATOR ] FPI: All expected trading days are present. [2025-09-16T18:31:26.312552] [DEBUG] [VALIDATOR ] FPI: Primary data validation passed. [2025-09-16T18:31:26.313211] [INFO ] [MAIN ] FPI: Fetched primary data is valid. [2025-09-16T18:31:26.313429] [DEBUG] [LOCAL ] FPI: Saving primary data. [2025-09-16T18:31:26.313356] [INFO ] [REMOTE ] FOUR: Full fetch complete (54071 records). [2025-09-16T18:31:26.313712] [DEBUG] [VALIDATOR ] FOUR: Validating primary data. [2025-09-16T18:31:26.349548] [DEBUG] [LOCAL ] FPI: Trimmed 33901 records to 23420 (keeping 150 days of data) [2025-09-16T18:31:26.828226] [DEBUG] [VALIDATOR ] FOUR: All expected trading days are present. [2025-09-16T18:31:26.828619] [DEBUG] [VALIDATOR ] FOUR: Primary data validation passed. [2025-09-16T18:31:26.831832] [INFO ] [MAIN ] FOUR: Fetched primary data is valid. [2025-09-16T18:31:26.832265] [DEBUG] [LOCAL ] FOUR: Saving primary data. [2025-09-16T18:31:26.836866] [DEBUG] [LOCAL ] FOUR: Trimmed 54071 records to 36659 (keeping 150 days of data) [2025-09-16T18:31:26.967299] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T18:31:26.987795] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-05. [2025-09-16T18:31:27.037690] [DEBUG] [LOCAL ] FPI: Primary data saved (23420 records). [2025-09-16T18:31:27.038882] [INFO ] [MAIN ] FPI: Full history saved. [2025-09-16T18:31:27.039023] [INFO ] [MAIN ] Update Progress: 864/2369 symbols completed. FPI took 26.4s. Estimated remaining: 2h 39m 12s. Rate: 9.5 symbols/min [2025-09-16T18:31:27.337519] [DEBUG] [LOCAL ] FOUR: Primary data saved (36659 records). [2025-09-16T18:31:27.341835] [INFO ] [MAIN ] FRGE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:31:27.342339] [DEBUG] [LOCAL ] FRGE: No existing data file found in primary storage. [2025-09-16T18:31:27.342247] [INFO ] [MAIN ] FOUR: Full history saved. [2025-09-16T18:31:27.342431] [WARN ] [MAIN ] FRGE: No existing data found, performing full refresh instead. [2025-09-16T18:31:27.342626] [INFO ] [MAIN ] Update Progress: 865/2369 symbols completed. FOUR took 36.4s. Estimated remaining: 2h 38m 56s. Rate: 9.5 symbols/min [2025-09-16T18:31:27.343193] [INFO ] [MAIN ] FRGE: Wiping existing primary data before full refresh. [2025-09-16T18:31:27.348520] [INFO ] [LOCAL ] FRGE: Starting deletion of primary data. [2025-09-16T18:31:27.376227] [DEBUG] [LOCAL ] FRGE: No primary data file found to delete. [2025-09-16T18:31:27.389445] [INFO ] [MAIN ] FRGE: Starting full primary refresh. [2025-09-16T18:31:27.594405] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T18:31:27.649215] [INFO ] [MAIN ] FRO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:31:27.649867] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-05. [2025-09-16T18:31:27.931308] [INFO ] [REMOTE ] FRGE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:31:27.943251] [DEBUG] [LOCAL ] FRO: Loaded 38124 existing records from primary storage. [2025-09-16T18:31:27.943362] [INFO ] [MAIN ] FRO: Loaded 38124 existing records. [2025-09-16T18:31:28.118305] [WARN ] [MAIN ] FRO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:31:28.118559] [INFO ] [MAIN ] FRO: Wiping existing primary data before full refresh. [2025-09-16T18:31:28.118610] [INFO ] [LOCAL ] FRO: Starting deletion of primary data. [2025-09-16T18:31:28.119579] [DEBUG] [LOCAL ] FRO: Deleted primary data file. [2025-09-16T18:31:28.119627] [INFO ] [MAIN ] FRO: Starting full primary refresh. [2025-09-16T18:31:28.380014] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-02. [2025-09-16T18:31:28.496976] [INFO ] [REMOTE ] FRO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:31:28.505768] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T18:31:28.631550] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T18:31:29.174909] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-06. [2025-09-16T18:31:29.365237] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-09-16T18:31:29.852361] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-09-16T18:31:29.974915] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-07. [2025-09-16T18:31:30.774912] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-02. [2025-09-16T18:31:31.006314] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-09-16T18:31:31.574918] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-08. [2025-09-16T18:31:32.374903] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-06. [2025-09-16T18:31:32.459169] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-09-16T18:31:32.727691] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-09-16T18:31:33.174907] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-03. [2025-09-16T18:31:33.974908] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-03. [2025-09-16T18:31:34.137888] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-09-16T18:31:34.303799] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-09-16T18:31:34.774904] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-09. [2025-09-16T18:31:35.419837] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-09-16T18:31:35.422296] [INFO ] [REMOTE ] FRA: Full fetch complete (16457 records). [2025-09-16T18:31:35.422390] [DEBUG] [VALIDATOR ] FRA: Validating primary data. [2025-09-16T18:31:35.580627] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-04. [2025-09-16T18:31:35.827390] [DEBUG] [VALIDATOR ] FRA: All expected trading days are present. [2025-09-16T18:31:35.827522] [DEBUG] [VALIDATOR ] FRA: Primary data validation passed. [2025-09-16T18:31:35.827982] [INFO ] [MAIN ] FRA: Fetched primary data is valid. [2025-09-16T18:31:35.828082] [DEBUG] [LOCAL ] FRA: Saving primary data. [2025-09-16T18:31:35.829574] [DEBUG] [LOCAL ] FRA: Trimmed 16457 records to 10140 (keeping 150 days of data) [2025-09-16T18:31:35.896397] [DEBUG] [LOCAL ] FRA: Primary data saved (10140 records). [2025-09-16T18:31:35.896557] [INFO ] [MAIN ] FRA: Full history saved. [2025-09-16T18:31:35.896715] [INFO ] [MAIN ] Update Progress: 866/2369 symbols completed. FRA took 15.9s. Estimated remaining: 2h 38m 53s. Rate: 9.5 symbols/min [2025-09-16T18:31:36.074238] [INFO ] [MAIN ] FRT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:31:36.074413] [DEBUG] [LOCAL ] FRT: No existing data file found in primary storage. [2025-09-16T18:31:36.074531] [WARN ] [MAIN ] FRT: No existing data found, performing full refresh instead. [2025-09-16T18:31:36.074872] [INFO ] [MAIN ] FRT: Wiping existing primary data before full refresh. [2025-09-16T18:31:36.074934] [INFO ] [LOCAL ] FRT: Starting deletion of primary data. [2025-09-16T18:31:36.075221] [DEBUG] [LOCAL ] FRT: No primary data file found to delete. [2025-09-16T18:31:36.075300] [INFO ] [MAIN ] FRT: Starting full primary refresh. [2025-09-16T18:31:36.230770] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-03 (2.26 seconds). [2025-09-16T18:31:36.379955] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-04. [2025-09-16T18:31:36.485421] [INFO ] [REMOTE ] FRT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:31:36.886199] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T18:31:37.174961] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-07. [2025-09-16T18:31:37.965461] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-09-16T18:31:37.975823] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-05. [2025-09-16T18:31:38.439889] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T18:31:38.774904] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-08. [2025-09-16T18:31:39.004999] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-09-16T18:31:39.574921] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-06. [2025-09-16T18:31:40.374907] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-02. [2025-09-16T18:31:40.387530] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-09-16T18:31:40.424003] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T18:31:41.174956] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-05. [2025-09-16T18:31:41.901810] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T18:31:41.974847] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-03. [2025-09-16T18:31:42.774949] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-09. [2025-09-16T18:31:42.951256] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-09-16T18:31:43.574980] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-06. [2025-09-16T18:31:43.860394] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-09-16T18:31:44.375021] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-07. [2025-09-16T18:31:44.411872] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T18:31:44.417723] [INFO ] [REMOTE ] FR: Full fetch complete (50673 records). [2025-09-16T18:31:44.417875] [DEBUG] [VALIDATOR ] FR: Validating primary data. [2025-09-16T18:31:44.801576] [DEBUG] [VALIDATOR ] FR: All expected trading days are present. [2025-09-16T18:31:44.801673] [DEBUG] [VALIDATOR ] FR: Primary data validation passed. [2025-09-16T18:31:44.802450] [INFO ] [MAIN ] FR: Fetched primary data is valid. [2025-09-16T18:31:44.802493] [DEBUG] [LOCAL ] FR: Saving primary data. [2025-09-16T18:31:44.803513] [DEBUG] [LOCAL ] FR: Trimmed 50673 records to 34514 (keeping 150 days of data) [2025-09-16T18:31:45.013557] [DEBUG] [LOCAL ] FR: Primary data saved (34514 records). [2025-09-16T18:31:45.013693] [INFO ] [MAIN ] FR: Full history saved. [2025-09-16T18:31:45.018533] [INFO ] [MAIN ] Update Progress: 867/2369 symbols completed. FR took 29.0s. Estimated remaining: 2h 38m 52s. Rate: 9.5 symbols/min [2025-09-16T18:31:45.188724] [INFO ] [MAIN ] FSCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:31:45.186512] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-04. [2025-09-16T18:31:45.188886] [DEBUG] [LOCAL ] FSCO: No existing data file found in primary storage. [2025-09-16T18:31:45.189604] [WARN ] [MAIN ] FSCO: No existing data found, performing full refresh instead. [2025-09-16T18:31:45.189764] [INFO ] [MAIN ] FSCO: Wiping existing primary data before full refresh. [2025-09-16T18:31:45.189819] [INFO ] [LOCAL ] FSCO: Starting deletion of primary data. [2025-09-16T18:31:45.189880] [DEBUG] [LOCAL ] FSCO: No primary data file found to delete. [2025-09-16T18:31:45.189926] [INFO ] [MAIN ] FSCO: Starting full primary refresh. [2025-09-16T18:31:45.565077] [INFO ] [REMOTE ] FSCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:31:45.560693] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-09-16T18:31:45.599919] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-06 (2.02 seconds). [2025-09-16T18:31:45.974915] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-02. [2025-09-16T18:31:46.774875] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-08. [2025-09-16T18:31:46.989665] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-09-16T18:31:47.438059] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-09-16T18:31:47.574911] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-07. [2025-09-16T18:31:47.598682] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-09-16T18:31:48.374976] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-03. [2025-09-16T18:31:49.174949] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-05. [2025-09-16T18:31:49.269631] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T18:31:49.744813] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T18:31:49.974939] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-08. [2025-09-16T18:31:50.758592] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T18:31:50.774914] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-06. [2025-09-16T18:31:51.574892] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-09. [2025-09-16T18:31:51.648502] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:31:52.149429] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T18:31:52.374899] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-09. [2025-09-16T18:31:52.450106] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-09-16T18:31:52.451954] [INFO ] [REMOTE ] FRGE: Full fetch complete (22065 records). [2025-09-16T18:31:52.452023] [DEBUG] [VALIDATOR ] FRGE: Validating primary data. [2025-09-16T18:31:52.820683] [DEBUG] [VALIDATOR ] FRGE: All expected trading days are present. [2025-09-16T18:31:52.820770] [DEBUG] [VALIDATOR ] FRGE: Primary data validation passed. [2025-09-16T18:31:52.821313] [INFO ] [MAIN ] FRGE: Fetched primary data is valid. [2025-09-16T18:31:52.821361] [DEBUG] [LOCAL ] FRGE: Saving primary data. [2025-09-16T18:31:52.822023] [DEBUG] [LOCAL ] FRGE: Trimmed 22065 records to 11551 (keeping 150 days of data) [2025-09-16T18:31:52.892382] [DEBUG] [LOCAL ] FRGE: Primary data saved (11551 records). [2025-09-16T18:31:52.892520] [INFO ] [MAIN ] FRGE: Full history saved. [2025-09-16T18:31:52.892597] [INFO ] [MAIN ] Update Progress: 868/2369 symbols completed. FRGE took 25.9s. Estimated remaining: 2h 38m 48s. Rate: 9.5 symbols/min [2025-09-16T18:31:53.062085] [INFO ] [MAIN ] FSK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:31:53.100938] [DEBUG] [LOCAL ] FSK: Loaded 35641 existing records from primary storage. [2025-09-16T18:31:53.101078] [INFO ] [MAIN ] FSK: Loaded 35641 existing records. [2025-09-16T18:31:53.179970] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-04. [2025-09-16T18:31:53.289837] [WARN ] [MAIN ] FSK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:31:53.290101] [INFO ] [MAIN ] FSK: Wiping existing primary data before full refresh. [2025-09-16T18:31:53.290170] [INFO ] [LOCAL ] FSK: Starting deletion of primary data. [2025-09-16T18:31:53.291007] [DEBUG] [LOCAL ] FSK: Deleted primary data file. [2025-09-16T18:31:53.291059] [INFO ] [MAIN ] FSK: Starting full primary refresh. [2025-09-16T18:31:53.691397] [INFO ] [REMOTE ] FSK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:31:53.974952] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-07. [2025-09-16T18:31:54.013037] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T18:31:54.020225] [INFO ] [REMOTE ] FRO: Full fetch complete (56522 records). [2025-09-16T18:31:54.020392] [DEBUG] [VALIDATOR ] FRO: Validating primary data. [2025-09-16T18:31:54.436968] [DEBUG] [VALIDATOR ] FRO: All expected trading days are present. [2025-09-16T18:31:54.437092] [DEBUG] [VALIDATOR ] FRO: Primary data validation passed. [2025-09-16T18:31:54.438139] [INFO ] [MAIN ] FRO: Fetched primary data is valid. [2025-09-16T18:31:54.438207] [DEBUG] [LOCAL ] FRO: Saving primary data. [2025-09-16T18:31:54.440027] [DEBUG] [LOCAL ] FRO: Trimmed 56522 records to 38509 (keeping 150 days of data) [2025-09-16T18:31:54.695634] [DEBUG] [LOCAL ] FRO: Primary data saved (38509 records). [2025-09-16T18:31:54.695798] [INFO ] [MAIN ] FRO: Full history saved. [2025-09-16T18:31:54.695901] [INFO ] [MAIN ] Update Progress: 869/2369 symbols completed. FRO took 27.4s. Estimated remaining: 2h 38m 34s. Rate: 9.5 symbols/min [2025-09-16T18:31:54.777232] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-02. [2025-09-16T18:31:54.871425] [INFO ] [MAIN ] FSLY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:31:54.896364] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T18:31:54.937550] [DEBUG] [LOCAL ] FSLY: Loaded 37625 existing records from primary storage. [2025-09-16T18:31:54.937780] [INFO ] [MAIN ] FSLY: Loaded 37625 existing records. [2025-09-16T18:31:55.118869] [WARN ] [MAIN ] FSLY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:31:55.119218] [INFO ] [MAIN ] FSLY: Wiping existing primary data before full refresh. [2025-09-16T18:31:55.119272] [INFO ] [LOCAL ] FSLY: Starting deletion of primary data. [2025-09-16T18:31:55.120330] [DEBUG] [LOCAL ] FSLY: Deleted primary data file. [2025-09-16T18:31:55.120382] [INFO ] [MAIN ] FSLY: Starting full primary refresh. [2025-09-16T18:31:55.495338] [INFO ] [REMOTE ] FSLY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:31:55.574907] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-05. [2025-09-16T18:31:55.698727] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-09-16T18:31:56.374916] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-02. [2025-09-16T18:31:56.561096] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T18:31:56.761669] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-09-16T18:31:57.174940] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-06. [2025-09-16T18:31:57.974905] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-08. [2025-09-16T18:31:58.249832] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-09-16T18:31:58.311386] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:31:58.774913] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-03. [2025-09-16T18:31:59.507615] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T18:31:59.574860] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-09. [2025-09-16T18:32:00.374908] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-03. [2025-09-16T18:32:00.693844] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-09-16T18:32:01.160915] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T18:32:01.166718] [INFO ] [REMOTE ] FRT: Full fetch complete (45998 records). [2025-09-16T18:32:01.166866] [DEBUG] [VALIDATOR ] FRT: Validating primary data. [2025-09-16T18:32:01.180187] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-04. [2025-09-16T18:32:01.558072] [DEBUG] [VALIDATOR ] FRT: All expected trading days are present. [2025-09-16T18:32:01.558157] [DEBUG] [VALIDATOR ] FRT: Primary data validation passed. [2025-09-16T18:32:01.558943] [INFO ] [MAIN ] FRT: Fetched primary data is valid. [2025-09-16T18:32:01.558999] [DEBUG] [LOCAL ] FRT: Saving primary data. [2025-09-16T18:32:01.560028] [DEBUG] [LOCAL ] FRT: Trimmed 45998 records to 31128 (keeping 150 days of data) [2025-09-16T18:32:01.759803] [DEBUG] [LOCAL ] FRT: Primary data saved (31128 records). [2025-09-16T18:32:01.759954] [INFO ] [MAIN ] FRT: Full history saved. [2025-09-16T18:32:01.760065] [INFO ] [MAIN ] Update Progress: 870/2369 symbols completed. FRT took 25.9s. Estimated remaining: 2h 38m 29s. Rate: 9.5 symbols/min [2025-09-16T18:32:01.938239] [INFO ] [MAIN ] FSM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:32:01.980070] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-07. [2025-09-16T18:32:01.987942] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T18:32:02.024874] [DEBUG] [LOCAL ] FSM: Loaded 39280 existing records from primary storage. [2025-09-16T18:32:02.025029] [INFO ] [MAIN ] FSM: Loaded 39280 existing records. [2025-09-16T18:32:02.212889] [WARN ] [MAIN ] FSM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:32:02.213158] [INFO ] [MAIN ] FSM: Wiping existing primary data before full refresh. [2025-09-16T18:32:02.213204] [INFO ] [LOCAL ] FSM: Starting deletion of primary data. [2025-09-16T18:32:02.213966] [DEBUG] [LOCAL ] FSM: Deleted primary data file. [2025-09-16T18:32:02.214014] [INFO ] [MAIN ] FSM: Starting full primary refresh. [2025-09-16T18:32:02.576871] [INFO ] [REMOTE ] FSM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:32:02.618251] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-09-16T18:32:02.774935] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-05. [2025-09-16T18:32:03.575071] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-02. [2025-09-16T18:32:03.682507] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T18:32:04.374915] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-04. [2025-09-16T18:32:04.471702] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T18:32:05.174919] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-08. [2025-09-16T18:32:05.876663] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T18:32:05.917546] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-02 (2.34 seconds). [2025-09-16T18:32:05.974903] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-05. [2025-09-16T18:32:06.614102] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T18:32:06.774900] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-09. [2025-09-16T18:32:07.574906] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-03. [2025-09-16T18:32:07.730339] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T18:32:08.151269] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T18:32:08.158355] [INFO ] [REMOTE ] FSCO: Full fetch complete (39927 records). [2025-09-16T18:32:08.158460] [DEBUG] [VALIDATOR ] FSCO: Validating primary data. [2025-09-16T18:32:08.380023] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-06. [2025-09-16T18:32:08.531992] [DEBUG] [VALIDATOR ] FSCO: All expected trading days are present. [2025-09-16T18:32:08.532069] [DEBUG] [VALIDATOR ] FSCO: Primary data validation passed. [2025-09-16T18:32:08.532719] [INFO ] [MAIN ] FSCO: Fetched primary data is valid. [2025-09-16T18:32:08.532765] [DEBUG] [LOCAL ] FSCO: Saving primary data. [2025-09-16T18:32:08.533631] [DEBUG] [LOCAL ] FSCO: Trimmed 39927 records to 27172 (keeping 150 days of data) [2025-09-16T18:32:08.699031] [DEBUG] [LOCAL ] FSCO: Primary data saved (27172 records). [2025-09-16T18:32:08.699169] [INFO ] [MAIN ] FSCO: Full history saved. [2025-09-16T18:32:08.699257] [INFO ] [MAIN ] Update Progress: 871/2369 symbols completed. FSCO took 23.7s. Estimated remaining: 2h 38m 23s. Rate: 9.5 symbols/min [2025-09-16T18:32:08.870278] [INFO ] [MAIN ] FSS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:32:08.894205] [DEBUG] [LOCAL ] FSS: Loaded 24725 existing records from primary storage. [2025-09-16T18:32:08.894361] [INFO ] [MAIN ] FSS: Loaded 24725 existing records. [2025-09-16T18:32:09.069381] [WARN ] [MAIN ] FSS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:32:09.069624] [INFO ] [MAIN ] FSS: Wiping existing primary data before full refresh. [2025-09-16T18:32:09.069688] [INFO ] [LOCAL ] FSS: Starting deletion of primary data. [2025-09-16T18:32:09.070387] [DEBUG] [LOCAL ] FSS: Deleted primary data file. [2025-09-16T18:32:09.070436] [INFO ] [MAIN ] FSS: Starting full primary refresh. [2025-09-16T18:32:09.183984] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-06. [2025-09-16T18:32:09.431379] [INFO ] [REMOTE ] FSS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:32:09.464098] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-09-16T18:32:09.974899] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-02. [2025-09-16T18:32:10.287424] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-09-16T18:32:10.774938] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-07. [2025-09-16T18:32:11.079529] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-09-16T18:32:11.462172] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T18:32:11.574908] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-03. [2025-09-16T18:32:12.183750] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-09-16T18:32:12.374922] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-07. [2025-09-16T18:32:13.174902] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-04. [2025-09-16T18:32:13.553859] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-09-16T18:32:13.960367] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T18:32:13.974851] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-04. [2025-09-16T18:32:14.774926] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-08. [2025-09-16T18:32:15.198658] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-09-16T18:32:15.262143] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-09-16T18:32:15.574897] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-05. [2025-09-16T18:32:16.374900] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-05. [2025-09-16T18:32:16.700771] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T18:32:17.174967] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-09. [2025-09-16T18:32:17.520399] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-09-16T18:32:17.722115] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T18:32:17.974907] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-06. [2025-09-16T18:32:18.597832] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T18:32:18.604129] [INFO ] [REMOTE ] FSK: Full fetch complete (53655 records). [2025-09-16T18:32:18.604248] [DEBUG] [VALIDATOR ] FSK: Validating primary data. [2025-09-16T18:32:18.777955] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-06. [2025-09-16T18:32:18.984218] [DEBUG] [VALIDATOR ] FSK: All expected trading days are present. [2025-09-16T18:32:18.984299] [DEBUG] [VALIDATOR ] FSK: Primary data validation passed. [2025-09-16T18:32:18.985147] [INFO ] [MAIN ] FSK: Fetched primary data is valid. [2025-09-16T18:32:18.985236] [DEBUG] [LOCAL ] FSK: Saving primary data. [2025-09-16T18:32:18.986472] [DEBUG] [LOCAL ] FSK: Trimmed 53655 records to 36028 (keeping 150 days of data) [2025-09-16T18:32:19.210609] [DEBUG] [LOCAL ] FSK: Primary data saved (36028 records). [2025-09-16T18:32:19.210773] [INFO ] [MAIN ] FSK: Full history saved. [2025-09-16T18:32:19.210861] [INFO ] [MAIN ] Update Progress: 872/2369 symbols completed. FSK took 26.3s. Estimated remaining: 2h 38m 24s. Rate: 9.5 symbols/min [2025-09-16T18:32:19.416662] [INFO ] [MAIN ] FT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:32:19.437263] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T18:32:19.446213] [DEBUG] [LOCAL ] FT: Loaded 4670 existing records from primary storage. [2025-09-16T18:32:19.446558] [INFO ] [MAIN ] FT: Loaded 4670 existing records. [2025-09-16T18:32:19.587371] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-07. [2025-09-16T18:32:19.624548] [WARN ] [MAIN ] FT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:32:19.624770] [INFO ] [MAIN ] FT: Wiping existing primary data before full refresh. [2025-09-16T18:32:19.624820] [INFO ] [LOCAL ] FT: Starting deletion of primary data. [2025-09-16T18:32:19.625035] [DEBUG] [LOCAL ] FT: Deleted primary data file. [2025-09-16T18:32:19.625081] [INFO ] [MAIN ] FT: Starting full primary refresh. [2025-09-16T18:32:19.979630] [INFO ] [REMOTE ] FT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:32:20.374882] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-02. [2025-09-16T18:32:20.740533] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-06 (1.96 seconds). [2025-09-16T18:32:20.822597] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-09-16T18:32:21.174873] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-08. [2025-09-16T18:32:21.277059] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-09-16T18:32:21.974967] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-07. [2025-09-16T18:32:22.384923] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-09-16T18:32:22.774918] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-03. [2025-09-16T18:32:23.574933] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-08. [2025-09-16T18:32:23.684984] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-09-16T18:32:23.948631] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-09-16T18:32:24.374895] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-04. [2025-09-16T18:32:25.130919] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T18:32:25.174892] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-09. [2025-09-16T18:32:25.974879] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-09. [2025-09-16T18:32:26.022938] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T18:32:26.026090] [INFO ] [REMOTE ] FSS: Full fetch complete (37938 records). [2025-09-16T18:32:26.026179] [DEBUG] [VALIDATOR ] FSS: Validating primary data. [2025-09-16T18:32:26.026740] [FATAL] [VALIDATOR ] FSS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:32:26.026815] [FATAL] [MAIN ] FSS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:32:26.026852] [WARN ] [MAIN ] FSS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:32:26.026889] [INFO ] [LOCAL ] FSS: Starting deletion of primary data. [2025-09-16T18:32:26.027000] [DEBUG] [LOCAL ] FSS: No primary data file found to delete. [2025-09-16T18:32:26.027038] [INFO ] [LOCAL ] FSS: Starting deletion of derivative data. [2025-09-16T18:32:26.027386] [DEBUG] [LOCAL ] FSS: Deleted derivative data file. [2025-09-16T18:32:26.027465] [INFO ] [MAIN ] Update Progress: 873/2369 symbols completed. FSS took 17.3s. Estimated remaining: 2h 38m 19s. Rate: 9.4 symbols/min [2025-09-16T18:32:26.196069] [INFO ] [MAIN ] FTHY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:32:26.205331] [DEBUG] [LOCAL ] FTHY: Loaded 8497 existing records from primary storage. [2025-09-16T18:32:26.205431] [INFO ] [MAIN ] FTHY: Loaded 8497 existing records. [2025-09-16T18:32:26.392712] [WARN ] [MAIN ] FTHY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:32:26.392966] [INFO ] [MAIN ] FTHY: Wiping existing primary data before full refresh. [2025-09-16T18:32:26.393017] [INFO ] [LOCAL ] FTHY: Starting deletion of primary data. [2025-09-16T18:32:26.393323] [DEBUG] [LOCAL ] FTHY: Deleted primary data file. [2025-09-16T18:32:26.393369] [INFO ] [MAIN ] FTHY: Starting full primary refresh. [2025-09-16T18:32:26.680947] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-04 (2.31 seconds). [2025-09-16T18:32:26.760115] [INFO ] [REMOTE ] FTHY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:32:26.774867] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-08. [2025-09-16T18:32:27.367698] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T18:32:27.374306] [INFO ] [REMOTE ] FSLY: Full fetch complete (55900 records). [2025-09-16T18:32:27.374479] [DEBUG] [VALIDATOR ] FSLY: Validating primary data. [2025-09-16T18:32:27.580945] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-02. [2025-09-16T18:32:27.757553] [DEBUG] [VALIDATOR ] FSLY: All expected trading days are present. [2025-09-16T18:32:27.757633] [DEBUG] [VALIDATOR ] FSLY: Primary data validation passed. [2025-09-16T18:32:27.758449] [INFO ] [MAIN ] FSLY: Fetched primary data is valid. [2025-09-16T18:32:27.758494] [DEBUG] [LOCAL ] FSLY: Saving primary data. [2025-09-16T18:32:27.759664] [DEBUG] [LOCAL ] FSLY: Trimmed 55900 records to 38013 (keeping 150 days of data) [2025-09-16T18:32:27.978872] [DEBUG] [LOCAL ] FSLY: Primary data saved (38013 records). [2025-09-16T18:32:27.979097] [INFO ] [MAIN ] FSLY: Full history saved. [2025-09-16T18:32:27.979197] [INFO ] [MAIN ] Update Progress: 874/2369 symbols completed. FSLY took 33.3s. Estimated remaining: 2h 38m 5s. Rate: 9.5 symbols/min [2025-09-16T18:32:28.153994] [INFO ] [MAIN ] FTI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:32:28.154162] [DEBUG] [LOCAL ] FTI: No existing data file found in primary storage. [2025-09-16T18:32:28.154204] [WARN ] [MAIN ] FTI: No existing data found, performing full refresh instead. [2025-09-16T18:32:28.154364] [INFO ] [MAIN ] FTI: Wiping existing primary data before full refresh. [2025-09-16T18:32:28.154406] [INFO ] [LOCAL ] FTI: Starting deletion of primary data. [2025-09-16T18:32:28.154448] [DEBUG] [LOCAL ] FTI: No primary data file found to delete. [2025-09-16T18:32:28.154483] [INFO ] [MAIN ] FTI: Starting full primary refresh. [2025-09-16T18:32:28.307755] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-09-16T18:32:28.383309] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-05. [2025-09-16T18:32:28.531765] [INFO ] [REMOTE ] FTI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:32:28.564164] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T18:32:28.906166] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-09-16T18:32:29.174964] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-06. [2025-09-16T18:32:29.641966] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-09-16T18:32:29.974910] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-07. [2025-09-16T18:32:30.526855] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-09-16T18:32:30.774920] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-03. [2025-09-16T18:32:31.574910] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-09. [2025-09-16T18:32:31.587475] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-09-16T18:32:32.374900] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-08. [2025-09-16T18:32:32.978321] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T18:32:32.992766] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T18:32:33.000991] [INFO ] [REMOTE ] FSM: Full fetch complete (57910 records). [2025-09-16T18:32:33.001162] [DEBUG] [VALIDATOR ] FSM: Validating primary data. [2025-09-16T18:32:33.186227] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-09. [2025-09-16T18:32:33.411557] [DEBUG] [VALIDATOR ] FSM: All expected trading days are present. [2025-09-16T18:32:33.411692] [DEBUG] [VALIDATOR ] FSM: Primary data validation passed. [2025-09-16T18:32:33.412706] [INFO ] [MAIN ] FSM: Fetched primary data is valid. [2025-09-16T18:32:33.412782] [DEBUG] [LOCAL ] FSM: Saving primary data. [2025-09-16T18:32:33.414231] [DEBUG] [LOCAL ] FSM: Trimmed 57910 records to 39669 (keeping 150 days of data) [2025-09-16T18:32:33.654592] [DEBUG] [LOCAL ] FSM: Primary data saved (39669 records). [2025-09-16T18:32:33.654970] [INFO ] [MAIN ] FSM: Full history saved. [2025-09-16T18:32:33.655118] [INFO ] [MAIN ] Update Progress: 875/2369 symbols completed. FSM took 31.9s. Estimated remaining: 2h 37m 57s. Rate: 9.5 symbols/min [2025-09-16T18:32:33.837691] [INFO ] [MAIN ] FTK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:32:33.841703] [DEBUG] [LOCAL ] FTK: No existing data file found in primary storage. [2025-09-16T18:32:33.841611] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-09-16T18:32:33.841830] [WARN ] [MAIN ] FTK: No existing data found, performing full refresh instead. [2025-09-16T18:32:33.843168] [INFO ] [REMOTE ] FT: Full fetch complete (6714 records). [2025-09-16T18:32:33.843300] [DEBUG] [VALIDATOR ] FT: Validating primary data. [2025-09-16T18:32:33.843448] [INFO ] [MAIN ] FTK: Wiping existing primary data before full refresh. [2025-09-16T18:32:33.888833] [INFO ] [LOCAL ] FTK: Starting deletion of primary data. [2025-09-16T18:32:33.916335] [DEBUG] [LOCAL ] FTK: No primary data file found to delete. [2025-09-16T18:32:33.922747] [INFO ] [MAIN ] FTK: Starting full primary refresh. [2025-09-16T18:32:33.979981] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-04. [2025-09-16T18:32:34.550147] [DEBUG] [VALIDATOR ] FT: All expected trading days are present. [2025-09-16T18:32:34.567506] [DEBUG] [VALIDATOR ] FT: Primary data validation passed. [2025-09-16T18:32:34.568883] [INFO ] [MAIN ] FT: Fetched primary data is valid. [2025-09-16T18:32:34.584130] [DEBUG] [LOCAL ] FT: Saving primary data. [2025-09-16T18:32:34.588617] [INFO ] [REMOTE ] FTK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:32:34.590062] [DEBUG] [LOCAL ] FT: Trimmed 6714 records to 4736 (keeping 150 days of data) [2025-09-16T18:32:34.618802] [DEBUG] [LOCAL ] FT: Primary data saved (4736 records). [2025-09-16T18:32:34.618966] [INFO ] [MAIN ] FT: Full history saved. [2025-09-16T18:32:34.619067] [INFO ] [MAIN ] Update Progress: 876/2369 symbols completed. FT took 15.4s. Estimated remaining: 2h 37m 42s. Rate: 9.5 symbols/min [2025-09-16T18:32:34.783934] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-02. [2025-09-16T18:32:34.802813] [INFO ] [MAIN ] FTS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:32:34.816383] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-09-16T18:32:34.848241] [DEBUG] [LOCAL ] FTS: Loaded 31167 existing records from primary storage. [2025-09-16T18:32:34.848385] [INFO ] [MAIN ] FTS: Loaded 31167 existing records. [2025-09-16T18:32:35.032262] [WARN ] [MAIN ] FTS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:32:35.032530] [INFO ] [MAIN ] FTS: Wiping existing primary data before full refresh. [2025-09-16T18:32:35.032576] [INFO ] [LOCAL ] FTS: Starting deletion of primary data. [2025-09-16T18:32:35.033402] [DEBUG] [LOCAL ] FTS: Deleted primary data file. [2025-09-16T18:32:35.033451] [INFO ] [MAIN ] FTS: Starting full primary refresh. [2025-09-16T18:32:35.392389] [INFO ] [REMOTE ] FTS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:32:35.574873] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-02. [2025-09-16T18:32:36.271183] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T18:32:36.374920] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-02. [2025-09-16T18:32:36.614891] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T18:32:37.129845] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T18:32:37.174873] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-03. [2025-09-16T18:32:37.974924] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-05. [2025-09-16T18:32:38.186748] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-09-16T18:32:38.656990] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-09-16T18:32:38.774922] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-04. [2025-09-16T18:32:39.574939] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-06. [2025-09-16T18:32:39.913839] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-09-16T18:32:40.347344] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T18:32:40.374865] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-07. [2025-09-16T18:32:40.879451] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-09-16T18:32:41.174905] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-03. [2025-09-16T18:32:41.975017] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-03. [2025-09-16T18:32:41.975716] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T18:32:42.774957] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-08. [2025-09-16T18:32:43.323587] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T18:32:43.574983] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-04. [2025-09-16T18:32:43.683750] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-09-16T18:32:44.374928] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-05. [2025-09-16T18:32:44.426971] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-09-16T18:32:45.174917] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-05. [2025-09-16T18:32:45.594243] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T18:32:45.974910] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-04. [2025-09-16T18:32:46.380269] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-09-16T18:32:46.774902] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-06. [2025-09-16T18:32:47.574925] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-09. [2025-09-16T18:32:48.013375] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-09-16T18:32:48.374944] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-06. [2025-09-16T18:32:48.422424] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T18:32:48.506170] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T18:32:48.508132] [INFO ] [REMOTE ] FTHY: Full fetch complete (12682 records). [2025-09-16T18:32:48.508202] [DEBUG] [VALIDATOR ] FTHY: Validating primary data. [2025-09-16T18:32:48.871156] [DEBUG] [VALIDATOR ] FTHY: All expected trading days are present. [2025-09-16T18:32:48.871244] [DEBUG] [VALIDATOR ] FTHY: Primary data validation passed. [2025-09-16T18:32:48.871569] [INFO ] [MAIN ] FTHY: Fetched primary data is valid. [2025-09-16T18:32:48.871617] [DEBUG] [LOCAL ] FTHY: Saving primary data. [2025-09-16T18:32:48.872212] [DEBUG] [LOCAL ] FTHY: Trimmed 12682 records to 8557 (keeping 150 days of data) [2025-09-16T18:32:48.925749] [DEBUG] [LOCAL ] FTHY: Primary data saved (8557 records). [2025-09-16T18:32:48.925897] [INFO ] [MAIN ] FTHY: Full history saved. [2025-09-16T18:32:48.926005] [INFO ] [MAIN ] Update Progress: 877/2369 symbols completed. FTHY took 22.9s. Estimated remaining: 2h 37m 49s. Rate: 9.5 symbols/min [2025-09-16T18:32:49.174941] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-05. [2025-09-16T18:32:49.664898] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T18:32:49.974935] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-07. [2025-09-16T18:32:50.579034] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T18:32:50.774940] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-06. [2025-09-16T18:32:51.574928] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-07. [2025-09-16T18:32:52.130822] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-09-16T18:32:52.374963] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-07. [2025-09-16T18:32:52.578199] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T18:32:53.174920] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-08. [2025-09-16T18:32:53.757856] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-09-16T18:32:53.974901] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-08. [2025-09-16T18:32:54.019888] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-09-16T18:32:54.774914] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-09. [2025-09-16T18:32:54.900890] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-07 (4.93 seconds). [2025-09-16T18:32:55.560101] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T18:32:55.574845] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-09. [2025-09-16T18:32:55.915776] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T18:32:55.919409] [INFO ] [REMOTE ] FTK: Full fetch complete (33369 records). [2025-09-16T18:32:55.919482] [DEBUG] [VALIDATOR ] FTK: Validating primary data. [2025-09-16T18:32:55.920055] [FATAL] [VALIDATOR ] FTK: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:32:55.920128] [FATAL] [MAIN ] FTK: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:32:55.920165] [WARN ] [MAIN ] FTK: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:32:55.920209] [INFO ] [LOCAL ] FTK: Starting deletion of primary data. [2025-09-16T18:32:55.920265] [DEBUG] [LOCAL ] FTK: No primary data file found to delete. [2025-09-16T18:32:55.920319] [INFO ] [LOCAL ] FTK: Starting deletion of derivative data. [2025-09-16T18:32:55.920375] [DEBUG] [LOCAL ] FTK: No derivative data file found to delete. [2025-09-16T18:32:55.920434] [INFO ] [MAIN ] Update Progress: 878/2369 symbols completed. FTK took 22.3s. Estimated remaining: 2h 37m 44s. Rate: 9.5 symbols/min [2025-09-16T18:32:56.374914] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-08. [2025-09-16T18:32:57.280930] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T18:32:57.287425] [INFO ] [REMOTE ] FTI: Full fetch complete (57956 records). [2025-09-16T18:32:57.287531] [DEBUG] [VALIDATOR ] FTI: Validating primary data. [2025-09-16T18:32:57.688612] [DEBUG] [VALIDATOR ] FTI: All expected trading days are present. [2025-09-16T18:32:57.688737] [DEBUG] [VALIDATOR ] FTI: Primary data validation passed. [2025-09-16T18:32:57.689674] [INFO ] [MAIN ] FTI: Fetched primary data is valid. [2025-09-16T18:32:57.689727] [DEBUG] [LOCAL ] FTI: Saving primary data. [2025-09-16T18:32:57.691076] [DEBUG] [LOCAL ] FTI: Trimmed 57956 records to 39696 (keeping 150 days of data) [2025-09-16T18:32:57.957181] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T18:32:57.969442] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-09. [2025-09-16T18:32:57.980727] [DEBUG] [LOCAL ] FTI: Primary data saved (39696 records). [2025-09-16T18:32:57.980958] [INFO ] [MAIN ] FTI: Full history saved. [2025-09-16T18:32:57.981047] [INFO ] [MAIN ] Update Progress: 879/2369 symbols completed. FTI took 30.0s. Estimated remaining: 2h 37m 30s. Rate: 9.5 symbols/min [2025-09-16T18:32:59.466776] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:32:59.471691] [INFO ] [REMOTE ] FTS: Full fetch complete (45964 records). [2025-09-16T18:32:59.471824] [DEBUG] [VALIDATOR ] FTS: Validating primary data. [2025-09-16T18:32:59.845200] [DEBUG] [VALIDATOR ] FTS: All expected trading days are present. [2025-09-16T18:32:59.845283] [DEBUG] [VALIDATOR ] FTS: Primary data validation passed. [2025-09-16T18:32:59.845968] [INFO ] [MAIN ] FTS: Fetched primary data is valid. [2025-09-16T18:32:59.846015] [DEBUG] [LOCAL ] FTS: Saving primary data. [2025-09-16T18:32:59.847053] [DEBUG] [LOCAL ] FTS: Trimmed 45964 records to 31471 (keeping 150 days of data) [2025-09-16T18:33:00.045827] [DEBUG] [LOCAL ] FTS: Primary data saved (31471 records). [2025-09-16T18:33:00.046019] [INFO ] [MAIN ] FTS: Full history saved. [2025-09-16T18:33:00.046146] [INFO ] [MAIN ] Update Progress: 880/2369 symbols completed. FTS took 25.4s. Estimated remaining: 2h 37m 16s. Rate: 9.5 symbols/min [2025-09-16T18:33:00.624778] [INFO ] [MAIN ] FTV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:00.755353] [INFO ] [MAIN ] FUL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:00.775764] [INFO ] [MAIN ] FUBO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:00.778364] [INFO ] [MAIN ] FTWO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:00.778755] [DEBUG] [LOCAL ] FUL: No existing data file found in primary storage. [2025-09-16T18:33:00.779132] [WARN ] [MAIN ] FUL: No existing data found, performing full refresh instead. [2025-09-16T18:33:00.780059] [INFO ] [MAIN ] FUL: Wiping existing primary data before full refresh. [2025-09-16T18:33:00.780181] [INFO ] [LOCAL ] FUL: Starting deletion of primary data. [2025-09-16T18:33:00.780752] [DEBUG] [LOCAL ] FUL: No primary data file found to delete. [2025-09-16T18:33:00.780802] [INFO ] [MAIN ] FUL: Starting full primary refresh. [2025-09-16T18:33:00.813118] [DEBUG] [LOCAL ] FTWO: Loaded 1273 existing records from primary storage. [2025-09-16T18:33:00.828822] [INFO ] [MAIN ] FTWO: Loaded 1273 existing records. [2025-09-16T18:33:01.225908] [WARN ] [MAIN ] FTWO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:01.281217] [DEBUG] [LOCAL ] FUBO: Loaded 39688 existing records from primary storage. [2025-09-16T18:33:01.280285] [DEBUG] [LOCAL ] FTV: Loaded 39181 existing records from primary storage. [2025-09-16T18:33:01.281669] [INFO ] [MAIN ] FUBO: Loaded 39688 existing records. [2025-09-16T18:33:01.281599] [INFO ] [MAIN ] FTV: Loaded 39181 existing records. [2025-09-16T18:33:01.307102] [INFO ] [MAIN ] FTWO: Wiping existing primary data before full refresh. [2025-09-16T18:33:01.326190] [INFO ] [LOCAL ] FTWO: Starting deletion of primary data. [2025-09-16T18:33:01.437769] [DEBUG] [LOCAL ] FTWO: Deleted primary data file. [2025-09-16T18:33:01.496457] [INFO ] [MAIN ] FTWO: Starting full primary refresh. [2025-09-16T18:33:01.539316] [INFO ] [REMOTE ] FUL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:01.643574] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-02. [2025-09-16T18:33:01.861318] [WARN ] [MAIN ] FUBO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:01.861500] [WARN ] [MAIN ] FTV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:01.902545] [INFO ] [MAIN ] FUBO: Wiping existing primary data before full refresh. [2025-09-16T18:33:01.904702] [INFO ] [MAIN ] FTV: Wiping existing primary data before full refresh. [2025-09-16T18:33:01.910961] [INFO ] [LOCAL ] FTV: Starting deletion of primary data. [2025-09-16T18:33:01.911123] [INFO ] [LOCAL ] FUBO: Starting deletion of primary data. [2025-09-16T18:33:01.912237] [DEBUG] [LOCAL ] FTV: Deleted primary data file. [2025-09-16T18:33:01.941900] [INFO ] [MAIN ] FTV: Starting full primary refresh. [2025-09-16T18:33:01.962938] [DEBUG] [LOCAL ] FUBO: Deleted primary data file. [2025-09-16T18:33:01.991005] [INFO ] [MAIN ] FUBO: Starting full primary refresh. [2025-09-16T18:33:02.330973] [INFO ] [REMOTE ] FTWO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:02.376439] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-02. [2025-09-16T18:33:02.830967] [INFO ] [REMOTE ] FTV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:02.826224] [INFO ] [REMOTE ] FUBO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:02.831292] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-02. [2025-09-16T18:33:02.831521] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-02. [2025-09-16T18:33:03.172626] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-09-16T18:33:03.174016] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-03. [2025-09-16T18:33:03.374086] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T18:33:03.380420] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-03. [2025-09-16T18:33:03.454182] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-03 (0.28 seconds). [2025-09-16T18:33:03.455374] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-04. [2025-09-16T18:33:03.899830] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-09-16T18:33:03.901190] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-05. [2025-09-16T18:33:04.240032] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-05 (0.34 seconds). [2025-09-16T18:33:04.374866] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-06. [2025-09-16T18:33:04.470376] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:33:04.686296] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-09-16T18:33:04.828414] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T18:33:04.929141] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-02 (2.10 seconds). [2025-09-16T18:33:05.174916] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-07. [2025-09-16T18:33:05.541367] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-09-16T18:33:05.974919] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-03. [2025-09-16T18:33:06.774899] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-03. [2025-09-16T18:33:07.574911] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-04. [2025-09-16T18:33:07.927067] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-09-16T18:33:08.374897] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-04. [2025-09-16T18:33:08.641082] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-09-16T18:33:09.180842] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-08. [2025-09-16T18:33:09.194883] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-03 (2.42 seconds). [2025-09-16T18:33:09.642380] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-09-16T18:33:09.974938] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-04. [2025-09-16T18:33:10.065977] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-09-16T18:33:10.774911] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-05. [2025-09-16T18:33:11.574931] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-05. [2025-09-16T18:33:12.112290] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T18:33:12.178032] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-04 (2.20 seconds). [2025-09-16T18:33:12.374922] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-06. [2025-09-16T18:33:12.922511] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T18:33:13.174933] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-06. [2025-09-16T18:33:13.761727] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T18:33:13.974933] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-09. [2025-09-16T18:33:14.517240] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-09-16T18:33:14.530438] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T18:33:14.531597] [INFO ] [REMOTE ] FTWO: Full fetch complete (2006 records). [2025-09-16T18:33:14.531685] [DEBUG] [VALIDATOR ] FTWO: Validating primary data. [2025-09-16T18:33:14.532130] [FATAL] [VALIDATOR ] FTWO: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:33:14.532210] [FATAL] [MAIN ] FTWO: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:33:14.532247] [WARN ] [MAIN ] FTWO: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:33:14.532287] [INFO ] [LOCAL ] FTWO: Starting deletion of primary data. [2025-09-16T18:33:14.532344] [DEBUG] [LOCAL ] FTWO: No primary data file found to delete. [2025-09-16T18:33:14.532380] [INFO ] [LOCAL ] FTWO: Starting deletion of derivative data. [2025-09-16T18:33:14.532656] [DEBUG] [LOCAL ] FTWO: Deleted derivative data file. [2025-09-16T18:33:14.532743] [INFO ] [MAIN ] Update Progress: 881/2369 symbols completed. FTWO took 14.5s. Estimated remaining: 2h 37m 24s. Rate: 9.5 symbols/min [2025-09-16T18:33:14.707679] [INFO ] [MAIN ] FUN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:14.743826] [DEBUG] [LOCAL ] FUN: Loaded 38428 existing records from primary storage. [2025-09-16T18:33:14.744031] [INFO ] [MAIN ] FUN: Loaded 38428 existing records. [2025-09-16T18:33:14.780066] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-07. [2025-09-16T18:33:14.927023] [WARN ] [MAIN ] FUN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:14.927312] [INFO ] [MAIN ] FUN: Wiping existing primary data before full refresh. [2025-09-16T18:33:14.927370] [INFO ] [LOCAL ] FUN: Starting deletion of primary data. [2025-09-16T18:33:14.928214] [DEBUG] [LOCAL ] FUN: Deleted primary data file. [2025-09-16T18:33:14.928264] [INFO ] [MAIN ] FUN: Starting full primary refresh. [2025-09-16T18:33:15.277885] [INFO ] [REMOTE ] FUN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:15.574908] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-02. [2025-09-16T18:33:16.374895] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-05. [2025-09-16T18:33:16.762548] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-09-16T18:33:17.174906] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-08. [2025-09-16T18:33:17.236226] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T18:33:17.974917] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-07. [2025-09-16T18:33:18.601843] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-05 (2.23 seconds). [2025-09-16T18:33:18.700161] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T18:33:18.774867] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-09. [2025-09-16T18:33:19.350219] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-09-16T18:33:19.574908] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-03. [2025-09-16T18:33:20.374863] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-06. [2025-09-16T18:33:20.393139] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T18:33:20.399516] [INFO ] [REMOTE ] FTV: Full fetch complete (57504 records). [2025-09-16T18:33:20.399611] [DEBUG] [VALIDATOR ] FTV: Validating primary data. [2025-09-16T18:33:20.785915] [DEBUG] [VALIDATOR ] FTV: All expected trading days are present. [2025-09-16T18:33:20.785993] [DEBUG] [VALIDATOR ] FTV: Primary data validation passed. [2025-09-16T18:33:20.786762] [INFO ] [MAIN ] FTV: Fetched primary data is valid. [2025-09-16T18:33:20.786835] [DEBUG] [LOCAL ] FTV: Saving primary data. [2025-09-16T18:33:20.788065] [DEBUG] [LOCAL ] FTV: Trimmed 57504 records to 39565 (keeping 150 days of data) [2025-09-16T18:33:21.034566] [DEBUG] [LOCAL ] FTV: Primary data saved (39565 records). [2025-09-16T18:33:21.034722] [INFO ] [MAIN ] FTV: Full history saved. [2025-09-16T18:33:21.034807] [INFO ] [MAIN ] Update Progress: 882/2369 symbols completed. FTV took 21.0s. Estimated remaining: 2h 37m 18s. Rate: 9.5 symbols/min [2025-09-16T18:33:21.178841] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-08. [2025-09-16T18:33:21.210451] [INFO ] [MAIN ] FVR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:21.227309] [DEBUG] [LOCAL ] FVR: Loaded 16121 existing records from primary storage. [2025-09-16T18:33:21.227455] [INFO ] [MAIN ] FVR: Loaded 16121 existing records. [2025-09-16T18:33:21.405220] [WARN ] [MAIN ] FVR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:21.405679] [INFO ] [MAIN ] FVR: Wiping existing primary data before full refresh. [2025-09-16T18:33:21.405734] [INFO ] [LOCAL ] FVR: Starting deletion of primary data. [2025-09-16T18:33:21.415999] [DEBUG] [LOCAL ] FVR: Deleted primary data file. [2025-09-16T18:33:21.429879] [INFO ] [MAIN ] FVR: Starting full primary refresh. [2025-09-16T18:33:21.534606] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-09-16T18:33:21.805329] [INFO ] [REMOTE ] FVR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:21.974883] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-02. [2025-09-16T18:33:22.154996] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-09-16T18:33:22.336586] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-06 (1.96 seconds). [2025-09-16T18:33:22.774909] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-07. [2025-09-16T18:33:23.016456] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T18:33:23.574928] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-03. [2025-09-16T18:33:24.374871] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-09. [2025-09-16T18:33:24.699205] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-09-16T18:33:24.877929] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-09-16T18:33:25.174917] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-04. [2025-09-16T18:33:25.974903] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-08. [2025-09-16T18:33:26.038497] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-09-16T18:33:26.042986] [INFO ] [REMOTE ] FUL: Full fetch complete (35686 records). [2025-09-16T18:33:26.043078] [DEBUG] [VALIDATOR ] FUL: Validating primary data. [2025-09-16T18:33:26.423581] [DEBUG] [VALIDATOR ] FUL: All expected trading days are present. [2025-09-16T18:33:26.423682] [DEBUG] [VALIDATOR ] FUL: Primary data validation passed. [2025-09-16T18:33:26.424381] [INFO ] [MAIN ] FUL: Fetched primary data is valid. [2025-09-16T18:33:26.424437] [DEBUG] [LOCAL ] FUL: Saving primary data. [2025-09-16T18:33:26.426547] [DEBUG] [LOCAL ] FUL: Trimmed 35686 records to 22955 (keeping 150 days of data) [2025-09-16T18:33:26.533876] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-09-16T18:33:26.594854] [DEBUG] [LOCAL ] FUL: Primary data saved (22955 records). [2025-09-16T18:33:26.595015] [INFO ] [MAIN ] FUL: Full history saved. [2025-09-16T18:33:26.595115] [INFO ] [MAIN ] Update Progress: 883/2369 symbols completed. FUL took 26.5s. Estimated remaining: 2h 37m 10s. Rate: 9.5 symbols/min [2025-09-16T18:33:26.779479] [INFO ] [MAIN ] FVRR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:26.778045] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-05. [2025-09-16T18:33:26.808823] [DEBUG] [LOCAL ] FVRR: Loaded 29697 existing records from primary storage. [2025-09-16T18:33:26.808963] [INFO ] [MAIN ] FVRR: Loaded 29697 existing records. [2025-09-16T18:33:26.997550] [WARN ] [MAIN ] FVRR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:26.997855] [INFO ] [MAIN ] FVRR: Wiping existing primary data before full refresh. [2025-09-16T18:33:26.997909] [INFO ] [LOCAL ] FVRR: Starting deletion of primary data. [2025-09-16T18:33:26.998636] [DEBUG] [LOCAL ] FVRR: Deleted primary data file. [2025-09-16T18:33:26.998721] [INFO ] [MAIN ] FVRR: Starting full primary refresh. [2025-09-16T18:33:27.356284] [INFO ] [REMOTE ] FVRR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:27.574891] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-02. [2025-09-16T18:33:27.757288] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-09-16T18:33:28.102415] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-09-16T18:33:28.374898] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-09. [2025-09-16T18:33:29.174900] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-06. [2025-09-16T18:33:29.916362] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-02 (2.34 seconds). [2025-09-16T18:33:29.974852] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-03. [2025-09-16T18:33:30.139201] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-09-16T18:33:30.539946] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-09-16T18:33:30.546363] [INFO ] [REMOTE ] FUBO: Full fetch complete (58379 records). [2025-09-16T18:33:30.546467] [DEBUG] [VALIDATOR ] FUBO: Validating primary data. [2025-09-16T18:33:30.780074] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-07. [2025-09-16T18:33:30.933812] [DEBUG] [VALIDATOR ] FUBO: All expected trading days are present. [2025-09-16T18:33:30.933892] [DEBUG] [VALIDATOR ] FUBO: Primary data validation passed. [2025-09-16T18:33:30.935143] [INFO ] [MAIN ] FUBO: Fetched primary data is valid. [2025-09-16T18:33:30.935193] [DEBUG] [LOCAL ] FUBO: Saving primary data. [2025-09-16T18:33:30.936435] [DEBUG] [LOCAL ] FUBO: Trimmed 58379 records to 40078 (keeping 150 days of data) [2025-09-16T18:33:31.180133] [DEBUG] [LOCAL ] FUBO: Primary data saved (40078 records). [2025-09-16T18:33:31.180279] [INFO ] [MAIN ] FUBO: Full history saved. [2025-09-16T18:33:31.182259] [INFO ] [MAIN ] Update Progress: 884/2369 symbols completed. FUBO took 31.1s. Estimated remaining: 2h 37m 1s. Rate: 9.5 symbols/min [2025-09-16T18:33:31.367488] [INFO ] [MAIN ] FXED: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:31.370457] [DEBUG] [LOCAL ] FXED: Loaded 868 existing records from primary storage. [2025-09-16T18:33:31.370629] [INFO ] [MAIN ] FXED: Loaded 868 existing records. [2025-09-16T18:33:31.581762] [WARN ] [MAIN ] FXED: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:31.576273] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-04. [2025-09-16T18:33:31.589393] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T18:33:31.589725] [INFO ] [MAIN ] FXED: Wiping existing primary data before full refresh. [2025-09-16T18:33:31.598868] [INFO ] [LOCAL ] FXED: Starting deletion of primary data. [2025-09-16T18:33:31.599509] [DEBUG] [LOCAL ] FXED: Deleted primary data file. [2025-09-16T18:33:31.599599] [INFO ] [MAIN ] FXED: Starting full primary refresh. [2025-09-16T18:33:31.975136] [INFO ] [REMOTE ] FXED: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:32.010603] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-09-16T18:33:32.374900] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-04. [2025-09-16T18:33:33.174910] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-08. [2025-09-16T18:33:33.305162] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T18:33:33.426953] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-09-16T18:33:33.979975] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-05. [2025-09-16T18:33:33.987962] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T18:33:34.774890] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-09. [2025-09-16T18:33:35.067565] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-09-16T18:33:35.574875] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-06. [2025-09-16T18:33:35.677913] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-09-16T18:33:35.680329] [INFO ] [REMOTE ] FVR: Full fetch complete (23269 records). [2025-09-16T18:33:35.680407] [DEBUG] [VALIDATOR ] FVR: Validating primary data. [2025-09-16T18:33:35.680976] [FATAL] [VALIDATOR ] FVR: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:33:35.681074] [FATAL] [MAIN ] FVR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:33:35.681235] [WARN ] [MAIN ] FVR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:33:35.681294] [INFO ] [LOCAL ] FVR: Starting deletion of primary data. [2025-09-16T18:33:35.681353] [DEBUG] [LOCAL ] FVR: No primary data file found to delete. [2025-09-16T18:33:35.681390] [INFO ] [LOCAL ] FVR: Starting deletion of derivative data. [2025-09-16T18:33:35.681624] [DEBUG] [LOCAL ] FVR: Deleted derivative data file. [2025-09-16T18:33:35.681737] [INFO ] [MAIN ] Update Progress: 885/2369 symbols completed. FVR took 14.6s. Estimated remaining: 2h 36m 51s. Rate: 9.5 symbols/min [2025-09-16T18:33:35.858275] [INFO ] [MAIN ] G: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:35.894668] [DEBUG] [LOCAL ] G: Loaded 36743 existing records from primary storage. [2025-09-16T18:33:35.894811] [INFO ] [MAIN ] G: Loaded 36743 existing records. [2025-09-16T18:33:36.076179] [WARN ] [MAIN ] G: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:36.076402] [INFO ] [MAIN ] G: Wiping existing primary data before full refresh. [2025-09-16T18:33:36.076446] [INFO ] [LOCAL ] G: Starting deletion of primary data. [2025-09-16T18:33:36.077219] [DEBUG] [LOCAL ] G: Deleted primary data file. [2025-09-16T18:33:36.077268] [INFO ] [MAIN ] G: Starting full primary refresh. [2025-09-16T18:33:36.380040] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-05. [2025-09-16T18:33:36.443816] [INFO ] [REMOTE ] G: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:36.639943] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T18:33:37.174953] [INFO ] [REMOTE ] G: Fetching primary data for 2025-02. [2025-09-16T18:33:37.776852] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T18:33:37.974956] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-02. [2025-09-16T18:33:38.395632] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-09-16T18:33:38.468659] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T18:33:38.774917] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-06. [2025-09-16T18:33:39.574907] [INFO ] [REMOTE ] G: Fetching primary data for 2025-03. [2025-09-16T18:33:40.073958] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-09-16T18:33:40.374927] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-07. [2025-09-16T18:33:40.974994] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T18:33:41.174912] [INFO ] [REMOTE ] G: Fetching primary data for 2025-04. [2025-09-16T18:33:41.743287] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T18:33:41.974905] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-07. [2025-09-16T18:33:42.382178] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-09-16T18:33:42.774945] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-03. [2025-09-16T18:33:43.055445] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-03 (0.28 seconds). [2025-09-16T18:33:43.574919] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-04. [2025-09-16T18:33:43.659581] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-09-16T18:33:43.959907] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-09-16T18:33:44.374920] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-08. [2025-09-16T18:33:45.174945] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-08. [2025-09-16T18:33:45.974896] [INFO ] [REMOTE ] G: Fetching primary data for 2025-05. [2025-09-16T18:33:46.034802] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T18:33:46.774937] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-09. [2025-09-16T18:33:46.956957] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-09-16T18:33:47.574903] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-09. [2025-09-16T18:33:47.794851] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-09-16T18:33:48.338414] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T18:33:48.343562] [INFO ] [REMOTE ] FVRR: Full fetch complete (44023 records). [2025-09-16T18:33:48.343753] [DEBUG] [VALIDATOR ] FVRR: Validating primary data. [2025-09-16T18:33:48.380115] [INFO ] [REMOTE ] G: Fetching primary data for 2025-06. [2025-09-16T18:33:48.726848] [DEBUG] [VALIDATOR ] FVRR: All expected trading days are present. [2025-09-16T18:33:48.726931] [DEBUG] [VALIDATOR ] FVRR: Primary data validation passed. [2025-09-16T18:33:48.727585] [INFO ] [MAIN ] FVRR: Fetched primary data is valid. [2025-09-16T18:33:48.727671] [DEBUG] [LOCAL ] FVRR: Saving primary data. [2025-09-16T18:33:48.728625] [DEBUG] [LOCAL ] FVRR: Trimmed 44023 records to 29987 (keeping 150 days of data) [2025-09-16T18:33:48.907792] [DEBUG] [LOCAL ] FVRR: Primary data saved (29987 records). [2025-09-16T18:33:48.908034] [INFO ] [MAIN ] FVRR: Full history saved. [2025-09-16T18:33:48.908161] [INFO ] [MAIN ] Update Progress: 886/2369 symbols completed. FVRR took 22.3s. Estimated remaining: 2h 36m 57s. Rate: 9.4 symbols/min [2025-09-16T18:33:49.077335] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:33:49.105982] [INFO ] [MAIN ] GAB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:49.108978] [INFO ] [REMOTE ] FUN: Full fetch complete (55665 records). [2025-09-16T18:33:49.109181] [DEBUG] [VALIDATOR ] FUN: Validating primary data. [2025-09-16T18:33:49.180430] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-05. [2025-09-16T18:33:49.360011] [DEBUG] [LOCAL ] GAB: Loaded 19323 existing records from primary storage. [2025-09-16T18:33:49.392697] [INFO ] [MAIN ] GAB: Loaded 19323 existing records. [2025-09-16T18:33:49.686345] [DEBUG] [VALIDATOR ] FUN: All expected trading days are present. [2025-09-16T18:33:49.686457] [WARN ] [MAIN ] GAB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:49.686679] [DEBUG] [VALIDATOR ] FUN: Primary data validation passed. [2025-09-16T18:33:49.687527] [INFO ] [MAIN ] FUN: Fetched primary data is valid. [2025-09-16T18:33:49.686714] [INFO ] [MAIN ] GAB: Wiping existing primary data before full refresh. [2025-09-16T18:33:49.687613] [DEBUG] [LOCAL ] FUN: Saving primary data. [2025-09-16T18:33:49.687763] [INFO ] [LOCAL ] GAB: Starting deletion of primary data. [2025-09-16T18:33:49.688250] [DEBUG] [LOCAL ] GAB: Deleted primary data file. [2025-09-16T18:33:49.688333] [INFO ] [MAIN ] GAB: Starting full primary refresh. [2025-09-16T18:33:49.732962] [DEBUG] [LOCAL ] FUN: Trimmed 55665 records to 38812 (keeping 150 days of data) [2025-09-16T18:33:49.901416] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-09-16T18:33:49.977151] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-06. [2025-09-16T18:33:50.151511] [INFO ] [REMOTE ] GAB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:50.185963] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-09-16T18:33:50.322582] [DEBUG] [LOCAL ] FUN: Primary data saved (38812 records). [2025-09-16T18:33:50.322757] [INFO ] [MAIN ] FUN: Full history saved. [2025-09-16T18:33:50.322879] [INFO ] [MAIN ] Update Progress: 887/2369 symbols completed. FUN took 35.8s. Estimated remaining: 2h 36m 42s. Rate: 9.5 symbols/min [2025-09-16T18:33:50.406043] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-09-16T18:33:50.495385] [INFO ] [MAIN ] GAM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:50.499306] [DEBUG] [LOCAL ] GAM: Loaded 2775 existing records from primary storage. [2025-09-16T18:33:50.499377] [INFO ] [MAIN ] GAM: Loaded 2775 existing records. [2025-09-16T18:33:50.685292] [WARN ] [MAIN ] GAM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:50.685518] [INFO ] [MAIN ] GAM: Wiping existing primary data before full refresh. [2025-09-16T18:33:50.685576] [INFO ] [LOCAL ] GAM: Starting deletion of primary data. [2025-09-16T18:33:50.685812] [DEBUG] [LOCAL ] GAM: Deleted primary data file. [2025-09-16T18:33:50.685860] [INFO ] [MAIN ] GAM: Starting full primary refresh. [2025-09-16T18:33:50.782857] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-07. [2025-09-16T18:33:51.037862] [INFO ] [REMOTE ] GAM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:51.084696] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-07 (0.30 seconds). [2025-09-16T18:33:51.574944] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-02. [2025-09-16T18:33:52.374933] [INFO ] [REMOTE ] G: Fetching primary data for 2025-07. [2025-09-16T18:33:52.499759] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-09-16T18:33:53.174921] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-08. [2025-09-16T18:33:53.582154] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-09-16T18:33:53.842218] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-09-16T18:33:53.974934] [INFO ] [REMOTE ] G: Fetching primary data for 2025-08. [2025-09-16T18:33:54.774896] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-09. [2025-09-16T18:33:55.234578] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-09-16T18:33:55.235316] [INFO ] [REMOTE ] FXED: Full fetch complete (1274 records). [2025-09-16T18:33:55.235390] [DEBUG] [VALIDATOR ] FXED: Validating primary data. [2025-09-16T18:33:55.235761] [FATAL] [VALIDATOR ] FXED: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:33:55.235838] [FATAL] [MAIN ] FXED: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:33:55.235876] [WARN ] [MAIN ] FXED: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:33:55.235920] [INFO ] [LOCAL ] FXED: Starting deletion of primary data. [2025-09-16T18:33:55.235988] [DEBUG] [LOCAL ] FXED: No primary data file found to delete. [2025-09-16T18:33:55.236027] [INFO ] [LOCAL ] FXED: Starting deletion of derivative data. [2025-09-16T18:33:55.236376] [DEBUG] [LOCAL ] FXED: Deleted derivative data file. [2025-09-16T18:33:55.236493] [INFO ] [MAIN ] Update Progress: 888/2369 symbols completed. FXED took 24.1s. Estimated remaining: 2h 36m 33s. Rate: 9.5 symbols/min [2025-09-16T18:33:55.419004] [INFO ] [MAIN ] GAP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:33:55.455253] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T18:33:55.507716] [DEBUG] [LOCAL ] GAP: Loaded 39617 existing records from primary storage. [2025-09-16T18:33:55.507941] [INFO ] [MAIN ] GAP: Loaded 39617 existing records. [2025-09-16T18:33:55.580007] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-02. [2025-09-16T18:33:55.691475] [WARN ] [MAIN ] GAP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:33:55.691799] [INFO ] [MAIN ] GAP: Wiping existing primary data before full refresh. [2025-09-16T18:33:55.691865] [INFO ] [LOCAL ] GAP: Starting deletion of primary data. [2025-09-16T18:33:55.692931] [DEBUG] [LOCAL ] GAP: Deleted primary data file. [2025-09-16T18:33:55.692983] [INFO ] [MAIN ] GAP: Starting full primary refresh. [2025-09-16T18:33:56.055218] [INFO ] [REMOTE ] GAP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:33:56.374899] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-02. [2025-09-16T18:33:56.681006] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T18:33:57.174900] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-03. [2025-09-16T18:33:57.643484] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T18:33:57.974910] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-03. [2025-09-16T18:33:58.068017] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-09-16T18:33:58.774936] [INFO ] [REMOTE ] G: Fetching primary data for 2025-09. [2025-09-16T18:33:59.574935] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-03. [2025-09-16T18:34:00.046823] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-09-16T18:34:00.148983] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T18:34:00.155615] [INFO ] [REMOTE ] G: Full fetch complete (54192 records). [2025-09-16T18:34:00.155750] [DEBUG] [VALIDATOR ] G: Validating primary data. [2025-09-16T18:34:00.380071] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-04. [2025-09-16T18:34:00.451675] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-09-16T18:34:00.553426] [DEBUG] [VALIDATOR ] G: All expected trading days are present. [2025-09-16T18:34:00.553502] [DEBUG] [VALIDATOR ] G: Primary data validation passed. [2025-09-16T18:34:00.554272] [INFO ] [MAIN ] G: Fetched primary data is valid. [2025-09-16T18:34:00.554320] [DEBUG] [LOCAL ] G: Saving primary data. [2025-09-16T18:34:00.555432] [DEBUG] [LOCAL ] G: Trimmed 54192 records to 37121 (keeping 150 days of data) [2025-09-16T18:34:00.789529] [DEBUG] [LOCAL ] G: Primary data saved (37121 records). [2025-09-16T18:34:00.789695] [INFO ] [MAIN ] G: Full history saved. [2025-09-16T18:34:00.791066] [INFO ] [MAIN ] Update Progress: 889/2369 symbols completed. G took 25.1s. Estimated remaining: 2h 36m 26s. Rate: 9.5 symbols/min [2025-09-16T18:34:00.965805] [INFO ] [MAIN ] GATX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:34:00.966021] [DEBUG] [LOCAL ] GATX: No existing data file found in primary storage. [2025-09-16T18:34:00.966062] [WARN ] [MAIN ] GATX: No existing data found, performing full refresh instead. [2025-09-16T18:34:00.966192] [INFO ] [MAIN ] GATX: Wiping existing primary data before full refresh. [2025-09-16T18:34:00.966253] [INFO ] [LOCAL ] GATX: Starting deletion of primary data. [2025-09-16T18:34:00.966317] [DEBUG] [LOCAL ] GATX: No primary data file found to delete. [2025-09-16T18:34:00.966366] [INFO ] [MAIN ] GATX: Starting full primary refresh. [2025-09-16T18:34:01.180004] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-04. [2025-09-16T18:34:01.334486] [INFO ] [REMOTE ] GATX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:34:01.974938] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-04. [2025-09-16T18:34:02.144864] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T18:34:02.774946] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-05. [2025-09-16T18:34:03.350383] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-09-16T18:34:03.574889] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-05. [2025-09-16T18:34:03.946417] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-04 (2.77 seconds). [2025-09-16T18:34:03.972927] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T18:34:03.999085] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T18:34:04.374918] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-06. [2025-09-16T18:34:05.174911] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-05. [2025-09-16T18:34:05.241785] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-09-16T18:34:05.974913] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-07. [2025-09-16T18:34:06.774924] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-02. [2025-09-16T18:34:06.800849] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-09-16T18:34:07.059271] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-09-16T18:34:07.574931] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-06. [2025-09-16T18:34:07.786078] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-09-16T18:34:08.374905] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-06. [2025-09-16T18:34:09.174901] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-03. [2025-09-16T18:34:09.667250] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-06 (2.09 seconds). [2025-09-16T18:34:09.754538] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T18:34:09.974899] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-07. [2025-09-16T18:34:10.146287] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-09-16T18:34:10.774905] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-07. [2025-09-16T18:34:11.574904] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-08. [2025-09-16T18:34:11.909823] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-09-16T18:34:12.310334] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-09-16T18:34:12.310551] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-09-16T18:34:12.374907] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-08. [2025-09-16T18:34:13.174938] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-08. [2025-09-16T18:34:13.974923] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-09. [2025-09-16T18:34:14.199866] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T18:34:14.226823] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T18:34:14.517265] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-09-16T18:34:14.518428] [INFO ] [REMOTE ] GAM: Full fetch complete (4947 records). [2025-09-16T18:34:14.518501] [DEBUG] [VALIDATOR ] GAM: Validating primary data. [2025-09-16T18:34:14.780000] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-09. [2025-09-16T18:34:14.880003] [DEBUG] [VALIDATOR ] GAM: All expected trading days are present. [2025-09-16T18:34:14.880086] [DEBUG] [VALIDATOR ] GAM: Primary data validation passed. [2025-09-16T18:34:14.880242] [INFO ] [MAIN ] GAM: Fetched primary data is valid. [2025-09-16T18:34:14.880285] [DEBUG] [LOCAL ] GAM: Saving primary data. [2025-09-16T18:34:14.880681] [DEBUG] [LOCAL ] GAM: Trimmed 4947 records to 2806 (keeping 150 days of data) [2025-09-16T18:34:14.897928] [DEBUG] [LOCAL ] GAM: Primary data saved (2806 records). [2025-09-16T18:34:14.898051] [INFO ] [MAIN ] GAM: Full history saved. [2025-09-16T18:34:14.898141] [INFO ] [MAIN ] Update Progress: 890/2369 symbols completed. GAM took 24.6s. Estimated remaining: 2h 36m 32s. Rate: 9.4 symbols/min [2025-09-16T18:34:15.069968] [INFO ] [MAIN ] GBAB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:34:15.080275] [DEBUG] [LOCAL ] GBAB: Loaded 9535 existing records from primary storage. [2025-09-16T18:34:15.080383] [INFO ] [MAIN ] GBAB: Loaded 9535 existing records. [2025-09-16T18:34:15.254668] [WARN ] [MAIN ] GBAB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:34:15.254926] [INFO ] [MAIN ] GBAB: Wiping existing primary data before full refresh. [2025-09-16T18:34:15.254973] [INFO ] [LOCAL ] GBAB: Starting deletion of primary data. [2025-09-16T18:34:15.255310] [DEBUG] [LOCAL ] GBAB: Deleted primary data file. [2025-09-16T18:34:15.255354] [INFO ] [MAIN ] GBAB: Starting full primary refresh. [2025-09-16T18:34:15.582434] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-09. [2025-09-16T18:34:15.624953] [INFO ] [REMOTE ] GBAB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:34:16.374937] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-02. [2025-09-16T18:34:16.567054] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-09-16T18:34:16.573758] [INFO ] [REMOTE ] GAP: Full fetch complete (58330 records). [2025-09-16T18:34:16.573921] [DEBUG] [VALIDATOR ] GAP: Validating primary data. [2025-09-16T18:34:16.859193] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-09-16T18:34:16.897927] [INFO ] [REMOTE ] GAB: Full fetch complete (29499 records). [2025-09-16T18:34:16.898500] [DEBUG] [VALIDATOR ] GAB: Validating primary data. [2025-09-16T18:34:17.143395] [DEBUG] [VALIDATOR ] GAP: All expected trading days are present. [2025-09-16T18:34:17.148628] [DEBUG] [VALIDATOR ] GAP: Primary data validation passed. [2025-09-16T18:34:17.154744] [INFO ] [MAIN ] GAP: Fetched primary data is valid. [2025-09-16T18:34:17.180435] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-04. [2025-09-16T18:34:17.170808] [DEBUG] [LOCAL ] GAP: Saving primary data. [2025-09-16T18:34:17.321434] [DEBUG] [LOCAL ] GAP: Trimmed 58330 records to 40007 (keeping 150 days of data) [2025-09-16T18:34:17.506043] [DEBUG] [VALIDATOR ] GAB: All expected trading days are present. [2025-09-16T18:34:17.506748] [DEBUG] [VALIDATOR ] GAB: Primary data validation passed. [2025-09-16T18:34:17.507921] [INFO ] [MAIN ] GAB: Fetched primary data is valid. [2025-09-16T18:34:17.508227] [DEBUG] [LOCAL ] GAB: Saving primary data. [2025-09-16T18:34:17.511475] [DEBUG] [LOCAL ] GAB: Trimmed 29499 records to 19527 (keeping 150 days of data) [2025-09-16T18:34:17.641307] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T18:34:17.763022] [DEBUG] [LOCAL ] GAB: Primary data saved (19527 records). [2025-09-16T18:34:17.764887] [INFO ] [MAIN ] GAB: Full history saved. [2025-09-16T18:34:17.765019] [INFO ] [MAIN ] Update Progress: 891/2369 symbols completed. GAB took 28.9s. Estimated remaining: 2h 36m 20s. Rate: 9.5 symbols/min [2025-09-16T18:34:17.960691] [INFO ] [MAIN ] GBCI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:34:17.962239] [DEBUG] [LOCAL ] GBCI: No existing data file found in primary storage. [2025-09-16T18:34:17.962531] [WARN ] [MAIN ] GBCI: No existing data found, performing full refresh instead. [2025-09-16T18:34:17.963769] [INFO ] [MAIN ] GBCI: Wiping existing primary data before full refresh. [2025-09-16T18:34:17.964867] [INFO ] [LOCAL ] GBCI: Starting deletion of primary data. [2025-09-16T18:34:17.965443] [DEBUG] [LOCAL ] GBCI: No primary data file found to delete. [2025-09-16T18:34:17.965807] [INFO ] [MAIN ] GBCI: Starting full primary refresh. [2025-09-16T18:34:17.980017] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-03. [2025-09-16T18:34:18.405685] [DEBUG] [LOCAL ] GAP: Primary data saved (40007 records). [2025-09-16T18:34:18.421095] [INFO ] [MAIN ] GAP: Full history saved. [2025-09-16T18:34:18.424465] [INFO ] [MAIN ] Update Progress: 892/2369 symbols completed. GAP took 23.2s. Estimated remaining: 2h 36m 4s. Rate: 9.5 symbols/min [2025-09-16T18:34:18.424346] [INFO ] [REMOTE ] GBCI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:34:18.599462] [INFO ] [MAIN ] GBLI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:34:18.599855] [DEBUG] [LOCAL ] GBLI: No existing data file found in primary storage. [2025-09-16T18:34:18.600003] [WARN ] [MAIN ] GBLI: No existing data found, performing full refresh instead. [2025-09-16T18:34:18.608837] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-09-16T18:34:18.612247] [INFO ] [MAIN ] GBLI: Wiping existing primary data before full refresh. [2025-09-16T18:34:18.612408] [INFO ] [LOCAL ] GBLI: Starting deletion of primary data. [2025-09-16T18:34:18.612489] [DEBUG] [LOCAL ] GBLI: No primary data file found to delete. [2025-09-16T18:34:18.612527] [INFO ] [MAIN ] GBLI: Starting full primary refresh. [2025-09-16T18:34:18.729005] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-09-16T18:34:18.782962] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-05. [2025-09-16T18:34:18.986163] [INFO ] [REMOTE ] GBLI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:34:19.574910] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-02. [2025-09-16T18:34:19.923199] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T18:34:20.374929] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-04. [2025-09-16T18:34:21.154911] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T18:34:21.174843] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-03. [2025-09-16T18:34:21.312097] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-09-16T18:34:21.974901] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-02. [2025-09-16T18:34:22.666194] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T18:34:22.774863] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-06. [2025-09-16T18:34:22.842277] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T18:34:23.574917] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-05. [2025-09-16T18:34:23.798548] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-09-16T18:34:24.231025] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-09-16T18:34:24.374892] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-04. [2025-09-16T18:34:25.174897] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-03. [2025-09-16T18:34:25.541388] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-09-16T18:34:25.974907] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-07. [2025-09-16T18:34:26.001199] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-09-16T18:34:26.774916] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-04. [2025-09-16T18:34:26.829918] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-09-16T18:34:27.125451] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-09-16T18:34:27.574909] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-06. [2025-09-16T18:34:28.374906] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-08. [2025-09-16T18:34:28.644780] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T18:34:29.174931] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-05. [2025-09-16T18:34:29.451796] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T18:34:29.974899] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-07. [2025-09-16T18:34:30.412925] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-09-16T18:34:30.774907] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-09. [2025-09-16T18:34:30.824701] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-09-16T18:34:31.574882] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-06. [2025-09-16T18:34:31.819556] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-09-16T18:34:31.822414] [INFO ] [REMOTE ] GATX: Full fetch complete (19052 records). [2025-09-16T18:34:31.822491] [DEBUG] [VALIDATOR ] GATX: Validating primary data. [2025-09-16T18:34:32.211209] [DEBUG] [VALIDATOR ] GATX: All expected trading days are present. [2025-09-16T18:34:32.211292] [DEBUG] [VALIDATOR ] GATX: Primary data validation passed. [2025-09-16T18:34:32.211767] [INFO ] [MAIN ] GATX: Fetched primary data is valid. [2025-09-16T18:34:32.211813] [DEBUG] [LOCAL ] GATX: Saving primary data. [2025-09-16T18:34:32.212385] [DEBUG] [LOCAL ] GATX: Trimmed 19052 records to 13206 (keeping 150 days of data) [2025-09-16T18:34:32.303108] [DEBUG] [LOCAL ] GATX: Primary data saved (13206 records). [2025-09-16T18:34:32.303260] [INFO ] [MAIN ] GATX: Full history saved. [2025-09-16T18:34:32.303337] [INFO ] [MAIN ] Update Progress: 893/2369 symbols completed. GATX took 31.5s. Estimated remaining: 2h 36m 10s. Rate: 9.5 symbols/min [2025-09-16T18:34:32.384830] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-08. [2025-09-16T18:34:32.486685] [INFO ] [MAIN ] GBTG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:34:32.517034] [DEBUG] [LOCAL ] GBTG: Loaded 29741 existing records from primary storage. [2025-09-16T18:34:32.517244] [INFO ] [MAIN ] GBTG: Loaded 29741 existing records. [2025-09-16T18:34:32.693135] [WARN ] [MAIN ] GBTG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:34:32.693429] [INFO ] [MAIN ] GBTG: Wiping existing primary data before full refresh. [2025-09-16T18:34:32.693616] [INFO ] [LOCAL ] GBTG: Starting deletion of primary data. [2025-09-16T18:34:32.694337] [DEBUG] [LOCAL ] GBTG: Deleted primary data file. [2025-09-16T18:34:32.694389] [INFO ] [MAIN ] GBTG: Starting full primary refresh. [2025-09-16T18:34:32.869781] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T18:34:33.083602] [INFO ] [REMOTE ] GBTG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:34:33.174919] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-07. [2025-09-16T18:34:33.223997] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-09-16T18:34:33.974891] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-09. [2025-09-16T18:34:34.661936] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T18:34:34.774873] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-08. [2025-09-16T18:34:34.873441] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-09-16T18:34:34.875475] [INFO ] [REMOTE ] GBAB: Full fetch complete (14440 records). [2025-09-16T18:34:34.875545] [DEBUG] [VALIDATOR ] GBAB: Validating primary data. [2025-09-16T18:34:35.246081] [DEBUG] [VALIDATOR ] GBAB: All expected trading days are present. [2025-09-16T18:34:35.246216] [DEBUG] [VALIDATOR ] GBAB: Primary data validation passed. [2025-09-16T18:34:35.246576] [INFO ] [MAIN ] GBAB: Fetched primary data is valid. [2025-09-16T18:34:35.246618] [DEBUG] [LOCAL ] GBAB: Saving primary data. [2025-09-16T18:34:35.247136] [DEBUG] [LOCAL ] GBAB: Trimmed 14440 records to 9666 (keeping 150 days of data) [2025-09-16T18:34:35.306220] [DEBUG] [LOCAL ] GBAB: Primary data saved (9666 records). [2025-09-16T18:34:35.306351] [INFO ] [MAIN ] GBAB: Full history saved. [2025-09-16T18:34:35.306438] [INFO ] [MAIN ] Update Progress: 894/2369 symbols completed. GBAB took 20.4s. Estimated remaining: 2h 35m 59s. Rate: 9.5 symbols/min [2025-09-16T18:34:35.481631] [INFO ] [MAIN ] GBX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:34:35.501174] [DEBUG] [LOCAL ] GBX: Loaded 20311 existing records from primary storage. [2025-09-16T18:34:35.501317] [INFO ] [MAIN ] GBX: Loaded 20311 existing records. [2025-09-16T18:34:35.580017] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-02. [2025-09-16T18:34:35.681251] [WARN ] [MAIN ] GBX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:34:35.681480] [INFO ] [MAIN ] GBX: Wiping existing primary data before full refresh. [2025-09-16T18:34:35.681533] [INFO ] [LOCAL ] GBX: Starting deletion of primary data. [2025-09-16T18:34:35.682125] [DEBUG] [LOCAL ] GBX: Deleted primary data file. [2025-09-16T18:34:35.682178] [INFO ] [MAIN ] GBX: Starting full primary refresh. [2025-09-16T18:34:36.034296] [INFO ] [REMOTE ] GBX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:34:36.197426] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T18:34:36.374896] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-09. [2025-09-16T18:34:36.942426] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T18:34:37.174911] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-05. [2025-09-16T18:34:37.898600] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-09-16T18:34:37.910992] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T18:34:37.915535] [INFO ] [REMOTE ] GBCI: Full fetch complete (43482 records). [2025-09-16T18:34:37.915619] [DEBUG] [VALIDATOR ] GBCI: Validating primary data. [2025-09-16T18:34:37.980052] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-06. [2025-09-16T18:34:38.296244] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-06 (0.32 seconds). [2025-09-16T18:34:38.296436] [DEBUG] [VALIDATOR ] GBCI: All expected trading days are present. [2025-09-16T18:34:38.297350] [DEBUG] [VALIDATOR ] GBCI: Primary data validation passed. [2025-09-16T18:34:38.298079] [INFO ] [MAIN ] GBCI: Fetched primary data is valid. [2025-09-16T18:34:38.298135] [DEBUG] [LOCAL ] GBCI: Saving primary data. [2025-09-16T18:34:38.298989] [DEBUG] [LOCAL ] GBCI: Trimmed 43482 records to 29687 (keeping 150 days of data) [2025-09-16T18:34:38.476970] [DEBUG] [LOCAL ] GBCI: Primary data saved (29687 records). [2025-09-16T18:34:38.477109] [INFO ] [MAIN ] GBCI: Full history saved. [2025-09-16T18:34:38.477185] [INFO ] [MAIN ] Update Progress: 895/2369 symbols completed. GBCI took 20.7s. Estimated remaining: 2h 35m 47s. Rate: 9.5 symbols/min [2025-09-16T18:34:38.650413] [INFO ] [MAIN ] GCI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:34:38.681955] [DEBUG] [LOCAL ] GCI: Loaded 32264 existing records from primary storage. [2025-09-16T18:34:38.682092] [INFO ] [MAIN ] GCI: Loaded 32264 existing records. [2025-09-16T18:34:38.780578] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-07. [2025-09-16T18:34:38.861268] [WARN ] [MAIN ] GCI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:34:38.861513] [INFO ] [MAIN ] GCI: Wiping existing primary data before full refresh. [2025-09-16T18:34:38.861560] [INFO ] [LOCAL ] GCI: Starting deletion of primary data. [2025-09-16T18:34:38.862166] [DEBUG] [LOCAL ] GCI: Deleted primary data file. [2025-09-16T18:34:38.862215] [INFO ] [MAIN ] GCI: Starting full primary refresh. [2025-09-16T18:34:39.117996] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-09-16T18:34:39.222298] [INFO ] [REMOTE ] GCI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:34:39.574901] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-03. [2025-09-16T18:34:40.374903] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-02. [2025-09-16T18:34:41.038127] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T18:34:41.174857] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-04. [2025-09-16T18:34:41.974911] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-02. [2025-09-16T18:34:42.053264] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T18:34:42.774883] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-03. [2025-09-16T18:34:43.559620] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-04 (2.38 seconds). [2025-09-16T18:34:43.574841] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-05. [2025-09-16T18:34:43.850883] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-09-16T18:34:44.165312] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T18:34:44.374921] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-08. [2025-09-16T18:34:44.845350] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-09-16T18:34:44.967244] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T18:34:45.174913] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-04. [2025-09-16T18:34:45.974924] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-03. [2025-09-16T18:34:46.304031] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-09-16T18:34:46.774908] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-05. [2025-09-16T18:34:46.852122] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-09-16T18:34:47.574917] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-04. [2025-09-16T18:34:47.999472] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T18:34:48.374970] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-09. [2025-09-16T18:34:48.824003] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-09-16T18:34:48.846626] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-09-16T18:34:48.847513] [INFO ] [REMOTE ] GBLI: Full fetch complete (1050 records). [2025-09-16T18:34:48.847588] [DEBUG] [VALIDATOR ] GBLI: Validating primary data. [2025-09-16T18:34:48.848032] [FATAL] [VALIDATOR ] GBLI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:34:48.848140] [FATAL] [MAIN ] GBLI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:34:48.848181] [WARN ] [MAIN ] GBLI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:34:48.848222] [INFO ] [LOCAL ] GBLI: Starting deletion of primary data. [2025-09-16T18:34:48.848283] [DEBUG] [LOCAL ] GBLI: No primary data file found to delete. [2025-09-16T18:34:48.848325] [INFO ] [LOCAL ] GBLI: Starting deletion of derivative data. [2025-09-16T18:34:48.848403] [DEBUG] [LOCAL ] GBLI: No derivative data file found to delete. [2025-09-16T18:34:48.848522] [INFO ] [MAIN ] Update Progress: 896/2369 symbols completed. GBLI took 30.4s. Estimated remaining: 2h 35m 47s. Rate: 9.5 symbols/min [2025-09-16T18:34:49.064228] [INFO ] [MAIN ] GCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:34:49.083335] [DEBUG] [LOCAL ] GCO: Loaded 17261 existing records from primary storage. [2025-09-16T18:34:49.083487] [INFO ] [MAIN ] GCO: Loaded 17261 existing records. [2025-09-16T18:34:49.180132] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-06. [2025-09-16T18:34:49.275191] [WARN ] [MAIN ] GCO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:34:49.275438] [INFO ] [MAIN ] GCO: Wiping existing primary data before full refresh. [2025-09-16T18:34:49.275486] [INFO ] [LOCAL ] GCO: Starting deletion of primary data. [2025-09-16T18:34:49.275951] [DEBUG] [LOCAL ] GCO: Deleted primary data file. [2025-09-16T18:34:49.276001] [INFO ] [MAIN ] GCO: Starting full primary refresh. [2025-09-16T18:34:49.633588] [INFO ] [REMOTE ] GCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:34:49.974897] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-02. [2025-09-16T18:34:50.357147] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-09-16T18:34:50.774910] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-07. [2025-09-16T18:34:51.066336] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T18:34:51.574926] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-03. [2025-09-16T18:34:52.006812] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-09-16T18:34:52.374919] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-06. [2025-09-16T18:34:52.422274] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-09-16T18:34:53.174921] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-04. [2025-09-16T18:34:53.935921] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T18:34:53.974858] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-07. [2025-09-16T18:34:54.557856] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-09-16T18:34:54.774906] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-05. [2025-09-16T18:34:55.174924] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T18:34:55.574897] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-08. [2025-09-16T18:34:55.691676] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-09-16T18:34:56.374899] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-05. [2025-09-16T18:34:56.586096] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T18:34:57.174919] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-08. [2025-09-16T18:34:57.246301] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-09-16T18:34:57.974947] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-06. [2025-09-16T18:34:58.677075] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-09-16T18:34:58.710268] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T18:34:58.774879] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-07. [2025-09-16T18:34:59.574909] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-09. [2025-09-16T18:34:59.962552] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-09-16T18:35:00.374931] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-06. [2025-09-16T18:35:00.968473] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T18:35:00.973726] [INFO ] [REMOTE ] GBTG: Full fetch complete (43561 records). [2025-09-16T18:35:00.973853] [DEBUG] [VALIDATOR ] GBTG: Validating primary data. [2025-09-16T18:35:01.181475] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-09. [2025-09-16T18:35:01.364566] [DEBUG] [VALIDATOR ] GBTG: All expected trading days are present. [2025-09-16T18:35:01.364678] [DEBUG] [VALIDATOR ] GBTG: Primary data validation passed. [2025-09-16T18:35:01.365334] [INFO ] [MAIN ] GBTG: Fetched primary data is valid. [2025-09-16T18:35:01.365378] [DEBUG] [LOCAL ] GBTG: Saving primary data. [2025-09-16T18:35:01.366574] [DEBUG] [LOCAL ] GBTG: Trimmed 43561 records to 30025 (keeping 150 days of data) [2025-09-16T18:35:01.539722] [DEBUG] [LOCAL ] GBTG: Primary data saved (30025 records). [2025-09-16T18:35:01.539850] [INFO ] [MAIN ] GBTG: Full history saved. [2025-09-16T18:35:01.539933] [INFO ] [MAIN ] Update Progress: 897/2369 symbols completed. GBTG took 29.2s. Estimated remaining: 2h 35m 51s. Rate: 9.4 symbols/min [2025-09-16T18:35:01.718705] [INFO ] [MAIN ] GCTS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:01.718975] [DEBUG] [LOCAL ] GCTS: No existing data file found in primary storage. [2025-09-16T18:35:01.719056] [WARN ] [MAIN ] GCTS: No existing data found, performing full refresh instead. [2025-09-16T18:35:01.719337] [INFO ] [MAIN ] GCTS: Wiping existing primary data before full refresh. [2025-09-16T18:35:01.719415] [INFO ] [LOCAL ] GCTS: Starting deletion of primary data. [2025-09-16T18:35:01.719462] [DEBUG] [LOCAL ] GCTS: No primary data file found to delete. [2025-09-16T18:35:01.719535] [INFO ] [MAIN ] GCTS: Starting full primary refresh. [2025-09-16T18:35:01.828586] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T18:35:01.976455] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-08. [2025-09-16T18:35:02.187070] [INFO ] [REMOTE ] GCTS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:02.779977] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-07. [2025-09-16T18:35:02.789163] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T18:35:02.794920] [INFO ] [REMOTE ] GCI: Full fetch complete (47995 records). [2025-09-16T18:35:02.795053] [DEBUG] [VALIDATOR ] GCI: Validating primary data. [2025-09-16T18:35:03.182220] [DEBUG] [VALIDATOR ] GCI: All expected trading days are present. [2025-09-16T18:35:03.182311] [DEBUG] [VALIDATOR ] GCI: Primary data validation passed. [2025-09-16T18:35:03.183047] [INFO ] [MAIN ] GCI: Fetched primary data is valid. [2025-09-16T18:35:03.183102] [DEBUG] [LOCAL ] GCI: Saving primary data. [2025-09-16T18:35:03.198728] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-09-16T18:35:03.204313] [DEBUG] [LOCAL ] GCI: Trimmed 47995 records to 32576 (keeping 150 days of data) [2025-09-16T18:35:03.392794] [DEBUG] [LOCAL ] GCI: Primary data saved (32576 records). [2025-09-16T18:35:03.392961] [INFO ] [MAIN ] GCI: Full history saved. [2025-09-16T18:35:03.393082] [INFO ] [MAIN ] Update Progress: 898/2369 symbols completed. GCI took 24.9s. Estimated remaining: 2h 35m 38s. Rate: 9.5 symbols/min [2025-09-16T18:35:03.564942] [INFO ] [MAIN ] GCV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:03.572464] [DEBUG] [LOCAL ] GCV: Loaded 5377 existing records from primary storage. [2025-09-16T18:35:03.572555] [INFO ] [MAIN ] GCV: Loaded 5377 existing records. [2025-09-16T18:35:03.583606] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-02. [2025-09-16T18:35:03.757886] [WARN ] [MAIN ] GCV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:35:03.769850] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-09-16T18:35:03.770051] [INFO ] [MAIN ] GCV: Wiping existing primary data before full refresh. [2025-09-16T18:35:03.774219] [INFO ] [LOCAL ] GCV: Starting deletion of primary data. [2025-09-16T18:35:03.774982] [DEBUG] [LOCAL ] GCV: Deleted primary data file. [2025-09-16T18:35:03.775038] [INFO ] [MAIN ] GCV: Starting full primary refresh. [2025-09-16T18:35:04.132439] [INFO ] [REMOTE ] GCV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:04.374936] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-08. [2025-09-16T18:35:04.495236] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-09-16T18:35:05.125321] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T18:35:05.174907] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-09. [2025-09-16T18:35:05.974926] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-03. [2025-09-16T18:35:06.017411] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T18:35:06.020802] [INFO ] [REMOTE ] GCO: Full fetch complete (27925 records). [2025-09-16T18:35:06.020894] [DEBUG] [VALIDATOR ] GCO: Validating primary data. [2025-09-16T18:35:06.384537] [DEBUG] [VALIDATOR ] GCO: All expected trading days are present. [2025-09-16T18:35:06.384614] [DEBUG] [VALIDATOR ] GCO: Primary data validation passed. [2025-09-16T18:35:06.385171] [INFO ] [MAIN ] GCO: Fetched primary data is valid. [2025-09-16T18:35:06.385216] [DEBUG] [LOCAL ] GCO: Saving primary data. [2025-09-16T18:35:06.385914] [DEBUG] [LOCAL ] GCO: Trimmed 27925 records to 17442 (keeping 150 days of data) [2025-09-16T18:35:06.492896] [DEBUG] [LOCAL ] GCO: Primary data saved (17442 records). [2025-09-16T18:35:06.493050] [INFO ] [MAIN ] GCO: Full history saved. [2025-09-16T18:35:06.493140] [INFO ] [MAIN ] Update Progress: 899/2369 symbols completed. GCO took 17.6s. Estimated remaining: 2h 35m 26s. Rate: 9.5 symbols/min [2025-09-16T18:35:06.677932] [INFO ] [MAIN ] GD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:06.714541] [DEBUG] [LOCAL ] GD: Loaded 34238 existing records from primary storage. [2025-09-16T18:35:06.714735] [INFO ] [MAIN ] GD: Loaded 34238 existing records. [2025-09-16T18:35:06.780710] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-02. [2025-09-16T18:35:06.903785] [WARN ] [MAIN ] GD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:35:06.904076] [INFO ] [MAIN ] GD: Wiping existing primary data before full refresh. [2025-09-16T18:35:06.904126] [INFO ] [LOCAL ] GD: Starting deletion of primary data. [2025-09-16T18:35:06.905002] [DEBUG] [LOCAL ] GD: Deleted primary data file. [2025-09-16T18:35:06.905064] [INFO ] [MAIN ] GD: Starting full primary refresh. [2025-09-16T18:35:07.103249] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-09-16T18:35:07.286347] [INFO ] [REMOTE ] GD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:07.513239] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-09-16T18:35:07.574877] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-02. [2025-09-16T18:35:08.374879] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-09. [2025-09-16T18:35:09.174912] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-03. [2025-09-16T18:35:09.202567] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T18:35:09.265314] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T18:35:09.269597] [INFO ] [REMOTE ] GBX: Full fetch complete (32583 records). [2025-09-16T18:35:09.269732] [DEBUG] [VALIDATOR ] GBX: Validating primary data. [2025-09-16T18:35:09.607209] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-09-16T18:35:09.649912] [DEBUG] [VALIDATOR ] GBX: All expected trading days are present. [2025-09-16T18:35:09.650000] [DEBUG] [VALIDATOR ] GBX: Primary data validation passed. [2025-09-16T18:35:09.650700] [INFO ] [MAIN ] GBX: Fetched primary data is valid. [2025-09-16T18:35:09.650759] [DEBUG] [LOCAL ] GBX: Saving primary data. [2025-09-16T18:35:09.651608] [DEBUG] [LOCAL ] GBX: Trimmed 32583 records to 20479 (keeping 150 days of data) [2025-09-16T18:35:09.777532] [DEBUG] [LOCAL ] GBX: Primary data saved (20479 records). [2025-09-16T18:35:09.777675] [INFO ] [MAIN ] GBX: Full history saved. [2025-09-16T18:35:09.777756] [INFO ] [MAIN ] Update Progress: 900/2369 symbols completed. GBX took 34.5s. Estimated remaining: 2h 35m 15s. Rate: 9.5 symbols/min [2025-09-16T18:35:09.950903] [INFO ] [MAIN ] GDDY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:09.974918] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-03. [2025-09-16T18:35:09.987998] [DEBUG] [LOCAL ] GDDY: Loaded 36534 existing records from primary storage. [2025-09-16T18:35:09.988154] [INFO ] [MAIN ] GDDY: Loaded 36534 existing records. [2025-09-16T18:35:10.172381] [WARN ] [MAIN ] GDDY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:35:10.172685] [INFO ] [MAIN ] GDDY: Wiping existing primary data before full refresh. [2025-09-16T18:35:10.172735] [INFO ] [LOCAL ] GDDY: Starting deletion of primary data. [2025-09-16T18:35:10.173623] [DEBUG] [LOCAL ] GDDY: Deleted primary data file. [2025-09-16T18:35:10.173700] [INFO ] [MAIN ] GDDY: Starting full primary refresh. [2025-09-16T18:35:10.565692] [INFO ] [REMOTE ] GDDY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:10.774899] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-02. [2025-09-16T18:35:11.574902] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-04. [2025-09-16T18:35:11.826564] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-09-16T18:35:12.214288] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-09-16T18:35:12.374869] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-04. [2025-09-16T18:35:12.863093] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-02 (2.09 seconds). [2025-09-16T18:35:13.003833] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-09-16T18:35:13.174897] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-05. [2025-09-16T18:35:13.974933] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-03. [2025-09-16T18:35:14.429031] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T18:35:14.774895] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-05. [2025-09-16T18:35:15.191819] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T18:35:15.526118] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T18:35:15.574920] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-04. [2025-09-16T18:35:16.374895] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-06. [2025-09-16T18:35:17.118934] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-09-16T18:35:17.174928] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-06. [2025-09-16T18:35:17.478623] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-09-16T18:35:17.974912] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-04. [2025-09-16T18:35:18.545123] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T18:35:18.774869] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-07. [2025-09-16T18:35:19.574903] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-05. [2025-09-16T18:35:20.123460] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T18:35:20.198826] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-04 (2.22 seconds). [2025-09-16T18:35:20.374906] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-05. [2025-09-16T18:35:21.174889] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-07. [2025-09-16T18:35:21.283910] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-09-16T18:35:21.624918] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T18:35:21.852156] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-09-16T18:35:21.974913] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-08. [2025-09-16T18:35:22.587527] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-09-16T18:35:22.774894] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-08. [2025-09-16T18:35:23.574968] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-06. [2025-09-16T18:35:23.784062] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T18:35:24.374907] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-09. [2025-09-16T18:35:25.174902] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-06. [2025-09-16T18:35:25.478207] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-09-16T18:35:25.496105] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-09-16T18:35:25.499424] [INFO ] [REMOTE ] GCTS: Full fetch complete (20479 records). [2025-09-16T18:35:25.499535] [DEBUG] [VALIDATOR ] GCTS: Validating primary data. [2025-09-16T18:35:25.905366] [DEBUG] [VALIDATOR ] GCTS: All expected trading days are present. [2025-09-16T18:35:25.905467] [DEBUG] [VALIDATOR ] GCTS: Primary data validation passed. [2025-09-16T18:35:25.906003] [INFO ] [MAIN ] GCTS: Fetched primary data is valid. [2025-09-16T18:35:25.906062] [DEBUG] [LOCAL ] GCTS: Saving primary data. [2025-09-16T18:35:25.906768] [DEBUG] [LOCAL ] GCTS: Trimmed 20479 records to 15587 (keeping 150 days of data) [2025-09-16T18:35:25.975180] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-07. [2025-09-16T18:35:26.001148] [DEBUG] [LOCAL ] GCTS: Primary data saved (15587 records). [2025-09-16T18:35:26.001300] [INFO ] [MAIN ] GCTS: Full history saved. [2025-09-16T18:35:26.001395] [INFO ] [MAIN ] Update Progress: 901/2369 symbols completed. GCTS took 24.5s. Estimated remaining: 2h 35m 24s. Rate: 9.4 symbols/min [2025-09-16T18:35:26.171041] [INFO ] [MAIN ] GDIV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:26.175536] [DEBUG] [LOCAL ] GDIV: Loaded 2590 existing records from primary storage. [2025-09-16T18:35:26.175633] [INFO ] [MAIN ] GDIV: Loaded 2590 existing records. [2025-09-16T18:35:26.354822] [WARN ] [MAIN ] GDIV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:35:26.355102] [INFO ] [MAIN ] GDIV: Wiping existing primary data before full refresh. [2025-09-16T18:35:26.355158] [INFO ] [LOCAL ] GDIV: Starting deletion of primary data. [2025-09-16T18:35:26.355410] [DEBUG] [LOCAL ] GDIV: Deleted primary data file. [2025-09-16T18:35:26.355464] [INFO ] [MAIN ] GDIV: Starting full primary refresh. [2025-09-16T18:35:26.726507] [INFO ] [REMOTE ] GDIV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:26.757608] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T18:35:26.774858] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-07. [2025-09-16T18:35:27.574927] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-02. [2025-09-16T18:35:27.874479] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-09-16T18:35:28.353909] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-09-16T18:35:28.374926] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-08. [2025-09-16T18:35:28.474388] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T18:35:29.174904] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-03. [2025-09-16T18:35:29.689153] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T18:35:29.803559] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-09-16T18:35:29.974904] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-09. [2025-09-16T18:35:30.774905] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-04. [2025-09-16T18:35:31.233613] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-09-16T18:35:31.580084] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-09. [2025-09-16T18:35:31.588407] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T18:35:31.594010] [INFO ] [REMOTE ] GD: Full fetch complete (52162 records). [2025-09-16T18:35:31.594119] [DEBUG] [VALIDATOR ] GD: Validating primary data. [2025-09-16T18:35:31.980514] [DEBUG] [VALIDATOR ] GD: All expected trading days are present. [2025-09-16T18:35:31.980593] [DEBUG] [VALIDATOR ] GD: Primary data validation passed. [2025-09-16T18:35:31.981304] [INFO ] [MAIN ] GD: Fetched primary data is valid. [2025-09-16T18:35:31.981352] [DEBUG] [LOCAL ] GD: Saving primary data. [2025-09-16T18:35:31.982408] [DEBUG] [LOCAL ] GD: Trimmed 52162 records to 34593 (keeping 150 days of data) [2025-09-16T18:35:32.205036] [DEBUG] [LOCAL ] GD: Primary data saved (34593 records). [2025-09-16T18:35:32.205197] [INFO ] [MAIN ] GD: Full history saved. [2025-09-16T18:35:32.205296] [INFO ] [MAIN ] Update Progress: 902/2369 symbols completed. GD took 25.7s. Estimated remaining: 2h 35m 18s. Rate: 9.4 symbols/min [2025-09-16T18:35:32.381717] [INFO ] [MAIN ] GDL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:32.380021] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-05. [2025-09-16T18:35:32.386577] [DEBUG] [LOCAL ] GDL: Loaded 1737 existing records from primary storage. [2025-09-16T18:35:32.386708] [INFO ] [MAIN ] GDL: Loaded 1737 existing records. [2025-09-16T18:35:32.510403] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T18:35:32.546359] [INFO ] [REMOTE ] GCV: Full fetch complete (7464 records). [2025-09-16T18:35:32.555388] [DEBUG] [VALIDATOR ] GCV: Validating primary data. [2025-09-16T18:35:32.569856] [FATAL] [VALIDATOR ] GCV: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:35:32.574331] [FATAL] [MAIN ] GCV: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:35:32.574200] [WARN ] [MAIN ] GDL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:35:32.574638] [INFO ] [MAIN ] GDL: Wiping existing primary data before full refresh. [2025-09-16T18:35:32.574481] [WARN ] [MAIN ] GCV: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:35:32.574813] [INFO ] [LOCAL ] GCV: Starting deletion of primary data. [2025-09-16T18:35:32.574763] [INFO ] [LOCAL ] GDL: Starting deletion of primary data. [2025-09-16T18:35:32.574911] [DEBUG] [LOCAL ] GCV: No primary data file found to delete. [2025-09-16T18:35:32.575004] [INFO ] [LOCAL ] GCV: Starting deletion of derivative data. [2025-09-16T18:35:32.575259] [DEBUG] [LOCAL ] GDL: Deleted primary data file. [2025-09-16T18:35:32.575337] [INFO ] [MAIN ] GDL: Starting full primary refresh. [2025-09-16T18:35:32.606254] [DEBUG] [LOCAL ] GCV: Deleted derivative data file. [2025-09-16T18:35:32.640073] [INFO ] [MAIN ] Update Progress: 903/2369 symbols completed. GCV took 29.2s. Estimated remaining: 2h 35m 2s. Rate: 9.5 symbols/min [2025-09-16T18:35:33.000402] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-09-16T18:35:33.001857] [INFO ] [MAIN ] GDO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:33.113261] [DEBUG] [LOCAL ] GDO: Loaded 4329 existing records from primary storage. [2025-09-16T18:35:33.122606] [INFO ] [MAIN ] GDO: Loaded 4329 existing records. [2025-09-16T18:35:33.180596] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-06. [2025-09-16T18:35:33.141664] [INFO ] [REMOTE ] GDL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:33.324866] [WARN ] [MAIN ] GDO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:35:33.325105] [INFO ] [MAIN ] GDO: Wiping existing primary data before full refresh. [2025-09-16T18:35:33.325153] [INFO ] [LOCAL ] GDO: Starting deletion of primary data. [2025-09-16T18:35:33.325404] [DEBUG] [LOCAL ] GDO: Deleted primary data file. [2025-09-16T18:35:33.325449] [INFO ] [MAIN ] GDO: Starting full primary refresh. [2025-09-16T18:35:33.678609] [INFO ] [REMOTE ] GDO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:33.679214] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-09-16T18:35:33.974903] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-02. [2025-09-16T18:35:34.764457] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T18:35:34.774852] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-03. [2025-09-16T18:35:35.198463] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-09-16T18:35:35.574897] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-07. [2025-09-16T18:35:36.117049] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-09-16T18:35:36.374909] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-04. [2025-09-16T18:35:36.707124] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-09-16T18:35:37.174911] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-08. [2025-09-16T18:35:37.974939] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-05. [2025-09-16T18:35:38.396134] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T18:35:38.775033] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-08. [2025-09-16T18:35:39.081278] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-09-16T18:35:39.348417] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-09-16T18:35:39.574924] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-09. [2025-09-16T18:35:40.070954] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-09-16T18:35:40.071918] [INFO ] [REMOTE ] GDIV: Full fetch complete (4132 records). [2025-09-16T18:35:40.071999] [DEBUG] [VALIDATOR ] GDIV: Validating primary data. [2025-09-16T18:35:40.391110] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-06. [2025-09-16T18:35:40.429916] [DEBUG] [VALIDATOR ] GDIV: All expected trading days are present. [2025-09-16T18:35:40.429995] [DEBUG] [VALIDATOR ] GDIV: Primary data validation passed. [2025-09-16T18:35:40.430121] [INFO ] [MAIN ] GDIV: Fetched primary data is valid. [2025-09-16T18:35:40.430159] [DEBUG] [LOCAL ] GDIV: Saving primary data. [2025-09-16T18:35:40.430564] [DEBUG] [LOCAL ] GDIV: Trimmed 4132 records to 2606 (keeping 150 days of data) [2025-09-16T18:35:40.447967] [DEBUG] [LOCAL ] GDIV: Primary data saved (2606 records). [2025-09-16T18:35:40.448097] [INFO ] [MAIN ] GDIV: Full history saved. [2025-09-16T18:35:40.448199] [INFO ] [MAIN ] Update Progress: 904/2369 symbols completed. GDIV took 14.4s. Estimated remaining: 2h 34m 58s. Rate: 9.5 symbols/min [2025-09-16T18:35:40.630884] [INFO ] [MAIN ] GDOT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:40.658123] [DEBUG] [LOCAL ] GDOT: Loaded 27340 existing records from primary storage. [2025-09-16T18:35:40.658276] [INFO ] [MAIN ] GDOT: Loaded 27340 existing records. [2025-09-16T18:35:40.839434] [WARN ] [MAIN ] GDOT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:35:40.839727] [INFO ] [MAIN ] GDOT: Wiping existing primary data before full refresh. [2025-09-16T18:35:40.839784] [INFO ] [LOCAL ] GDOT: Starting deletion of primary data. [2025-09-16T18:35:40.840408] [DEBUG] [LOCAL ] GDOT: Deleted primary data file. [2025-09-16T18:35:40.840457] [INFO ] [MAIN ] GDOT: Starting full primary refresh. [2025-09-16T18:35:40.945435] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-09-16T18:35:41.180033] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-02. [2025-09-16T18:35:41.244283] [INFO ] [REMOTE ] GDOT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:41.974964] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-02. [2025-09-16T18:35:42.135358] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-09-16T18:35:42.774910] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-09. [2025-09-16T18:35:43.101847] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-09-16T18:35:43.574927] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-03. [2025-09-16T18:35:44.374914] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-07. [2025-09-16T18:35:44.577261] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-09-16T18:35:44.583558] [INFO ] [REMOTE ] GDDY: Full fetch complete (54419 records). [2025-09-16T18:35:44.583721] [DEBUG] [VALIDATOR ] GDDY: Validating primary data. [2025-09-16T18:35:44.892228] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-09-16T18:35:44.988818] [DEBUG] [VALIDATOR ] GDDY: All expected trading days are present. [2025-09-16T18:35:44.988927] [DEBUG] [VALIDATOR ] GDDY: Primary data validation passed. [2025-09-16T18:35:44.989725] [INFO ] [MAIN ] GDDY: Fetched primary data is valid. [2025-09-16T18:35:44.989778] [DEBUG] [LOCAL ] GDDY: Saving primary data. [2025-09-16T18:35:44.991112] [DEBUG] [LOCAL ] GDDY: Trimmed 54419 records to 36908 (keeping 150 days of data) [2025-09-16T18:35:45.191818] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-08. [2025-09-16T18:35:45.223412] [DEBUG] [LOCAL ] GDDY: Primary data saved (36908 records). [2025-09-16T18:35:45.223576] [INFO ] [MAIN ] GDDY: Full history saved. [2025-09-16T18:35:45.223692] [INFO ] [MAIN ] Update Progress: 905/2369 symbols completed. GDDY took 35.4s. Estimated remaining: 2h 34m 49s. Rate: 9.5 symbols/min [2025-09-16T18:35:45.397703] [INFO ] [MAIN ] GDV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:45.429608] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-09-16T18:35:45.449989] [DEBUG] [LOCAL ] GDV: Loaded 14265 existing records from primary storage. [2025-09-16T18:35:45.450167] [INFO ] [MAIN ] GDV: Loaded 14265 existing records. [2025-09-16T18:35:45.631776] [WARN ] [MAIN ] GDV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:35:45.632064] [INFO ] [MAIN ] GDV: Wiping existing primary data before full refresh. [2025-09-16T18:35:45.632119] [INFO ] [LOCAL ] GDV: Starting deletion of primary data. [2025-09-16T18:35:45.632568] [DEBUG] [LOCAL ] GDV: Deleted primary data file. [2025-09-16T18:35:45.632614] [INFO ] [MAIN ] GDV: Starting full primary refresh. [2025-09-16T18:35:45.736152] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-09-16T18:35:45.992991] [INFO ] [REMOTE ] GDV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:45.988854] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-03. [2025-09-16T18:35:46.484588] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-09-16T18:35:46.774929] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-02. [2025-09-16T18:35:47.574909] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-04. [2025-09-16T18:35:48.224238] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-09-16T18:35:48.380019] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-04. [2025-09-16T18:35:48.390284] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T18:35:49.174949] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-05. [2025-09-16T18:35:49.714021] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T18:35:49.767901] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-09-16T18:35:49.974958] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-09. [2025-09-16T18:35:50.774921] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-05. [2025-09-16T18:35:50.783481] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-09-16T18:35:50.784443] [INFO ] [REMOTE ] GDL: Full fetch complete (2543 records). [2025-09-16T18:35:50.784510] [DEBUG] [VALIDATOR ] GDL: Validating primary data. [2025-09-16T18:35:50.784878] [FATAL] [VALIDATOR ] GDL: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:35:50.784956] [FATAL] [MAIN ] GDL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:35:50.785010] [WARN ] [MAIN ] GDL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:35:50.785056] [INFO ] [LOCAL ] GDL: Starting deletion of primary data. [2025-09-16T18:35:50.785109] [DEBUG] [LOCAL ] GDL: No primary data file found to delete. [2025-09-16T18:35:50.785145] [INFO ] [LOCAL ] GDL: Starting deletion of derivative data. [2025-09-16T18:35:50.785395] [DEBUG] [LOCAL ] GDL: Deleted derivative data file. [2025-09-16T18:35:50.785476] [INFO ] [MAIN ] Update Progress: 906/2369 symbols completed. GDL took 18.6s. Estimated remaining: 2h 34m 41s. Rate: 9.5 symbols/min [2025-09-16T18:35:50.961371] [INFO ] [MAIN ] GE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:51.010925] [DEBUG] [LOCAL ] GE: Loaded 39678 existing records from primary storage. [2025-09-16T18:35:51.011150] [INFO ] [MAIN ] GE: Loaded 39678 existing records. [2025-09-16T18:35:51.209047] [WARN ] [MAIN ] GE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:35:51.209336] [INFO ] [MAIN ] GE: Wiping existing primary data before full refresh. [2025-09-16T18:35:51.209402] [INFO ] [LOCAL ] GE: Starting deletion of primary data. [2025-09-16T18:35:51.210490] [DEBUG] [LOCAL ] GE: Deleted primary data file. [2025-09-16T18:35:51.210556] [INFO ] [MAIN ] GE: Starting full primary refresh. [2025-09-16T18:35:51.580121] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-03. [2025-09-16T18:35:51.583778] [INFO ] [REMOTE ] GE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:35:52.305304] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T18:35:52.374869] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-06. [2025-09-16T18:35:52.788296] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-09-16T18:35:53.174945] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-06. [2025-09-16T18:35:53.652962] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T18:35:53.974913] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-07. [2025-09-16T18:35:54.378693] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T18:35:54.774905] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-02. [2025-09-16T18:35:55.574909] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-07. [2025-09-16T18:35:55.970327] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-09-16T18:35:56.059814] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-09-16T18:35:56.374866] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-08. [2025-09-16T18:35:56.436636] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T18:35:57.174911] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-08. [2025-09-16T18:35:57.511024] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T18:35:57.679605] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-09-16T18:35:57.974931] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-04. [2025-09-16T18:35:58.774905] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-09. [2025-09-16T18:35:59.228877] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-09-16T18:35:59.358483] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-09-16T18:35:59.359984] [INFO ] [REMOTE ] GDO: Full fetch complete (6513 records). [2025-09-16T18:35:59.360077] [DEBUG] [VALIDATOR ] GDO: Validating primary data. [2025-09-16T18:35:59.580013] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-03. [2025-09-16T18:35:59.749161] [DEBUG] [VALIDATOR ] GDO: All expected trading days are present. [2025-09-16T18:35:59.749242] [DEBUG] [VALIDATOR ] GDO: Primary data validation passed. [2025-09-16T18:35:59.749426] [INFO ] [MAIN ] GDO: Fetched primary data is valid. [2025-09-16T18:35:59.749478] [DEBUG] [LOCAL ] GDO: Saving primary data. [2025-09-16T18:35:59.749896] [DEBUG] [LOCAL ] GDO: Trimmed 6513 records to 4393 (keeping 150 days of data) [2025-09-16T18:35:59.777966] [DEBUG] [LOCAL ] GDO: Primary data saved (4393 records). [2025-09-16T18:35:59.778067] [INFO ] [MAIN ] GDO: Full history saved. [2025-09-16T18:35:59.778181] [INFO ] [MAIN ] Update Progress: 907/2369 symbols completed. GDO took 27.1s. Estimated remaining: 2h 34m 39s. Rate: 9.5 symbols/min [2025-09-16T18:35:59.953684] [INFO ] [MAIN ] GEF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:35:59.971925] [DEBUG] [LOCAL ] GEF: Loaded 15263 existing records from primary storage. [2025-09-16T18:35:59.972085] [INFO ] [MAIN ] GEF: Loaded 15263 existing records. [2025-09-16T18:36:00.168278] [WARN ] [MAIN ] GEF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:00.168585] [INFO ] [MAIN ] GEF: Wiping existing primary data before full refresh. [2025-09-16T18:36:00.168990] [INFO ] [LOCAL ] GEF: Starting deletion of primary data. [2025-09-16T18:36:00.169494] [DEBUG] [LOCAL ] GEF: Deleted primary data file. [2025-09-16T18:36:00.169559] [INFO ] [MAIN ] GEF: Starting full primary refresh. [2025-09-16T18:36:00.380024] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-05. [2025-09-16T18:36:00.548698] [INFO ] [REMOTE ] GEF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:01.174902] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-09. [2025-09-16T18:36:01.622530] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-09-16T18:36:01.812807] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-03 (2.23 seconds). [2025-09-16T18:36:01.974908] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-04. [2025-09-16T18:36:02.474434] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T18:36:02.481803] [INFO ] [REMOTE ] GDOT: Full fetch complete (40983 records). [2025-09-16T18:36:02.482017] [DEBUG] [VALIDATOR ] GDOT: Validating primary data. [2025-09-16T18:36:02.483134] [FATAL] [VALIDATOR ] GDOT: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:36:02.483292] [FATAL] [MAIN ] GDOT: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:36:02.483365] [WARN ] [MAIN ] GDOT: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:36:02.483438] [INFO ] [LOCAL ] GDOT: Starting deletion of primary data. [2025-09-16T18:36:02.483527] [DEBUG] [LOCAL ] GDOT: No primary data file found to delete. [2025-09-16T18:36:02.483589] [INFO ] [LOCAL ] GDOT: Starting deletion of derivative data. [2025-09-16T18:36:02.483929] [DEBUG] [LOCAL ] GDOT: Deleted derivative data file. [2025-09-16T18:36:02.484134] [INFO ] [MAIN ] Update Progress: 908/2369 symbols completed. GDOT took 22.0s. Estimated remaining: 2h 34m 27s. Rate: 9.5 symbols/min [2025-09-16T18:36:02.689109] [INFO ] [MAIN ] GEL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:36:02.714551] [DEBUG] [LOCAL ] GEL: Loaded 24836 existing records from primary storage. [2025-09-16T18:36:02.714720] [INFO ] [MAIN ] GEL: Loaded 24836 existing records. [2025-09-16T18:36:02.780356] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-06. [2025-09-16T18:36:02.894928] [WARN ] [MAIN ] GEL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:02.895161] [INFO ] [MAIN ] GEL: Wiping existing primary data before full refresh. [2025-09-16T18:36:02.895207] [INFO ] [LOCAL ] GEL: Starting deletion of primary data. [2025-09-16T18:36:02.895853] [DEBUG] [LOCAL ] GEL: Deleted primary data file. [2025-09-16T18:36:02.895909] [INFO ] [MAIN ] GEL: Starting full primary refresh. [2025-09-16T18:36:03.246789] [INFO ] [REMOTE ] GEL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:03.445721] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-09-16T18:36:03.574945] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-02. [2025-09-16T18:36:04.374890] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-07. [2025-09-16T18:36:04.951705] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T18:36:05.135328] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-04 (3.16 seconds). [2025-09-16T18:36:05.174864] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-05. [2025-09-16T18:36:05.219139] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-09-16T18:36:05.974945] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-03. [2025-09-16T18:36:06.774904] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-08. [2025-09-16T18:36:07.105057] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-09-16T18:36:07.147854] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T18:36:07.574937] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-06. [2025-09-16T18:36:07.819940] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T18:36:08.374905] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-04. [2025-09-16T18:36:09.174940] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-09. [2025-09-16T18:36:09.268028] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-09-16T18:36:09.441815] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-09-16T18:36:09.974902] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-07. [2025-09-16T18:36:10.091000] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T18:36:10.093399] [INFO ] [REMOTE ] GDV: Full fetch complete (21730 records). [2025-09-16T18:36:10.093487] [DEBUG] [VALIDATOR ] GDV: Validating primary data. [2025-09-16T18:36:10.485796] [DEBUG] [VALIDATOR ] GDV: All expected trading days are present. [2025-09-16T18:36:10.485875] [DEBUG] [VALIDATOR ] GDV: Primary data validation passed. [2025-09-16T18:36:10.486318] [INFO ] [MAIN ] GDV: Fetched primary data is valid. [2025-09-16T18:36:10.486358] [DEBUG] [LOCAL ] GDV: Saving primary data. [2025-09-16T18:36:10.486958] [DEBUG] [LOCAL ] GDV: Trimmed 21730 records to 14371 (keeping 150 days of data) [2025-09-16T18:36:10.578599] [DEBUG] [LOCAL ] GDV: Primary data saved (14371 records). [2025-09-16T18:36:10.578764] [INFO ] [MAIN ] GDV: Full history saved. [2025-09-16T18:36:10.578869] [INFO ] [MAIN ] Update Progress: 909/2369 symbols completed. GDV took 25.4s. Estimated remaining: 2h 34m 24s. Rate: 9.5 symbols/min [2025-09-16T18:36:10.751491] [INFO ] [MAIN ] GENI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:36:10.774918] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-05. [2025-09-16T18:36:10.792722] [DEBUG] [LOCAL ] GENI: Loaded 38999 existing records from primary storage. [2025-09-16T18:36:10.792854] [INFO ] [MAIN ] GENI: Loaded 38999 existing records. [2025-09-16T18:36:10.970182] [WARN ] [MAIN ] GENI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:10.970457] [INFO ] [MAIN ] GENI: Wiping existing primary data before full refresh. [2025-09-16T18:36:10.970503] [INFO ] [LOCAL ] GENI: Starting deletion of primary data. [2025-09-16T18:36:10.971301] [DEBUG] [LOCAL ] GENI: Deleted primary data file. [2025-09-16T18:36:10.971350] [INFO ] [MAIN ] GENI: Starting full primary refresh. [2025-09-16T18:36:11.326968] [INFO ] [REMOTE ] GENI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:11.574900] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-02. [2025-09-16T18:36:11.744736] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T18:36:12.046939] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-09-16T18:36:12.374892] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-08. [2025-09-16T18:36:13.174878] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-02. [2025-09-16T18:36:13.209465] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T18:36:13.974917] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-03. [2025-09-16T18:36:14.065538] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-09-16T18:36:14.319594] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-09-16T18:36:14.774905] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-06. [2025-09-16T18:36:15.424242] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T18:36:15.574899] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-03. [2025-09-16T18:36:15.937948] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-09-16T18:36:16.374906] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-09. [2025-09-16T18:36:16.948613] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T18:36:17.174904] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-04. [2025-09-16T18:36:17.974899] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-04. [2025-09-16T18:36:18.184771] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-09-16T18:36:18.190559] [INFO ] [REMOTE ] GE: Full fetch complete (58001 records). [2025-09-16T18:36:18.190681] [DEBUG] [VALIDATOR ] GE: Validating primary data. [2025-09-16T18:36:18.191269] [FATAL] [VALIDATOR ] GE: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:36:18.191345] [FATAL] [MAIN ] GE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:36:18.191382] [WARN ] [MAIN ] GE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:36:18.191433] [INFO ] [LOCAL ] GE: Starting deletion of primary data. [2025-09-16T18:36:18.191490] [DEBUG] [LOCAL ] GE: No primary data file found to delete. [2025-09-16T18:36:18.191527] [INFO ] [LOCAL ] GE: Starting deletion of derivative data. [2025-09-16T18:36:18.191865] [DEBUG] [LOCAL ] GE: Deleted derivative data file. [2025-09-16T18:36:18.191959] [INFO ] [MAIN ] Update Progress: 910/2369 symbols completed. GE took 27.4s. Estimated remaining: 2h 34m 19s. Rate: 9.5 symbols/min [2025-09-16T18:36:18.369771] [INFO ] [MAIN ] GEO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:36:18.408558] [DEBUG] [LOCAL ] GEO: Loaded 38450 existing records from primary storage. [2025-09-16T18:36:18.408716] [INFO ] [MAIN ] GEO: Loaded 38450 existing records. [2025-09-16T18:36:18.607226] [WARN ] [MAIN ] GEO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:18.607603] [INFO ] [MAIN ] GEO: Wiping existing primary data before full refresh. [2025-09-16T18:36:18.607762] [INFO ] [LOCAL ] GEO: Starting deletion of primary data. [2025-09-16T18:36:18.608533] [DEBUG] [LOCAL ] GEO: Deleted primary data file. [2025-09-16T18:36:18.608585] [INFO ] [MAIN ] GEO: Starting full primary refresh. [2025-09-16T18:36:18.778874] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-07. [2025-09-16T18:36:18.914104] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-09-16T18:36:19.022771] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-09-16T18:36:19.041036] [INFO ] [REMOTE ] GEO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:19.574912] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-02. [2025-09-16T18:36:19.869011] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T18:36:20.374901] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-05. [2025-09-16T18:36:21.174919] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-08. [2025-09-16T18:36:21.573201] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-09-16T18:36:21.668325] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-09-16T18:36:21.974902] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-06. [2025-09-16T18:36:22.202140] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T18:36:22.774926] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-09. [2025-09-16T18:36:23.377433] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-09-16T18:36:23.574910] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-05. [2025-09-16T18:36:23.858533] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T18:36:23.862151] [INFO ] [REMOTE ] GEL: Full fetch complete (37619 records). [2025-09-16T18:36:23.862238] [DEBUG] [VALIDATOR ] GEL: Validating primary data. [2025-09-16T18:36:24.242769] [DEBUG] [VALIDATOR ] GEL: All expected trading days are present. [2025-09-16T18:36:24.242850] [DEBUG] [VALIDATOR ] GEL: Primary data validation passed. [2025-09-16T18:36:24.243438] [INFO ] [MAIN ] GEL: Fetched primary data is valid. [2025-09-16T18:36:24.243479] [DEBUG] [LOCAL ] GEL: Saving primary data. [2025-09-16T18:36:24.244346] [DEBUG] [LOCAL ] GEL: Trimmed 37619 records to 24970 (keeping 150 days of data) [2025-09-16T18:36:24.375084] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-07. [2025-09-16T18:36:24.413360] [DEBUG] [LOCAL ] GEL: Primary data saved (24970 records). [2025-09-16T18:36:24.413491] [INFO ] [MAIN ] GEL: Full history saved. [2025-09-16T18:36:24.413582] [INFO ] [MAIN ] Update Progress: 911/2369 symbols completed. GEL took 21.9s. Estimated remaining: 2h 34m 13s. Rate: 9.5 symbols/min [2025-09-16T18:36:24.586929] [INFO ] [MAIN ] GES: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:36:24.596835] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-09-16T18:36:24.628233] [DEBUG] [LOCAL ] GES: Loaded 28051 existing records from primary storage. [2025-09-16T18:36:24.628403] [INFO ] [MAIN ] GES: Loaded 28051 existing records. [2025-09-16T18:36:24.813041] [WARN ] [MAIN ] GES: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:24.813315] [INFO ] [MAIN ] GES: Wiping existing primary data before full refresh. [2025-09-16T18:36:24.813364] [INFO ] [LOCAL ] GES: Starting deletion of primary data. [2025-09-16T18:36:24.814024] [DEBUG] [LOCAL ] GES: Deleted primary data file. [2025-09-16T18:36:24.814075] [INFO ] [MAIN ] GES: Starting full primary refresh. [2025-09-16T18:36:25.176079] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-06. [2025-09-16T18:36:25.179133] [INFO ] [REMOTE ] GES: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:25.974897] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-03. [2025-09-16T18:36:26.298816] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T18:36:26.503364] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T18:36:26.774912] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-02. [2025-09-16T18:36:27.574921] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-08. [2025-09-16T18:36:27.822570] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-09-16T18:36:28.253524] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T18:36:28.374892] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-07. [2025-09-16T18:36:29.094161] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-09-16T18:36:29.174851] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-09. [2025-09-16T18:36:29.568705] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-09-16T18:36:29.974896] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-04. [2025-09-16T18:36:30.782766] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-08. [2025-09-16T18:36:30.784836] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T18:36:30.791329] [INFO ] [REMOTE ] GENI: Full fetch complete (57262 records). [2025-09-16T18:36:30.791415] [DEBUG] [VALIDATOR ] GENI: Validating primary data. [2025-09-16T18:36:31.181722] [DEBUG] [VALIDATOR ] GENI: All expected trading days are present. [2025-09-16T18:36:31.181819] [DEBUG] [VALIDATOR ] GENI: Primary data validation passed. [2025-09-16T18:36:31.182576] [INFO ] [MAIN ] GENI: Fetched primary data is valid. [2025-09-16T18:36:31.182617] [DEBUG] [LOCAL ] GENI: Saving primary data. [2025-09-16T18:36:31.183685] [DEBUG] [LOCAL ] GENI: Trimmed 57262 records to 39386 (keeping 150 days of data) [2025-09-16T18:36:31.421935] [DEBUG] [LOCAL ] GENI: Primary data saved (39386 records). [2025-09-16T18:36:31.422094] [INFO ] [MAIN ] GENI: Full history saved. [2025-09-16T18:36:31.422183] [INFO ] [MAIN ] Update Progress: 912/2369 symbols completed. GENI took 20.8s. Estimated remaining: 2h 34m 7s. Rate: 9.5 symbols/min [2025-09-16T18:36:31.577327] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-03. [2025-09-16T18:36:31.596224] [INFO ] [MAIN ] GETY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:36:31.622430] [DEBUG] [LOCAL ] GETY: Loaded 25721 existing records from primary storage. [2025-09-16T18:36:31.622550] [INFO ] [MAIN ] GETY: Loaded 25721 existing records. [2025-09-16T18:36:31.806981] [WARN ] [MAIN ] GETY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:31.807587] [INFO ] [MAIN ] GETY: Wiping existing primary data before full refresh. [2025-09-16T18:36:31.807837] [INFO ] [LOCAL ] GETY: Starting deletion of primary data. [2025-09-16T18:36:31.808405] [DEBUG] [LOCAL ] GETY: Deleted primary data file. [2025-09-16T18:36:31.808452] [INFO ] [MAIN ] GETY: Starting full primary refresh. [2025-09-16T18:36:32.135108] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-09-16T18:36:32.218316] [INFO ] [REMOTE ] GETY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:32.230914] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T18:36:32.374881] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-09. [2025-09-16T18:36:32.968362] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T18:36:33.174913] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-05. [2025-09-16T18:36:33.452889] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T18:36:33.457813] [INFO ] [REMOTE ] GEF: Full fetch complete (22288 records). [2025-09-16T18:36:33.457910] [DEBUG] [VALIDATOR ] GEF: Validating primary data. [2025-09-16T18:36:33.826779] [DEBUG] [VALIDATOR ] GEF: All expected trading days are present. [2025-09-16T18:36:33.826858] [DEBUG] [VALIDATOR ] GEF: Primary data validation passed. [2025-09-16T18:36:33.827303] [INFO ] [MAIN ] GEF: Fetched primary data is valid. [2025-09-16T18:36:33.827345] [DEBUG] [LOCAL ] GEF: Saving primary data. [2025-09-16T18:36:33.827983] [DEBUG] [LOCAL ] GEF: Trimmed 22288 records to 15462 (keeping 150 days of data) [2025-09-16T18:36:33.928366] [DEBUG] [LOCAL ] GEF: Primary data saved (15462 records). [2025-09-16T18:36:33.928523] [INFO ] [MAIN ] GEF: Full history saved. [2025-09-16T18:36:33.928635] [INFO ] [MAIN ] Update Progress: 913/2369 symbols completed. GEF took 34.2s. Estimated remaining: 2h 33m 55s. Rate: 9.5 symbols/min [2025-09-16T18:36:33.986140] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-02. [2025-09-16T18:36:34.123566] [INFO ] [MAIN ] GEV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:36:34.172890] [DEBUG] [LOCAL ] GEV: Loaded 39188 existing records from primary storage. [2025-09-16T18:36:34.173052] [INFO ] [MAIN ] GEV: Loaded 39188 existing records. [2025-09-16T18:36:34.379408] [WARN ] [MAIN ] GEV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:34.379759] [INFO ] [MAIN ] GEV: Wiping existing primary data before full refresh. [2025-09-16T18:36:34.379887] [INFO ] [LOCAL ] GEV: Starting deletion of primary data. [2025-09-16T18:36:34.381185] [DEBUG] [LOCAL ] GEV: Deleted primary data file. [2025-09-16T18:36:34.381268] [INFO ] [MAIN ] GEV: Starting full primary refresh. [2025-09-16T18:36:34.782534] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-04. [2025-09-16T18:36:34.903220] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T18:36:35.022213] [INFO ] [REMOTE ] GEV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:35.408926] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T18:36:35.574894] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-02. [2025-09-16T18:36:36.374904] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-06. [2025-09-16T18:36:36.752801] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-09-16T18:36:37.174926] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-03. [2025-09-16T18:36:37.954416] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T18:36:37.974853] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-07. [2025-09-16T18:36:38.208735] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-02 (2.63 seconds). [2025-09-16T18:36:38.566082] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-09-16T18:36:38.774892] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-03. [2025-09-16T18:36:39.574982] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-04. [2025-09-16T18:36:39.673050] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T18:36:40.374934] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-08. [2025-09-16T18:36:40.982278] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-03 (2.21 seconds). [2025-09-16T18:36:41.080792] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T18:36:41.174853] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-05. [2025-09-16T18:36:41.868207] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T18:36:41.974917] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-04. [2025-09-16T18:36:42.181190] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T18:36:42.774918] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-05. [2025-09-16T18:36:43.574922] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-06. [2025-09-16T18:36:44.093779] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-09-16T18:36:44.374908] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-09. [2025-09-16T18:36:44.424360] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T18:36:45.093572] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T18:36:45.174867] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-07. [2025-09-16T18:36:45.974059] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T18:36:45.979464] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-05. [2025-09-16T18:36:45.980891] [INFO ] [REMOTE ] GEO: Full fetch complete (56844 records). [2025-09-16T18:36:45.980987] [DEBUG] [VALIDATOR ] GEO: Validating primary data. [2025-09-16T18:36:46.368431] [DEBUG] [VALIDATOR ] GEO: All expected trading days are present. [2025-09-16T18:36:46.368528] [DEBUG] [VALIDATOR ] GEO: Primary data validation passed. [2025-09-16T18:36:46.369387] [INFO ] [MAIN ] GEO: Fetched primary data is valid. [2025-09-16T18:36:46.369468] [DEBUG] [LOCAL ] GEO: Saving primary data. [2025-09-16T18:36:46.371148] [DEBUG] [LOCAL ] GEO: Trimmed 56844 records to 38814 (keeping 150 days of data) [2025-09-16T18:36:46.508695] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-09-16T18:36:46.635315] [DEBUG] [LOCAL ] GEO: Primary data saved (38814 records). [2025-09-16T18:36:46.635469] [INFO ] [MAIN ] GEO: Full history saved. [2025-09-16T18:36:46.635569] [INFO ] [MAIN ] Update Progress: 914/2369 symbols completed. GEO took 28.4s. Estimated remaining: 2h 33m 59s. Rate: 9.4 symbols/min [2025-09-16T18:36:46.779174] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-06. [2025-09-16T18:36:46.809202] [INFO ] [MAIN ] GF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:36:46.813120] [DEBUG] [LOCAL ] GF: Loaded 2108 existing records from primary storage. [2025-09-16T18:36:46.813196] [INFO ] [MAIN ] GF: Loaded 2108 existing records. [2025-09-16T18:36:46.991982] [WARN ] [MAIN ] GF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:46.992224] [INFO ] [MAIN ] GF: Wiping existing primary data before full refresh. [2025-09-16T18:36:46.992271] [INFO ] [LOCAL ] GF: Starting deletion of primary data. [2025-09-16T18:36:46.992449] [DEBUG] [LOCAL ] GF: Deleted primary data file. [2025-09-16T18:36:46.992491] [INFO ] [MAIN ] GF: Starting full primary refresh. [2025-09-16T18:36:47.350597] [INFO ] [REMOTE ] GF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:47.574892] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-02. [2025-09-16T18:36:48.298435] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T18:36:48.374920] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-08. [2025-09-16T18:36:48.436903] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T18:36:48.518930] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-05 (2.54 seconds). [2025-09-16T18:36:49.174907] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-06. [2025-09-16T18:36:49.814506] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T18:36:49.974998] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-07. [2025-09-16T18:36:50.774949] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-03. [2025-09-16T18:36:51.390101] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-09-16T18:36:51.433345] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-06 (2.26 seconds). [2025-09-16T18:36:51.569574] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T18:36:51.574834] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-04. [2025-09-16T18:36:52.130983] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-09-16T18:36:52.374908] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-05. [2025-09-16T18:36:53.022240] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-09-16T18:36:53.174883] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-06. [2025-09-16T18:36:53.661288] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-09-16T18:36:53.974910] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-09. [2025-09-16T18:36:54.774903] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-07. [2025-09-16T18:36:55.272149] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-09-16T18:36:55.284454] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T18:36:55.288725] [INFO ] [REMOTE ] GETY: Full fetch complete (40402 records). [2025-09-16T18:36:55.288830] [DEBUG] [VALIDATOR ] GETY: Validating primary data. [2025-09-16T18:36:55.574865] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-08. [2025-09-16T18:36:55.660545] [DEBUG] [VALIDATOR ] GETY: All expected trading days are present. [2025-09-16T18:36:55.660658] [DEBUG] [VALIDATOR ] GETY: Primary data validation passed. [2025-09-16T18:36:55.661294] [INFO ] [MAIN ] GETY: Fetched primary data is valid. [2025-09-16T18:36:55.661336] [DEBUG] [LOCAL ] GETY: Saving primary data. [2025-09-16T18:36:55.662196] [DEBUG] [LOCAL ] GETY: Trimmed 40402 records to 25881 (keeping 150 days of data) [2025-09-16T18:36:55.807542] [DEBUG] [LOCAL ] GETY: Primary data saved (25881 records). [2025-09-16T18:36:55.807693] [INFO ] [MAIN ] GETY: Full history saved. [2025-09-16T18:36:55.807780] [INFO ] [MAIN ] Update Progress: 915/2369 symbols completed. GETY took 24.4s. Estimated remaining: 2h 33m 57s. Rate: 9.4 symbols/min [2025-09-16T18:36:55.986147] [INFO ] [MAIN ] GFF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:36:56.007414] [DEBUG] [LOCAL ] GFF: Loaded 22267 existing records from primary storage. [2025-09-16T18:36:56.007553] [INFO ] [MAIN ] GFF: Loaded 22267 existing records. [2025-09-16T18:36:56.164524] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-09-16T18:36:56.190834] [WARN ] [MAIN ] GFF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:56.191082] [INFO ] [MAIN ] GFF: Wiping existing primary data before full refresh. [2025-09-16T18:36:56.191143] [INFO ] [LOCAL ] GFF: Starting deletion of primary data. [2025-09-16T18:36:56.191758] [DEBUG] [LOCAL ] GFF: Deleted primary data file. [2025-09-16T18:36:56.191817] [INFO ] [MAIN ] GFF: Starting full primary refresh. [2025-09-16T18:36:56.380198] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-09. [2025-09-16T18:36:56.549752] [INFO ] [REMOTE ] GFF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:57.052840] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-09-16T18:36:57.053671] [INFO ] [REMOTE ] GF: Full fetch complete (4010 records). [2025-09-16T18:36:57.053742] [DEBUG] [VALIDATOR ] GF: Validating primary data. [2025-09-16T18:36:57.179965] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-07. [2025-09-16T18:36:57.430688] [DEBUG] [VALIDATOR ] GF: All expected trading days are present. [2025-09-16T18:36:57.430821] [DEBUG] [VALIDATOR ] GF: Primary data validation passed. [2025-09-16T18:36:57.430993] [INFO ] [MAIN ] GF: Fetched primary data is valid. [2025-09-16T18:36:57.431037] [DEBUG] [LOCAL ] GF: Saving primary data. [2025-09-16T18:36:57.431451] [DEBUG] [LOCAL ] GF: Trimmed 4010 records to 2114 (keeping 150 days of data) [2025-09-16T18:36:57.444591] [DEBUG] [LOCAL ] GF: Primary data saved (2114 records). [2025-09-16T18:36:57.444680] [INFO ] [MAIN ] GF: Full history saved. [2025-09-16T18:36:57.444783] [INFO ] [MAIN ] Update Progress: 916/2369 symbols completed. GF took 10.8s. Estimated remaining: 2h 33m 43s. Rate: 9.5 symbols/min [2025-09-16T18:36:57.613018] [INFO ] [MAIN ] GFI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:36:57.651125] [DEBUG] [LOCAL ] GFI: Loaded 39110 existing records from primary storage. [2025-09-16T18:36:57.651299] [INFO ] [MAIN ] GFI: Loaded 39110 existing records. [2025-09-16T18:36:57.832340] [WARN ] [MAIN ] GFI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:36:57.832568] [INFO ] [MAIN ] GFI: Wiping existing primary data before full refresh. [2025-09-16T18:36:57.832615] [INFO ] [LOCAL ] GFI: Starting deletion of primary data. [2025-09-16T18:36:57.833449] [DEBUG] [LOCAL ] GFI: Deleted primary data file. [2025-09-16T18:36:57.833525] [INFO ] [MAIN ] GFI: Starting full primary refresh. [2025-09-16T18:36:57.979960] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-08. [2025-09-16T18:36:58.198685] [INFO ] [REMOTE ] GFI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:36:58.774903] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-02. [2025-09-16T18:36:59.450491] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-07 (2.27 seconds). [2025-09-16T18:36:59.574891] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-08. [2025-09-16T18:36:59.954188] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-09-16T18:37:00.203816] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-09-16T18:37:00.374901] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-02. [2025-09-16T18:37:01.174914] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-09. [2025-09-16T18:37:01.980061] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-03. [2025-09-16T18:37:02.002156] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-08 (2.43 seconds). [2025-09-16T18:37:02.156973] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T18:37:02.774943] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-03. [2025-09-16T18:37:02.929470] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-09-16T18:37:02.934187] [INFO ] [REMOTE ] GES: Full fetch complete (44510 records). [2025-09-16T18:37:02.934313] [DEBUG] [VALIDATOR ] GES: Validating primary data. [2025-09-16T18:37:03.328820] [DEBUG] [VALIDATOR ] GES: All expected trading days are present. [2025-09-16T18:37:03.329041] [DEBUG] [VALIDATOR ] GES: Primary data validation passed. [2025-09-16T18:37:03.329816] [INFO ] [MAIN ] GES: Fetched primary data is valid. [2025-09-16T18:37:03.329886] [DEBUG] [LOCAL ] GES: Saving primary data. [2025-09-16T18:37:03.346700] [DEBUG] [LOCAL ] GES: Trimmed 44510 records to 28333 (keeping 150 days of data) [2025-09-16T18:37:03.351140] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T18:37:03.531210] [DEBUG] [LOCAL ] GES: Primary data saved (28333 records). [2025-09-16T18:37:03.531353] [INFO ] [MAIN ] GES: Full history saved. [2025-09-16T18:37:03.532670] [INFO ] [MAIN ] Update Progress: 917/2369 symbols completed. GES took 39.1s. Estimated remaining: 2h 33m 36s. Rate: 9.5 symbols/min [2025-09-16T18:37:03.574871] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-04. [2025-09-16T18:37:04.290911] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T18:37:04.374895] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-04. [2025-09-16T18:37:04.802266] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T18:37:05.174906] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-05. [2025-09-16T18:37:05.974937] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-09. [2025-09-16T18:37:06.112476] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-09-16T18:37:06.209208] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T18:37:06.774905] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-06. [2025-09-16T18:37:07.574914] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-05. [2025-09-16T18:37:07.663024] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T18:37:07.872838] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-09 (1.90 seconds). [2025-09-16T18:37:07.878679] [INFO ] [REMOTE ] GEV: Full fetch complete (57439 records). [2025-09-16T18:37:07.878807] [DEBUG] [VALIDATOR ] GEV: Validating primary data. [2025-09-16T18:37:07.879582] [FATAL] [VALIDATOR ] GEV: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:37:07.879706] [FATAL] [MAIN ] GEV: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:37:07.879743] [WARN ] [MAIN ] GEV: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:37:07.879809] [INFO ] [LOCAL ] GEV: Starting deletion of primary data. [2025-09-16T18:37:07.879883] [DEBUG] [LOCAL ] GEV: No primary data file found to delete. [2025-09-16T18:37:07.879923] [INFO ] [LOCAL ] GEV: Starting deletion of derivative data. [2025-09-16T18:37:07.880234] [DEBUG] [LOCAL ] GEV: Deleted derivative data file. [2025-09-16T18:37:07.881604] [INFO ] [MAIN ] Update Progress: 918/2369 symbols completed. GEV took 34.0s. Estimated remaining: 2h 33m 27s. Rate: 9.5 symbols/min [2025-09-16T18:37:08.374894] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-07. [2025-09-16T18:37:09.278823] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-09-16T18:37:09.284971] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-08. [2025-09-16T18:37:09.570459] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-09-16T18:37:09.974956] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-06. [2025-09-16T18:37:10.531561] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-09-16T18:37:10.774916] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-09. [2025-09-16T18:37:11.540167] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T18:37:11.574852] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-07. [2025-09-16T18:37:11.864322] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T18:37:11.867862] [INFO ] [REMOTE ] GFF: Full fetch complete (33758 records). [2025-09-16T18:37:11.867955] [DEBUG] [VALIDATOR ] GFF: Validating primary data. [2025-09-16T18:37:11.868423] [FATAL] [VALIDATOR ] GFF: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:37:11.868499] [FATAL] [MAIN ] GFF: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:37:11.868535] [WARN ] [MAIN ] GFF: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:37:11.868572] [INFO ] [LOCAL ] GFF: Starting deletion of primary data. [2025-09-16T18:37:11.868628] [DEBUG] [LOCAL ] GFF: No primary data file found to delete. [2025-09-16T18:37:11.868689] [INFO ] [LOCAL ] GFF: Starting deletion of derivative data. [2025-09-16T18:37:11.868951] [DEBUG] [LOCAL ] GFF: Deleted derivative data file. [2025-09-16T18:37:11.869028] [INFO ] [MAIN ] Update Progress: 919/2369 symbols completed. GFF took 16.1s. Estimated remaining: 2h 33m 17s. Rate: 9.5 symbols/min [2025-09-16T18:37:13.363454] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-09-16T18:37:13.375117] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-08. [2025-09-16T18:37:15.066862] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T18:37:15.078220] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-09. [2025-09-16T18:37:16.601191] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T18:37:16.607700] [INFO ] [REMOTE ] GFI: Full fetch complete (57730 records). [2025-09-16T18:37:16.607806] [DEBUG] [VALIDATOR ] GFI: Validating primary data. [2025-09-16T18:37:16.994782] [DEBUG] [VALIDATOR ] GFI: All expected trading days are present. [2025-09-16T18:37:16.994881] [DEBUG] [VALIDATOR ] GFI: Primary data validation passed. [2025-09-16T18:37:16.995828] [INFO ] [MAIN ] GFI: Fetched primary data is valid. [2025-09-16T18:37:16.995881] [DEBUG] [LOCAL ] GFI: Saving primary data. [2025-09-16T18:37:16.998023] [DEBUG] [LOCAL ] GFI: Trimmed 57730 records to 39500 (keeping 150 days of data) [2025-09-16T18:37:17.241231] [DEBUG] [LOCAL ] GFI: Primary data saved (39500 records). [2025-09-16T18:37:17.241388] [INFO ] [MAIN ] GFI: Full history saved. [2025-09-16T18:37:17.241488] [INFO ] [MAIN ] Update Progress: 920/2369 symbols completed. GFI took 19.8s. Estimated remaining: 2h 33m 9s. Rate: 9.5 symbols/min [2025-09-16T18:37:17.834965] [INFO ] [MAIN ] GFR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:17.950550] [INFO ] [MAIN ] GFL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:17.953158] [INFO ] [MAIN ] GGG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:17.994391] [INFO ] [MAIN ] GGB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:18.009136] [DEBUG] [LOCAL ] GFR: Loaded 5085 existing records from primary storage. [2025-09-16T18:37:18.009449] [INFO ] [MAIN ] GFR: Loaded 5085 existing records. [2025-09-16T18:37:18.237554] [DEBUG] [LOCAL ] GGG: Loaded 29577 existing records from primary storage. [2025-09-16T18:37:18.255330] [INFO ] [MAIN ] GGG: Loaded 29577 existing records. [2025-09-16T18:37:18.260269] [DEBUG] [LOCAL ] GFL: Loaded 34741 existing records from primary storage. [2025-09-16T18:37:18.254328] [DEBUG] [LOCAL ] GGB: Loaded 37887 existing records from primary storage. [2025-09-16T18:37:18.259602] [WARN ] [MAIN ] GFR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:37:18.290299] [INFO ] [MAIN ] GFL: Loaded 34741 existing records. [2025-09-16T18:37:18.301021] [INFO ] [MAIN ] GFR: Wiping existing primary data before full refresh. [2025-09-16T18:37:18.326682] [INFO ] [LOCAL ] GFR: Starting deletion of primary data. [2025-09-16T18:37:18.326756] [INFO ] [MAIN ] GGB: Loaded 37887 existing records. [2025-09-16T18:37:18.359376] [DEBUG] [LOCAL ] GFR: Deleted primary data file. [2025-09-16T18:37:18.423263] [INFO ] [MAIN ] GFR: Starting full primary refresh. [2025-09-16T18:37:18.808177] [WARN ] [MAIN ] GFL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:37:18.850255] [INFO ] [MAIN ] GFL: Wiping existing primary data before full refresh. [2025-09-16T18:37:18.882019] [WARN ] [MAIN ] GGG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:37:18.916097] [INFO ] [LOCAL ] GFL: Starting deletion of primary data. [2025-09-16T18:37:18.927175] [INFO ] [MAIN ] GGG: Wiping existing primary data before full refresh. [2025-09-16T18:37:18.927240] [DEBUG] [LOCAL ] GFL: Deleted primary data file. [2025-09-16T18:37:18.921285] [WARN ] [MAIN ] GGB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:37:18.937746] [INFO ] [LOCAL ] GGG: Starting deletion of primary data. [2025-09-16T18:37:18.943009] [INFO ] [MAIN ] GFL: Starting full primary refresh. [2025-09-16T18:37:18.988579] [DEBUG] [LOCAL ] GGG: Deleted primary data file. [2025-09-16T18:37:19.034820] [INFO ] [MAIN ] GGG: Starting full primary refresh. [2025-09-16T18:37:19.016473] [INFO ] [MAIN ] GGB: Wiping existing primary data before full refresh. [2025-09-16T18:37:19.135850] [INFO ] [LOCAL ] GGB: Starting deletion of primary data. [2025-09-16T18:37:19.249454] [DEBUG] [LOCAL ] GGB: Deleted primary data file. [2025-09-16T18:37:19.311100] [INFO ] [MAIN ] GGB: Starting full primary refresh. [2025-09-16T18:37:19.625828] [INFO ] [REMOTE ] GFR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:19.669338] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-02. [2025-09-16T18:37:20.101982] [INFO ] [REMOTE ] GGG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:20.130086] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-02. [2025-09-16T18:37:20.188873] [INFO ] [REMOTE ] GFL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:20.189387] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-02. [2025-09-16T18:37:20.266531] [INFO ] [REMOTE ] GGB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:20.266847] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-02. [2025-09-16T18:37:20.475768] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-09-16T18:37:20.478064] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-03. [2025-09-16T18:37:21.463603] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T18:37:21.481224] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-09-16T18:37:21.481805] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-03. [2025-09-16T18:37:21.487845] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-04. [2025-09-16T18:37:21.691175] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T18:37:21.701009] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-03. [2025-09-16T18:37:21.998426] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-09-16T18:37:22.003906] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-05. [2025-09-16T18:37:22.078708] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-09-16T18:37:22.086722] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-03. [2025-09-16T18:37:22.583655] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-09-16T18:37:22.586530] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-06. [2025-09-16T18:37:23.033683] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T18:37:23.044261] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-04. [2025-09-16T18:37:23.124753] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T18:37:23.282532] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T18:37:23.574945] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-04. [2025-09-16T18:37:23.697959] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-09-16T18:37:24.374868] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-04. [2025-09-16T18:37:24.504543] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T18:37:24.982817] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-09-16T18:37:25.174912] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-05. [2025-09-16T18:37:25.857103] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T18:37:25.974908] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-05. [2025-09-16T18:37:26.320023] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T18:37:26.774912] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-07. [2025-09-16T18:37:27.199096] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-09-16T18:37:27.244782] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T18:37:27.574921] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-06. [2025-09-16T18:37:28.374974] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-05. [2025-09-16T18:37:28.819814] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T18:37:29.174961] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-06. [2025-09-16T18:37:29.729267] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T18:37:29.974901] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-08. [2025-09-16T18:37:30.369220] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-09-16T18:37:30.452207] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T18:37:30.774953] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-07. [2025-09-16T18:37:31.574900] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-06. [2025-09-16T18:37:32.118340] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-09-16T18:37:32.374886] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-07. [2025-09-16T18:37:33.029160] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T18:37:33.174911] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-09. [2025-09-16T18:37:33.974860] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-07. [2025-09-16T18:37:34.032458] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T18:37:34.033592] [INFO ] [REMOTE ] GFR: Full fetch complete (9141 records). [2025-09-16T18:37:34.033703] [DEBUG] [VALIDATOR ] GFR: Validating primary data. [2025-09-16T18:37:34.356838] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-09-16T18:37:34.446989] [DEBUG] [VALIDATOR ] GFR: All expected trading days are present. [2025-09-16T18:37:34.447069] [DEBUG] [VALIDATOR ] GFR: Primary data validation passed. [2025-09-16T18:37:34.447311] [INFO ] [MAIN ] GFR: Fetched primary data is valid. [2025-09-16T18:37:34.447366] [DEBUG] [LOCAL ] GFR: Saving primary data. [2025-09-16T18:37:34.447808] [DEBUG] [LOCAL ] GFR: Trimmed 9141 records to 5148 (keeping 150 days of data) [2025-09-16T18:37:34.475501] [DEBUG] [LOCAL ] GFR: Primary data saved (5148 records). [2025-09-16T18:37:34.475577] [INFO ] [MAIN ] GFR: Full history saved. [2025-09-16T18:37:34.475684] [INFO ] [MAIN ] Update Progress: 921/2369 symbols completed. GFR took 17.2s. Estimated remaining: 2h 33m 20s. Rate: 9.4 symbols/min [2025-09-16T18:37:34.651172] [INFO ] [MAIN ] GGT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:34.663343] [DEBUG] [LOCAL ] GGT: Loaded 11031 existing records from primary storage. [2025-09-16T18:37:34.663461] [INFO ] [MAIN ] GGT: Loaded 11031 existing records. [2025-09-16T18:37:34.779965] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-08. [2025-09-16T18:37:34.842243] [WARN ] [MAIN ] GGT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:37:34.842448] [INFO ] [MAIN ] GGT: Wiping existing primary data before full refresh. [2025-09-16T18:37:34.842494] [INFO ] [LOCAL ] GGT: Starting deletion of primary data. [2025-09-16T18:37:34.842858] [DEBUG] [LOCAL ] GGT: Deleted primary data file. [2025-09-16T18:37:34.842907] [INFO ] [MAIN ] GGT: Starting full primary refresh. [2025-09-16T18:37:35.191135] [INFO ] [REMOTE ] GGT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:35.574897] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-08. [2025-09-16T18:37:35.682976] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T18:37:36.334379] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T18:37:36.374867] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-02. [2025-09-16T18:37:36.795481] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-09-16T18:37:37.174897] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-09. [2025-09-16T18:37:37.226351] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-09-16T18:37:37.974899] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-03. [2025-09-16T18:37:38.143743] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T18:37:38.148408] [INFO ] [REMOTE ] GGG: Full fetch complete (44559 records). [2025-09-16T18:37:38.148500] [DEBUG] [VALIDATOR ] GGG: Validating primary data. [2025-09-16T18:37:38.554785] [DEBUG] [VALIDATOR ] GGG: All expected trading days are present. [2025-09-16T18:37:38.554874] [DEBUG] [VALIDATOR ] GGG: Primary data validation passed. [2025-09-16T18:37:38.555606] [INFO ] [MAIN ] GGG: Fetched primary data is valid. [2025-09-16T18:37:38.555669] [DEBUG] [LOCAL ] GGG: Saving primary data. [2025-09-16T18:37:38.556768] [DEBUG] [LOCAL ] GGG: Trimmed 44559 records to 29913 (keeping 150 days of data) [2025-09-16T18:37:38.770801] [DEBUG] [LOCAL ] GGG: Primary data saved (29913 records). [2025-09-16T18:37:38.771062] [INFO ] [MAIN ] GGG: Full history saved. [2025-09-16T18:37:38.771235] [INFO ] [MAIN ] Update Progress: 922/2369 symbols completed. GGG took 21.5s. Estimated remaining: 2h 33m 10s. Rate: 9.4 symbols/min [2025-09-16T18:37:38.782919] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-09. [2025-09-16T18:37:38.860328] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-09-16T18:37:38.962422] [INFO ] [MAIN ] GGZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:38.965919] [DEBUG] [LOCAL ] GGZ: Loaded 1696 existing records from primary storage. [2025-09-16T18:37:38.965994] [INFO ] [MAIN ] GGZ: Loaded 1696 existing records. [2025-09-16T18:37:39.139529] [WARN ] [MAIN ] GGZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:37:39.139781] [INFO ] [MAIN ] GGZ: Wiping existing primary data before full refresh. [2025-09-16T18:37:39.139827] [INFO ] [LOCAL ] GGZ: Starting deletion of primary data. [2025-09-16T18:37:39.140004] [DEBUG] [LOCAL ] GGZ: Deleted primary data file. [2025-09-16T18:37:39.140047] [INFO ] [MAIN ] GGZ: Starting full primary refresh. [2025-09-16T18:37:39.489885] [INFO ] [REMOTE ] GGZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:39.574886] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-04. [2025-09-16T18:37:40.334138] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-09-16T18:37:40.340032] [INFO ] [REMOTE ] GFL: Full fetch complete (52371 records). [2025-09-16T18:37:40.340178] [DEBUG] [VALIDATOR ] GFL: Validating primary data. [2025-09-16T18:37:40.383977] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-02. [2025-09-16T18:37:40.515762] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-09-16T18:37:40.738084] [DEBUG] [VALIDATOR ] GFL: All expected trading days are present. [2025-09-16T18:37:40.738169] [DEBUG] [VALIDATOR ] GFL: Primary data validation passed. [2025-09-16T18:37:40.738941] [INFO ] [MAIN ] GFL: Fetched primary data is valid. [2025-09-16T18:37:40.738989] [DEBUG] [LOCAL ] GFL: Saving primary data. [2025-09-16T18:37:40.740187] [DEBUG] [LOCAL ] GFL: Trimmed 52371 records to 35126 (keeping 150 days of data) [2025-09-16T18:37:40.958962] [DEBUG] [LOCAL ] GFL: Primary data saved (35126 records). [2025-09-16T18:37:40.959149] [INFO ] [MAIN ] GFL: Full history saved. [2025-09-16T18:37:40.960082] [INFO ] [MAIN ] Update Progress: 923/2369 symbols completed. GFL took 23.7s. Estimated remaining: 2h 32m 57s. Rate: 9.5 symbols/min [2025-09-16T18:37:41.143781] [INFO ] [MAIN ] GHC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:41.148598] [DEBUG] [LOCAL ] GHC: Loaded 2054 existing records from primary storage. [2025-09-16T18:37:41.148729] [INFO ] [MAIN ] GHC: Loaded 2054 existing records. [2025-09-16T18:37:41.180034] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-05. [2025-09-16T18:37:41.239781] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T18:37:41.332402] [WARN ] [MAIN ] GHC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:37:41.332626] [INFO ] [MAIN ] GHC: Wiping existing primary data before full refresh. [2025-09-16T18:37:41.332693] [INFO ] [LOCAL ] GHC: Starting deletion of primary data. [2025-09-16T18:37:41.332903] [DEBUG] [LOCAL ] GHC: Deleted primary data file. [2025-09-16T18:37:41.332957] [INFO ] [MAIN ] GHC: Starting full primary refresh. [2025-09-16T18:37:41.696240] [INFO ] [REMOTE ] GHC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:41.974903] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-03. [2025-09-16T18:37:42.240077] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-09-16T18:37:42.442724] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-09-16T18:37:42.774935] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-02. [2025-09-16T18:37:43.574910] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-08. [2025-09-16T18:37:43.700105] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-09-16T18:37:44.374925] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-03. [2025-09-16T18:37:44.851081] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-09-16T18:37:45.174861] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-04. [2025-09-16T18:37:45.221289] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T18:37:45.711260] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-09-16T18:37:45.974944] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-04. [2025-09-16T18:37:46.503729] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-09-16T18:37:46.774911] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-05. [2025-09-16T18:37:47.394057] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-09-16T18:37:47.574900] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-06. [2025-09-16T18:37:48.374884] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-09. [2025-09-16T18:37:48.626793] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T18:37:49.174917] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-05. [2025-09-16T18:37:49.662807] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-09-16T18:37:49.894112] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T18:37:49.900094] [INFO ] [REMOTE ] GGB: Full fetch complete (55453 records). [2025-09-16T18:37:49.900198] [DEBUG] [VALIDATOR ] GGB: Validating primary data. [2025-09-16T18:37:49.980029] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-06. [2025-09-16T18:37:50.300557] [DEBUG] [VALIDATOR ] GGB: All expected trading days are present. [2025-09-16T18:37:50.300658] [DEBUG] [VALIDATOR ] GGB: Primary data validation passed. [2025-09-16T18:37:50.301394] [INFO ] [MAIN ] GGB: Fetched primary data is valid. [2025-09-16T18:37:50.301437] [DEBUG] [LOCAL ] GGB: Saving primary data. [2025-09-16T18:37:50.302540] [DEBUG] [LOCAL ] GGB: Trimmed 55453 records to 38267 (keeping 150 days of data) [2025-09-16T18:37:50.524935] [DEBUG] [LOCAL ] GGB: Primary data saved (38267 records). [2025-09-16T18:37:50.525071] [INFO ] [MAIN ] GGB: Full history saved. [2025-09-16T18:37:50.525165] [INFO ] [MAIN ] Update Progress: 924/2369 symbols completed. GGB took 33.3s. Estimated remaining: 2h 32m 56s. Rate: 9.4 symbols/min [2025-09-16T18:37:50.704458] [INFO ] [MAIN ] GHG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:50.704843] [DEBUG] [LOCAL ] GHG: No existing data file found in primary storage. [2025-09-16T18:37:50.708505] [WARN ] [MAIN ] GHG: No existing data found, performing full refresh instead. [2025-09-16T18:37:50.709389] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-09-16T18:37:50.709546] [INFO ] [MAIN ] GHG: Wiping existing primary data before full refresh. [2025-09-16T18:37:50.711057] [INFO ] [LOCAL ] GHG: Starting deletion of primary data. [2025-09-16T18:37:50.711338] [DEBUG] [LOCAL ] GHG: No primary data file found to delete. [2025-09-16T18:37:50.711374] [INFO ] [MAIN ] GHG: Starting full primary refresh. [2025-09-16T18:37:50.780003] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-07. [2025-09-16T18:37:51.076744] [INFO ] [REMOTE ] GHG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:51.412901] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-09-16T18:37:51.574866] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-08. [2025-09-16T18:37:52.069619] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-09-16T18:37:52.374925] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-06. [2025-09-16T18:37:52.883462] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-09-16T18:37:53.174913] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-09. [2025-09-16T18:37:53.745951] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-09-16T18:37:53.746851] [INFO ] [REMOTE ] GHC: Full fetch complete (2700 records). [2025-09-16T18:37:53.746918] [DEBUG] [VALIDATOR ] GHC: Validating primary data. [2025-09-16T18:37:53.983773] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-07. [2025-09-16T18:37:54.100668] [DEBUG] [VALIDATOR ] GHC: All expected trading days are present. [2025-09-16T18:37:54.100754] [DEBUG] [VALIDATOR ] GHC: Primary data validation passed. [2025-09-16T18:37:54.100860] [INFO ] [MAIN ] GHC: Fetched primary data is valid. [2025-09-16T18:37:54.100899] [DEBUG] [LOCAL ] GHC: Saving primary data. [2025-09-16T18:37:54.101231] [DEBUG] [LOCAL ] GHC: Trimmed 2700 records to 2082 (keeping 150 days of data) [2025-09-16T18:37:54.114879] [DEBUG] [LOCAL ] GHC: Primary data saved (2082 records). [2025-09-16T18:37:54.114943] [INFO ] [MAIN ] GHC: Full history saved. [2025-09-16T18:37:54.115051] [INFO ] [MAIN ] Update Progress: 925/2369 symbols completed. GHC took 13.2s. Estimated remaining: 2h 32m 45s. Rate: 9.5 symbols/min [2025-09-16T18:37:54.286531] [INFO ] [MAIN ] GHI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:54.294479] [DEBUG] [LOCAL ] GHI: Loaded 5230 existing records from primary storage. [2025-09-16T18:37:54.294620] [INFO ] [MAIN ] GHI: Loaded 5230 existing records. [2025-09-16T18:37:54.475693] [WARN ] [MAIN ] GHI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:37:54.478675] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-09-16T18:37:54.478552] [INFO ] [MAIN ] GHI: Wiping existing primary data before full refresh. [2025-09-16T18:37:54.478806] [INFO ] [LOCAL ] GHI: Starting deletion of primary data. [2025-09-16T18:37:54.480202] [DEBUG] [LOCAL ] GHI: Deleted primary data file. [2025-09-16T18:37:54.480253] [INFO ] [MAIN ] GHI: Starting full primary refresh. [2025-09-16T18:37:54.780035] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-08. [2025-09-16T18:37:54.831147] [INFO ] [REMOTE ] GHI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:55.331010] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-09-16T18:37:55.574920] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-07. [2025-09-16T18:37:56.374892] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-09. [2025-09-16T18:37:56.467343] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-09-16T18:37:56.804973] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-09-16T18:37:56.805855] [INFO ] [REMOTE ] GGZ: Full fetch complete (2720 records). [2025-09-16T18:37:56.805941] [DEBUG] [VALIDATOR ] GGZ: Validating primary data. [2025-09-16T18:37:57.175381] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-08. [2025-09-16T18:37:57.176560] [DEBUG] [VALIDATOR ] GGZ: All expected trading days are present. [2025-09-16T18:37:57.176675] [DEBUG] [VALIDATOR ] GGZ: Primary data validation passed. [2025-09-16T18:37:57.176785] [INFO ] [MAIN ] GGZ: Fetched primary data is valid. [2025-09-16T18:37:57.176825] [DEBUG] [LOCAL ] GGZ: Saving primary data. [2025-09-16T18:37:57.177171] [DEBUG] [LOCAL ] GGZ: Trimmed 2720 records to 1704 (keeping 150 days of data) [2025-09-16T18:37:57.188561] [DEBUG] [LOCAL ] GGZ: Primary data saved (1704 records). [2025-09-16T18:37:57.188620] [INFO ] [MAIN ] GGZ: Full history saved. [2025-09-16T18:37:57.188731] [INFO ] [MAIN ] Update Progress: 926/2369 symbols completed. GGZ took 18.4s. Estimated remaining: 2h 32m 34s. Rate: 9.5 symbols/min [2025-09-16T18:37:57.361421] [INFO ] [MAIN ] GHLD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:37:57.366229] [DEBUG] [LOCAL ] GHLD: Loaded 3305 existing records from primary storage. [2025-09-16T18:37:57.366303] [INFO ] [MAIN ] GHLD: Loaded 3305 existing records. [2025-09-16T18:37:57.540402] [WARN ] [MAIN ] GHLD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:37:57.540635] [INFO ] [MAIN ] GHLD: Wiping existing primary data before full refresh. [2025-09-16T18:37:57.540708] [INFO ] [LOCAL ] GHLD: Starting deletion of primary data. [2025-09-16T18:37:57.540947] [DEBUG] [LOCAL ] GHLD: Deleted primary data file. [2025-09-16T18:37:57.540991] [INFO ] [MAIN ] GHLD: Starting full primary refresh. [2025-09-16T18:37:57.887554] [INFO ] [REMOTE ] GHLD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:37:57.974854] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-02. [2025-09-16T18:37:58.146351] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-09-16T18:37:58.429541] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-09-16T18:37:58.774918] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-02. [2025-09-16T18:37:59.305764] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-09-16T18:37:59.574946] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-03. [2025-09-16T18:38:00.066546] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-09-16T18:38:00.374889] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-09. [2025-09-16T18:38:01.174951] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-02. [2025-09-16T18:38:01.503962] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T18:38:01.506361] [INFO ] [REMOTE ] GGT: Full fetch complete (14989 records). [2025-09-16T18:38:01.506553] [DEBUG] [VALIDATOR ] GGT: Validating primary data. [2025-09-16T18:38:01.884607] [DEBUG] [VALIDATOR ] GGT: All expected trading days are present. [2025-09-16T18:38:01.884773] [DEBUG] [VALIDATOR ] GGT: Primary data validation passed. [2025-09-16T18:38:01.885404] [INFO ] [MAIN ] GGT: Fetched primary data is valid. [2025-09-16T18:38:01.885511] [DEBUG] [LOCAL ] GGT: Saving primary data. [2025-09-16T18:38:01.886620] [DEBUG] [LOCAL ] GGT: Trimmed 14989 records to 11129 (keeping 150 days of data) [2025-09-16T18:38:01.960619] [DEBUG] [LOCAL ] GGT: Primary data saved (11129 records). [2025-09-16T18:38:01.960811] [INFO ] [MAIN ] GGT: Full history saved. [2025-09-16T18:38:01.961074] [INFO ] [MAIN ] Update Progress: 927/2369 symbols completed. GGT took 27.5s. Estimated remaining: 2h 32m 25s. Rate: 9.5 symbols/min [2025-09-16T18:38:01.982479] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-04. [2025-09-16T18:38:02.213586] [INFO ] [MAIN ] GHM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:38:02.225517] [DEBUG] [LOCAL ] GHM: Loaded 10420 existing records from primary storage. [2025-09-16T18:38:02.225634] [INFO ] [MAIN ] GHM: Loaded 10420 existing records. [2025-09-16T18:38:02.408493] [WARN ] [MAIN ] GHM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:38:02.408808] [INFO ] [MAIN ] GHM: Wiping existing primary data before full refresh. [2025-09-16T18:38:02.408856] [INFO ] [LOCAL ] GHM: Starting deletion of primary data. [2025-09-16T18:38:02.409274] [DEBUG] [LOCAL ] GHM: Deleted primary data file. [2025-09-16T18:38:02.409321] [INFO ] [MAIN ] GHM: Starting full primary refresh. [2025-09-16T18:38:02.542634] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T18:38:02.700256] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-09-16T18:38:02.775069] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-05. [2025-09-16T18:38:02.778226] [INFO ] [REMOTE ] GHM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:38:03.357490] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-09-16T18:38:03.574884] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-03. [2025-09-16T18:38:04.314750] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-09-16T18:38:04.374906] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-06. [2025-09-16T18:38:04.835831] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-09-16T18:38:05.174913] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-03. [2025-09-16T18:38:05.712443] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-09-16T18:38:05.974953] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-07. [2025-09-16T18:38:06.582483] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-09-16T18:38:06.774916] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-02. [2025-09-16T18:38:07.574906] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-04. [2025-09-16T18:38:08.037582] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-09-16T18:38:08.320934] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T18:38:08.374849] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-03. [2025-09-16T18:38:09.174891] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-08. [2025-09-16T18:38:09.428156] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-09-16T18:38:09.728675] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-09-16T18:38:09.974928] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-04. [2025-09-16T18:38:10.774898] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-04. [2025-09-16T18:38:10.868220] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-09-16T18:38:11.441533] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-09-16T18:38:11.574931] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-05. [2025-09-16T18:38:11.975499] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-09-16T18:38:12.374924] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-09. [2025-09-16T18:38:13.174919] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-05. [2025-09-16T18:38:13.273740] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-09-16T18:38:13.274831] [INFO ] [REMOTE ] GHG: Full fetch complete (3333 records). [2025-09-16T18:38:13.274904] [DEBUG] [VALIDATOR ] GHG: Validating primary data. [2025-09-16T18:38:13.275276] [FATAL] [VALIDATOR ] GHG: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:38:13.275359] [FATAL] [MAIN ] GHG: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:38:13.275397] [WARN ] [MAIN ] GHG: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:38:13.275437] [INFO ] [LOCAL ] GHG: Starting deletion of primary data. [2025-09-16T18:38:13.275494] [DEBUG] [LOCAL ] GHG: No primary data file found to delete. [2025-09-16T18:38:13.275532] [INFO ] [LOCAL ] GHG: Starting deletion of derivative data. [2025-09-16T18:38:13.275604] [DEBUG] [LOCAL ] GHG: No derivative data file found to delete. [2025-09-16T18:38:13.275716] [INFO ] [MAIN ] Update Progress: 928/2369 symbols completed. GHG took 22.8s. Estimated remaining: 2h 32m 26s. Rate: 9.5 symbols/min [2025-09-16T18:38:13.447231] [INFO ] [MAIN ] GHY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:38:13.466091] [DEBUG] [LOCAL ] GHY: Loaded 14869 existing records from primary storage. [2025-09-16T18:38:13.466224] [INFO ] [MAIN ] GHY: Loaded 14869 existing records. [2025-09-16T18:38:13.647071] [WARN ] [MAIN ] GHY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:38:13.647325] [INFO ] [MAIN ] GHY: Wiping existing primary data before full refresh. [2025-09-16T18:38:13.647376] [INFO ] [LOCAL ] GHY: Starting deletion of primary data. [2025-09-16T18:38:13.647863] [DEBUG] [LOCAL ] GHY: Deleted primary data file. [2025-09-16T18:38:13.647912] [INFO ] [MAIN ] GHY: Starting full primary refresh. [2025-09-16T18:38:13.784162] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-09-16T18:38:13.979991] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-05. [2025-09-16T18:38:14.019505] [INFO ] [REMOTE ] GHY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:38:14.598487] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-09-16T18:38:14.774905] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-06. [2025-09-16T18:38:15.263023] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-09-16T18:38:15.574903] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-07. [2025-09-16T18:38:16.374927] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-06. [2025-09-16T18:38:16.427036] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-09-16T18:38:16.825823] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-09-16T18:38:17.174934] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-08. [2025-09-16T18:38:17.974904] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-02. [2025-09-16T18:38:18.053539] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-09-16T18:38:18.774910] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-06. [2025-09-16T18:38:19.203816] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-09-16T18:38:19.574893] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-09. [2025-09-16T18:38:19.865331] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-09-16T18:38:20.321833] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-09-16T18:38:20.323125] [INFO ] [REMOTE ] GHI: Full fetch complete (7572 records). [2025-09-16T18:38:20.323222] [DEBUG] [VALIDATOR ] GHI: Validating primary data. [2025-09-16T18:38:20.380270] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-07. [2025-09-16T18:38:20.699486] [DEBUG] [VALIDATOR ] GHI: All expected trading days are present. [2025-09-16T18:38:20.699567] [DEBUG] [VALIDATOR ] GHI: Primary data validation passed. [2025-09-16T18:38:20.699776] [INFO ] [MAIN ] GHI: Fetched primary data is valid. [2025-09-16T18:38:20.699819] [DEBUG] [LOCAL ] GHI: Saving primary data. [2025-09-16T18:38:20.700229] [DEBUG] [LOCAL ] GHI: Trimmed 7572 records to 5260 (keeping 150 days of data) [2025-09-16T18:38:20.733446] [DEBUG] [LOCAL ] GHI: Primary data saved (5260 records). [2025-09-16T18:38:20.733582] [INFO ] [MAIN ] GHI: Full history saved. [2025-09-16T18:38:20.733713] [INFO ] [MAIN ] Update Progress: 929/2369 symbols completed. GHI took 26.6s. Estimated remaining: 2h 32m 22s. Rate: 9.5 symbols/min [2025-09-16T18:38:20.904485] [INFO ] [MAIN ] GIB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:38:20.928028] [DEBUG] [LOCAL ] GIB: Loaded 22652 existing records from primary storage. [2025-09-16T18:38:20.928174] [INFO ] [MAIN ] GIB: Loaded 22652 existing records. [2025-09-16T18:38:21.107676] [WARN ] [MAIN ] GIB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:38:21.107903] [INFO ] [MAIN ] GIB: Wiping existing primary data before full refresh. [2025-09-16T18:38:21.107969] [INFO ] [LOCAL ] GIB: Starting deletion of primary data. [2025-09-16T18:38:21.108574] [DEBUG] [LOCAL ] GIB: Deleted primary data file. [2025-09-16T18:38:21.108619] [INFO ] [MAIN ] GIB: Starting full primary refresh. [2025-09-16T18:38:21.181922] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-03. [2025-09-16T18:38:21.461978] [INFO ] [REMOTE ] GIB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:38:21.485257] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-09-16T18:38:21.974901] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-07. [2025-09-16T18:38:22.121278] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T18:38:22.682342] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-09-16T18:38:22.774890] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-08. [2025-09-16T18:38:23.251467] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-09-16T18:38:23.574944] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-02. [2025-09-16T18:38:24.374931] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-04. [2025-09-16T18:38:25.138818] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T18:38:25.174876] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-08. [2025-09-16T18:38:25.557210] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T18:38:25.974915] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-09. [2025-09-16T18:38:26.133249] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-09-16T18:38:26.774914] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-09. [2025-09-16T18:38:26.898019] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T18:38:26.899196] [INFO ] [REMOTE ] GHLD: Full fetch complete (4082 records). [2025-09-16T18:38:26.899264] [DEBUG] [VALIDATOR ] GHLD: Validating primary data. [2025-09-16T18:38:27.265840] [DEBUG] [VALIDATOR ] GHLD: All expected trading days are present. [2025-09-16T18:38:27.265917] [DEBUG] [VALIDATOR ] GHLD: Primary data validation passed. [2025-09-16T18:38:27.266050] [INFO ] [MAIN ] GHLD: Fetched primary data is valid. [2025-09-16T18:38:27.266089] [DEBUG] [LOCAL ] GHLD: Saving primary data. [2025-09-16T18:38:27.266463] [DEBUG] [LOCAL ] GHLD: Trimmed 4082 records to 3340 (keeping 150 days of data) [2025-09-16T18:38:27.287828] [DEBUG] [LOCAL ] GHLD: Primary data saved (3340 records). [2025-09-16T18:38:27.287902] [INFO ] [MAIN ] GHLD: Full history saved. [2025-09-16T18:38:27.287990] [INFO ] [MAIN ] Update Progress: 930/2369 symbols completed. GHLD took 30.1s. Estimated remaining: 2h 32m 16s. Rate: 9.5 symbols/min [2025-09-16T18:38:27.458707] [INFO ] [MAIN ] GIC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:38:27.458927] [DEBUG] [LOCAL ] GIC: No existing data file found in primary storage. [2025-09-16T18:38:27.458965] [WARN ] [MAIN ] GIC: No existing data found, performing full refresh instead. [2025-09-16T18:38:27.459082] [INFO ] [MAIN ] GIC: Wiping existing primary data before full refresh. [2025-09-16T18:38:27.459121] [INFO ] [LOCAL ] GIC: Starting deletion of primary data. [2025-09-16T18:38:27.459161] [DEBUG] [LOCAL ] GIC: No primary data file found to delete. [2025-09-16T18:38:27.459194] [INFO ] [MAIN ] GIC: Starting full primary refresh. [2025-09-16T18:38:27.580053] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-03. [2025-09-16T18:38:27.816519] [INFO ] [REMOTE ] GIC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:38:27.823202] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T18:38:27.825497] [INFO ] [REMOTE ] GHM: Full fetch complete (16146 records). [2025-09-16T18:38:27.825573] [DEBUG] [VALIDATOR ] GHM: Validating primary data. [2025-09-16T18:38:28.189751] [DEBUG] [VALIDATOR ] GHM: All expected trading days are present. [2025-09-16T18:38:28.189838] [DEBUG] [VALIDATOR ] GHM: Primary data validation passed. [2025-09-16T18:38:28.190266] [INFO ] [MAIN ] GHM: Fetched primary data is valid. [2025-09-16T18:38:28.190308] [DEBUG] [LOCAL ] GHM: Saving primary data. [2025-09-16T18:38:28.190931] [DEBUG] [LOCAL ] GHM: Trimmed 16146 records to 10550 (keeping 150 days of data) [2025-09-16T18:38:28.256198] [DEBUG] [LOCAL ] GHM: Primary data saved (10550 records). [2025-09-16T18:38:28.256350] [INFO ] [MAIN ] GHM: Full history saved. [2025-09-16T18:38:28.256441] [INFO ] [MAIN ] Update Progress: 931/2369 symbols completed. GHM took 26.3s. Estimated remaining: 2h 32m 1s. Rate: 9.5 symbols/min [2025-09-16T18:38:28.379975] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-05. [2025-09-16T18:38:28.434030] [INFO ] [MAIN ] GIL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:38:28.475711] [DEBUG] [LOCAL ] GIL: Loaded 31665 existing records from primary storage. [2025-09-16T18:38:28.475919] [INFO ] [MAIN ] GIL: Loaded 31665 existing records. [2025-09-16T18:38:28.659031] [WARN ] [MAIN ] GIL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:38:28.659279] [INFO ] [MAIN ] GIL: Wiping existing primary data before full refresh. [2025-09-16T18:38:28.659327] [INFO ] [LOCAL ] GIL: Starting deletion of primary data. [2025-09-16T18:38:28.660264] [DEBUG] [LOCAL ] GIL: Deleted primary data file. [2025-09-16T18:38:28.660317] [INFO ] [MAIN ] GIL: Starting full primary refresh. [2025-09-16T18:38:29.012230] [INFO ] [REMOTE ] GIL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:38:29.110106] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T18:38:29.174864] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-04. [2025-09-16T18:38:29.541149] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-09-16T18:38:29.974899] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-02. [2025-09-16T18:38:30.662764] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T18:38:30.774898] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-05. [2025-09-16T18:38:31.175352] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-09-16T18:38:31.574911] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-03. [2025-09-16T18:38:31.762730] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T18:38:32.374890] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-06. [2025-09-16T18:38:32.508416] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-09-16T18:38:33.174889] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-04. [2025-09-16T18:38:33.623676] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T18:38:33.974885] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-06. [2025-09-16T18:38:34.133929] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-09-16T18:38:34.614255] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-09-16T18:38:34.774897] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-02. [2025-09-16T18:38:35.574906] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-05. [2025-09-16T18:38:36.161622] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-09-16T18:38:36.374892] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-07. [2025-09-16T18:38:36.410881] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T18:38:37.174874] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-06. [2025-09-16T18:38:37.436165] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-09-16T18:38:37.922136] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-09-16T18:38:37.974861] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-07. [2025-09-16T18:38:38.774928] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-07. [2025-09-16T18:38:38.832097] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-09-16T18:38:39.574893] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-08. [2025-09-16T18:38:39.835108] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-09-16T18:38:40.374866] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-08. [2025-09-16T18:38:40.736557] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T18:38:41.174865] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-03. [2025-09-16T18:38:41.616190] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T18:38:41.974891] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-09. [2025-09-16T18:38:42.703199] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T18:38:42.774970] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-09. [2025-09-16T18:38:43.099963] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T18:38:43.104198] [INFO ] [REMOTE ] GIB: Full fetch complete (35479 records). [2025-09-16T18:38:43.104369] [DEBUG] [VALIDATOR ] GIB: Validating primary data. [2025-09-16T18:38:43.486807] [DEBUG] [VALIDATOR ] GIB: All expected trading days are present. [2025-09-16T18:38:43.486902] [DEBUG] [VALIDATOR ] GIB: Primary data validation passed. [2025-09-16T18:38:43.487565] [INFO ] [MAIN ] GIB: Fetched primary data is valid. [2025-09-16T18:38:43.487611] [DEBUG] [LOCAL ] GIB: Saving primary data. [2025-09-16T18:38:43.488685] [DEBUG] [LOCAL ] GIB: Trimmed 35479 records to 22835 (keeping 150 days of data) [2025-09-16T18:38:43.584389] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-08. [2025-09-16T18:38:43.638935] [DEBUG] [LOCAL ] GIB: Primary data saved (22835 records). [2025-09-16T18:38:43.639084] [INFO ] [MAIN ] GIB: Full history saved. [2025-09-16T18:38:43.639179] [INFO ] [MAIN ] Update Progress: 932/2369 symbols completed. GIB took 22.9s. Estimated remaining: 2h 32m 9s. Rate: 9.4 symbols/min [2025-09-16T18:38:43.809367] [INFO ] [MAIN ] GIS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:38:43.855957] [DEBUG] [LOCAL ] GIS: Loaded 39641 existing records from primary storage. [2025-09-16T18:38:43.856105] [INFO ] [MAIN ] GIS: Loaded 39641 existing records. [2025-09-16T18:38:44.034315] [WARN ] [MAIN ] GIS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:38:44.034565] [INFO ] [MAIN ] GIS: Wiping existing primary data before full refresh. [2025-09-16T18:38:44.034613] [INFO ] [LOCAL ] GIS: Starting deletion of primary data. [2025-09-16T18:38:44.035781] [DEBUG] [LOCAL ] GIS: Deleted primary data file. [2025-09-16T18:38:44.035841] [INFO ] [MAIN ] GIS: Starting full primary refresh. [2025-09-16T18:38:44.300136] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-09-16T18:38:44.334115] [INFO ] [REMOTE ] GHY: Full fetch complete (22990 records). [2025-09-16T18:38:44.336731] [DEBUG] [VALIDATOR ] GHY: Validating primary data. [2025-09-16T18:38:44.383676] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-04. [2025-09-16T18:38:44.503190] [INFO ] [REMOTE ] GIS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:38:44.797228] [DEBUG] [VALIDATOR ] GHY: All expected trading days are present. [2025-09-16T18:38:44.806844] [DEBUG] [VALIDATOR ] GHY: Primary data validation passed. [2025-09-16T18:38:44.807337] [INFO ] [MAIN ] GHY: Fetched primary data is valid. [2025-09-16T18:38:44.807388] [DEBUG] [LOCAL ] GHY: Saving primary data. [2025-09-16T18:38:44.808238] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-09-16T18:38:44.812795] [DEBUG] [LOCAL ] GHY: Trimmed 22990 records to 14955 (keeping 150 days of data) [2025-09-16T18:38:44.909898] [DEBUG] [LOCAL ] GHY: Primary data saved (14955 records). [2025-09-16T18:38:44.910049] [INFO ] [MAIN ] GHY: Full history saved. [2025-09-16T18:38:44.910134] [INFO ] [MAIN ] Update Progress: 933/2369 symbols completed. GHY took 31.6s. Estimated remaining: 2h 31m 54s. Rate: 9.5 symbols/min [2025-09-16T18:38:45.079491] [INFO ] [MAIN ] GKOS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:38:45.112050] [DEBUG] [LOCAL ] GKOS: Loaded 31298 existing records from primary storage. [2025-09-16T18:38:45.112183] [INFO ] [MAIN ] GKOS: Loaded 31298 existing records. [2025-09-16T18:38:45.180083] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-09. [2025-09-16T18:38:45.294789] [WARN ] [MAIN ] GKOS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:38:45.295026] [INFO ] [MAIN ] GKOS: Wiping existing primary data before full refresh. [2025-09-16T18:38:45.295075] [INFO ] [LOCAL ] GKOS: Starting deletion of primary data. [2025-09-16T18:38:45.295906] [DEBUG] [LOCAL ] GKOS: Deleted primary data file. [2025-09-16T18:38:45.295955] [INFO ] [MAIN ] GKOS: Starting full primary refresh. [2025-09-16T18:38:45.644875] [INFO ] [REMOTE ] GKOS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:38:45.974870] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-02. [2025-09-16T18:38:46.186032] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T18:38:46.188488] [INFO ] [REMOTE ] GIC: Full fetch complete (15090 records). [2025-09-16T18:38:46.188583] [DEBUG] [VALIDATOR ] GIC: Validating primary data. [2025-09-16T18:38:46.555011] [DEBUG] [VALIDATOR ] GIC: All expected trading days are present. [2025-09-16T18:38:46.555227] [DEBUG] [VALIDATOR ] GIC: Primary data validation passed. [2025-09-16T18:38:46.555598] [INFO ] [MAIN ] GIC: Fetched primary data is valid. [2025-09-16T18:38:46.555675] [DEBUG] [LOCAL ] GIC: Saving primary data. [2025-09-16T18:38:46.580477] [DEBUG] [LOCAL ] GIC: Trimmed 15090 records to 9935 (keeping 150 days of data) [2025-09-16T18:38:46.582587] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-04 (2.20 seconds). [2025-09-16T18:38:46.654173] [DEBUG] [LOCAL ] GIC: Primary data saved (9935 records). [2025-09-16T18:38:46.654308] [INFO ] [MAIN ] GIC: Full history saved. [2025-09-16T18:38:46.654404] [INFO ] [MAIN ] Update Progress: 934/2369 symbols completed. GIC took 19.4s. Estimated remaining: 2h 31m 41s. Rate: 9.5 symbols/min [2025-09-16T18:38:46.780071] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-02. [2025-09-16T18:38:46.824954] [INFO ] [MAIN ] GL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:38:46.862055] [DEBUG] [LOCAL ] GL: Loaded 27677 existing records from primary storage. [2025-09-16T18:38:46.862200] [INFO ] [MAIN ] GL: Loaded 27677 existing records. [2025-09-16T18:38:47.037177] [WARN ] [MAIN ] GL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:38:47.037383] [INFO ] [MAIN ] GL: Wiping existing primary data before full refresh. [2025-09-16T18:38:47.037428] [INFO ] [LOCAL ] GL: Starting deletion of primary data. [2025-09-16T18:38:47.038189] [DEBUG] [LOCAL ] GL: Deleted primary data file. [2025-09-16T18:38:47.038238] [INFO ] [MAIN ] GL: Starting full primary refresh. [2025-09-16T18:38:47.389903] [INFO ] [REMOTE ] GL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:38:47.415744] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T18:38:47.574856] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-02. [2025-09-16T18:38:48.374914] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-03. [2025-09-16T18:38:48.440708] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T18:38:49.125105] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T18:38:49.174879] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-03. [2025-09-16T18:38:49.588263] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-09-16T18:38:49.974942] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-03. [2025-09-16T18:38:50.775335] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-05. [2025-09-16T18:38:50.777696] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T18:38:51.574955] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-04. [2025-09-16T18:38:51.613957] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T18:38:52.363107] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T18:38:52.374836] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-06. [2025-09-16T18:38:53.059426] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T18:38:53.174890] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-05. [2025-09-16T18:38:53.845700] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-09-16T18:38:53.974870] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-07. [2025-09-16T18:38:54.710266] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T18:38:54.774953] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-04. [2025-09-16T18:38:55.427328] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T18:38:55.574878] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-08. [2025-09-16T18:38:56.374898] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-04. [2025-09-16T18:38:56.750973] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-09-16T18:38:57.174934] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-06. [2025-09-16T18:38:57.292551] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-09-16T18:38:57.834561] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T18:38:57.974909] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-05. [2025-09-16T18:38:58.375970] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T18:38:58.774943] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-09. [2025-09-16T18:38:59.560080] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T18:38:59.574847] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-06. [2025-09-16T18:38:59.969745] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-09-16T18:38:59.975291] [INFO ] [REMOTE ] GIL: Full fetch complete (46636 records). [2025-09-16T18:38:59.975392] [DEBUG] [VALIDATOR ] GIL: Validating primary data. [2025-09-16T18:39:00.381366] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-07. [2025-09-16T18:39:00.393805] [DEBUG] [VALIDATOR ] GIL: All expected trading days are present. [2025-09-16T18:39:00.393866] [DEBUG] [VALIDATOR ] GIL: Primary data validation passed. [2025-09-16T18:39:00.394631] [INFO ] [MAIN ] GIL: Fetched primary data is valid. [2025-09-16T18:39:00.394789] [DEBUG] [LOCAL ] GIL: Saving primary data. [2025-09-16T18:39:00.395896] [DEBUG] [LOCAL ] GIL: Trimmed 46636 records to 31974 (keeping 150 days of data) [2025-09-16T18:39:00.601266] [DEBUG] [LOCAL ] GIL: Primary data saved (31974 records). [2025-09-16T18:39:00.601402] [INFO ] [MAIN ] GIL: Full history saved. [2025-09-16T18:39:00.601485] [INFO ] [MAIN ] Update Progress: 935/2369 symbols completed. GIL took 32.3s. Estimated remaining: 2h 31m 46s. Rate: 9.4 symbols/min [2025-09-16T18:39:00.780607] [INFO ] [MAIN ] GLOB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:39:00.806883] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T18:39:00.844935] [DEBUG] [LOCAL ] GLOB: Loaded 35383 existing records from primary storage. [2025-09-16T18:39:00.845100] [INFO ] [MAIN ] GLOB: Loaded 35383 existing records. [2025-09-16T18:39:01.019182] [WARN ] [MAIN ] GLOB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:39:01.019408] [INFO ] [MAIN ] GLOB: Wiping existing primary data before full refresh. [2025-09-16T18:39:01.019456] [INFO ] [LOCAL ] GLOB: Starting deletion of primary data. [2025-09-16T18:39:01.020271] [DEBUG] [LOCAL ] GLOB: Deleted primary data file. [2025-09-16T18:39:01.020329] [INFO ] [MAIN ] GLOB: Starting full primary refresh. [2025-09-16T18:39:01.177337] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-05. [2025-09-16T18:39:01.378539] [INFO ] [REMOTE ] GLOB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:39:01.668692] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-09-16T18:39:01.974947] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-08. [2025-09-16T18:39:02.706881] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T18:39:02.774948] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-06. [2025-09-16T18:39:03.408377] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-09-16T18:39:03.574914] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-02. [2025-09-16T18:39:04.374876] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-07. [2025-09-16T18:39:04.748761] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-09-16T18:39:05.149573] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T18:39:05.174851] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-03. [2025-09-16T18:39:05.857605] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T18:39:05.974908] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-09. [2025-09-16T18:39:06.785772] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-07. [2025-09-16T18:39:06.791159] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T18:39:07.301461] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-09-16T18:39:07.305591] [INFO ] [REMOTE ] GL: Full fetch complete (41673 records). [2025-09-16T18:39:07.305705] [DEBUG] [VALIDATOR ] GL: Validating primary data. [2025-09-16T18:39:07.584113] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-04. [2025-09-16T18:39:07.699388] [DEBUG] [VALIDATOR ] GL: All expected trading days are present. [2025-09-16T18:39:07.699463] [DEBUG] [VALIDATOR ] GL: Primary data validation passed. [2025-09-16T18:39:07.700108] [INFO ] [MAIN ] GL: Fetched primary data is valid. [2025-09-16T18:39:07.700153] [DEBUG] [LOCAL ] GL: Saving primary data. [2025-09-16T18:39:07.701045] [DEBUG] [LOCAL ] GL: Trimmed 41673 records to 27947 (keeping 150 days of data) [2025-09-16T18:39:07.880141] [DEBUG] [LOCAL ] GL: Primary data saved (27947 records). [2025-09-16T18:39:07.880279] [INFO ] [MAIN ] GL: Full history saved. [2025-09-16T18:39:07.880362] [INFO ] [MAIN ] Update Progress: 936/2369 symbols completed. GL took 21.2s. Estimated remaining: 2h 31m 41s. Rate: 9.4 symbols/min [2025-09-16T18:39:08.048750] [INFO ] [MAIN ] GLP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:39:08.048933] [DEBUG] [LOCAL ] GLP: No existing data file found in primary storage. [2025-09-16T18:39:08.048971] [WARN ] [MAIN ] GLP: No existing data found, performing full refresh instead. [2025-09-16T18:39:08.049089] [INFO ] [MAIN ] GLP: Wiping existing primary data before full refresh. [2025-09-16T18:39:08.049127] [INFO ] [LOCAL ] GLP: Starting deletion of primary data. [2025-09-16T18:39:08.049168] [DEBUG] [LOCAL ] GLP: No primary data file found to delete. [2025-09-16T18:39:08.049208] [INFO ] [MAIN ] GLP: Starting full primary refresh. [2025-09-16T18:39:08.385087] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-08. [2025-09-16T18:39:08.423335] [INFO ] [REMOTE ] GLP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:39:08.680264] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T18:39:09.109419] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-09-16T18:39:09.174866] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-05. [2025-09-16T18:39:09.974917] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-08. [2025-09-16T18:39:10.054096] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:39:10.774890] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-02. [2025-09-16T18:39:11.097616] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-09-16T18:39:11.574895] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-06. [2025-09-16T18:39:11.742422] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-09-16T18:39:11.963394] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-09-16T18:39:12.374916] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-03. [2025-09-16T18:39:13.174899] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-09. [2025-09-16T18:39:13.185112] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-09-16T18:39:13.214076] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T18:39:13.974967] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-04. [2025-09-16T18:39:14.670964] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:39:14.676589] [INFO ] [REMOTE ] GKOS: Full fetch complete (46698 records). [2025-09-16T18:39:14.676794] [DEBUG] [VALIDATOR ] GKOS: Validating primary data. [2025-09-16T18:39:14.780881] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-09. [2025-09-16T18:39:14.823633] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-09-16T18:39:15.088183] [DEBUG] [VALIDATOR ] GKOS: All expected trading days are present. [2025-09-16T18:39:15.088308] [DEBUG] [VALIDATOR ] GKOS: Primary data validation passed. [2025-09-16T18:39:15.089262] [INFO ] [MAIN ] GKOS: Fetched primary data is valid. [2025-09-16T18:39:15.089313] [DEBUG] [LOCAL ] GKOS: Saving primary data. [2025-09-16T18:39:15.090453] [DEBUG] [LOCAL ] GKOS: Trimmed 46698 records to 31616 (keeping 150 days of data) [2025-09-16T18:39:15.282599] [DEBUG] [LOCAL ] GKOS: Primary data saved (31616 records). [2025-09-16T18:39:15.282782] [INFO ] [MAIN ] GKOS: Full history saved. [2025-09-16T18:39:15.282877] [INFO ] [MAIN ] Update Progress: 937/2369 symbols completed. GKOS took 30.4s. Estimated remaining: 2h 31m 37s. Rate: 9.4 symbols/min [2025-09-16T18:39:15.453236] [INFO ] [MAIN ] GLW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:39:15.491121] [DEBUG] [LOCAL ] GLW: Loaded 39598 existing records from primary storage. [2025-09-16T18:39:15.491296] [INFO ] [MAIN ] GLW: Loaded 39598 existing records. [2025-09-16T18:39:15.580802] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-07. [2025-09-16T18:39:15.667493] [WARN ] [MAIN ] GLW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:39:15.667783] [INFO ] [MAIN ] GLW: Wiping existing primary data before full refresh. [2025-09-16T18:39:15.667831] [INFO ] [LOCAL ] GLW: Starting deletion of primary data. [2025-09-16T18:39:15.668780] [DEBUG] [LOCAL ] GLW: Deleted primary data file. [2025-09-16T18:39:15.668828] [INFO ] [MAIN ] GLW: Starting full primary refresh. [2025-09-16T18:39:16.024764] [INFO ] [REMOTE ] GLW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:39:16.374933] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-05. [2025-09-16T18:39:16.789195] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-09-16T18:39:16.795628] [INFO ] [REMOTE ] GIS: Full fetch complete (58353 records). [2025-09-16T18:39:16.795773] [DEBUG] [VALIDATOR ] GIS: Validating primary data. [2025-09-16T18:39:17.184916] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-02. [2025-09-16T18:39:17.187904] [DEBUG] [VALIDATOR ] GIS: All expected trading days are present. [2025-09-16T18:39:17.188145] [DEBUG] [VALIDATOR ] GIS: Primary data validation passed. [2025-09-16T18:39:17.189986] [INFO ] [MAIN ] GIS: Fetched primary data is valid. [2025-09-16T18:39:17.190076] [DEBUG] [LOCAL ] GIS: Saving primary data. [2025-09-16T18:39:17.198435] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T18:39:17.201979] [DEBUG] [LOCAL ] GIS: Trimmed 58353 records to 40031 (keeping 150 days of data) [2025-09-16T18:39:17.446025] [DEBUG] [LOCAL ] GIS: Primary data saved (40031 records). [2025-09-16T18:39:17.446313] [INFO ] [MAIN ] GIS: Full history saved. [2025-09-16T18:39:17.447086] [INFO ] [MAIN ] Update Progress: 938/2369 symbols completed. GIS took 33.8s. Estimated remaining: 2h 31m 24s. Rate: 9.5 symbols/min [2025-09-16T18:39:17.581233] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-09-16T18:39:17.660848] [INFO ] [MAIN ] GM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:39:17.698490] [DEBUG] [LOCAL ] GM: Loaded 39689 existing records from primary storage. [2025-09-16T18:39:17.698677] [INFO ] [MAIN ] GM: Loaded 39689 existing records. [2025-09-16T18:39:17.873316] [WARN ] [MAIN ] GM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:39:17.873561] [INFO ] [MAIN ] GM: Wiping existing primary data before full refresh. [2025-09-16T18:39:17.873608] [INFO ] [LOCAL ] GM: Starting deletion of primary data. [2025-09-16T18:39:17.874524] [DEBUG] [LOCAL ] GM: Deleted primary data file. [2025-09-16T18:39:17.874575] [INFO ] [MAIN ] GM: Starting full primary refresh. [2025-09-16T18:39:17.978779] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-08. [2025-09-16T18:39:18.231754] [INFO ] [REMOTE ] GM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:39:18.774865] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-02. [2025-09-16T18:39:19.170603] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-09-16T18:39:19.574879] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-03. [2025-09-16T18:39:19.626249] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T18:39:20.374931] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-06. [2025-09-16T18:39:21.010621] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-09-16T18:39:21.147102] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T18:39:21.184824] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-03. [2025-09-16T18:39:21.190717] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T18:39:21.974913] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-09. [2025-09-16T18:39:22.774931] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-04. [2025-09-16T18:39:23.522596] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-03 (2.34 seconds). [2025-09-16T18:39:23.574856] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-04. [2025-09-16T18:39:23.766486] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-09-16T18:39:23.772502] [INFO ] [REMOTE ] GLOB: Full fetch complete (50977 records). [2025-09-16T18:39:23.772612] [DEBUG] [VALIDATOR ] GLOB: Validating primary data. [2025-09-16T18:39:24.166837] [DEBUG] [VALIDATOR ] GLOB: All expected trading days are present. [2025-09-16T18:39:24.166930] [DEBUG] [VALIDATOR ] GLOB: Primary data validation passed. [2025-09-16T18:39:24.167665] [INFO ] [MAIN ] GLOB: Fetched primary data is valid. [2025-09-16T18:39:24.167713] [DEBUG] [LOCAL ] GLOB: Saving primary data. [2025-09-16T18:39:24.168912] [DEBUG] [LOCAL ] GLOB: Trimmed 50977 records to 35748 (keeping 150 days of data) [2025-09-16T18:39:24.375207] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-07. [2025-09-16T18:39:24.397366] [DEBUG] [LOCAL ] GLOB: Primary data saved (35748 records). [2025-09-16T18:39:24.397492] [INFO ] [MAIN ] GLOB: Full history saved. [2025-09-16T18:39:24.397594] [INFO ] [MAIN ] Update Progress: 939/2369 symbols completed. GLOB took 23.8s. Estimated remaining: 2h 31m 18s. Rate: 9.5 symbols/min [2025-09-16T18:39:24.570919] [INFO ] [MAIN ] GME: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:39:24.609420] [DEBUG] [LOCAL ] GME: Loaded 39747 existing records from primary storage. [2025-09-16T18:39:24.609573] [INFO ] [MAIN ] GME: Loaded 39747 existing records. [2025-09-16T18:39:24.795067] [WARN ] [MAIN ] GME: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:39:24.795540] [INFO ] [MAIN ] GME: Wiping existing primary data before full refresh. [2025-09-16T18:39:24.795692] [INFO ] [LOCAL ] GME: Starting deletion of primary data. [2025-09-16T18:39:24.796726] [DEBUG] [LOCAL ] GME: Deleted primary data file. [2025-09-16T18:39:24.796780] [INFO ] [MAIN ] GME: Starting full primary refresh. [2025-09-16T18:39:25.189786] [INFO ] [REMOTE ] GME: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:39:25.190225] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-02. [2025-09-16T18:39:25.193285] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-04 (2.42 seconds). [2025-09-16T18:39:25.209411] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-09-16T18:39:25.822498] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-04 (2.25 seconds). [2025-09-16T18:39:25.974942] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-08. [2025-09-16T18:39:26.777165] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-05. [2025-09-16T18:39:26.778638] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-09-16T18:39:27.496559] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-02 (2.31 seconds). [2025-09-16T18:39:27.574918] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-09. [2025-09-16T18:39:28.374935] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-03. [2025-09-16T18:39:28.510167] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T18:39:28.528443] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T18:39:28.530765] [INFO ] [REMOTE ] GLP: Full fetch complete (10314 records). [2025-09-16T18:39:28.530842] [DEBUG] [VALIDATOR ] GLP: Validating primary data. [2025-09-16T18:39:28.531280] [FATAL] [VALIDATOR ] GLP: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:39:28.531358] [FATAL] [MAIN ] GLP: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:39:28.531397] [WARN ] [MAIN ] GLP: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:39:28.531445] [INFO ] [LOCAL ] GLP: Starting deletion of primary data. [2025-09-16T18:39:28.531518] [DEBUG] [LOCAL ] GLP: No primary data file found to delete. [2025-09-16T18:39:28.531558] [INFO ] [LOCAL ] GLP: Starting deletion of derivative data. [2025-09-16T18:39:28.531615] [DEBUG] [LOCAL ] GLP: No derivative data file found to delete. [2025-09-16T18:39:28.531705] [INFO ] [MAIN ] Update Progress: 940/2369 symbols completed. GLP took 20.7s. Estimated remaining: 2h 31m 9s. Rate: 9.5 symbols/min [2025-09-16T18:39:28.717365] [INFO ] [MAIN ] GMED: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:39:28.752066] [DEBUG] [LOCAL ] GMED: Loaded 35554 existing records from primary storage. [2025-09-16T18:39:28.752224] [INFO ] [MAIN ] GMED: Loaded 35554 existing records. [2025-09-16T18:39:28.936440] [WARN ] [MAIN ] GMED: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:39:28.936750] [INFO ] [MAIN ] GMED: Wiping existing primary data before full refresh. [2025-09-16T18:39:28.936805] [INFO ] [LOCAL ] GMED: Starting deletion of primary data. [2025-09-16T18:39:28.937618] [DEBUG] [LOCAL ] GMED: Deleted primary data file. [2025-09-16T18:39:28.937696] [INFO ] [MAIN ] GMED: Starting full primary refresh. [2025-09-16T18:39:29.180039] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-05. [2025-09-16T18:39:29.291416] [INFO ] [REMOTE ] GMED: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:39:29.974917] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-02. [2025-09-16T18:39:30.774875] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-06. [2025-09-16T18:39:31.249145] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-03 (2.87 seconds). [2025-09-16T18:39:31.496406] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T18:39:31.556305] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-05 (2.38 seconds). [2025-09-16T18:39:31.574856] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-06. [2025-09-16T18:39:32.374916] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-03. [2025-09-16T18:39:32.728454] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-09-16T18:39:33.174927] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-07. [2025-09-16T18:39:33.467391] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-09-16T18:39:33.920865] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T18:39:33.974881] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-07. [2025-09-16T18:39:34.774897] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-04. [2025-09-16T18:39:35.053443] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T18:39:35.575052] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-08. [2025-09-16T18:39:35.894094] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T18:39:36.383769] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-04. [2025-09-16T18:39:36.391762] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T18:39:37.174942] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-05. [2025-09-16T18:39:37.316226] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-09-16T18:39:37.974910] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-09. [2025-09-16T18:39:38.652085] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T18:39:38.774899] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-08. [2025-09-16T18:39:39.196524] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-04 (2.81 seconds). [2025-09-16T18:39:39.574911] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-06. [2025-09-16T18:39:39.981667] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-09-16T18:39:39.987996] [INFO ] [REMOTE ] GLW: Full fetch complete (58307 records). [2025-09-16T18:39:39.988099] [DEBUG] [VALIDATOR ] GLW: Validating primary data. [2025-09-16T18:39:40.377848] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-05. [2025-09-16T18:39:40.392972] [DEBUG] [VALIDATOR ] GLW: All expected trading days are present. [2025-09-16T18:39:40.393047] [DEBUG] [VALIDATOR ] GLW: Primary data validation passed. [2025-09-16T18:39:40.393976] [INFO ] [MAIN ] GLW: Fetched primary data is valid. [2025-09-16T18:39:40.394053] [DEBUG] [LOCAL ] GLW: Saving primary data. [2025-09-16T18:39:40.395470] [DEBUG] [LOCAL ] GLW: Trimmed 58307 records to 39988 (keeping 150 days of data) [2025-09-16T18:39:40.676127] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-09-16T18:39:40.700952] [DEBUG] [LOCAL ] GLW: Primary data saved (39988 records). [2025-09-16T18:39:40.701130] [INFO ] [MAIN ] GLW: Full history saved. [2025-09-16T18:39:40.701230] [INFO ] [MAIN ] Update Progress: 941/2369 symbols completed. GLW took 25.4s. Estimated remaining: 2h 31m 11s. Rate: 9.4 symbols/min [2025-09-16T18:39:40.884324] [INFO ] [MAIN ] GMRE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:39:40.884549] [DEBUG] [LOCAL ] GMRE: No existing data file found in primary storage. [2025-09-16T18:39:40.884591] [WARN ] [MAIN ] GMRE: No existing data found, performing full refresh instead. [2025-09-16T18:39:40.884764] [INFO ] [MAIN ] GMRE: Wiping existing primary data before full refresh. [2025-09-16T18:39:40.884837] [INFO ] [LOCAL ] GMRE: Starting deletion of primary data. [2025-09-16T18:39:40.884884] [DEBUG] [LOCAL ] GMRE: No primary data file found to delete. [2025-09-16T18:39:40.884918] [INFO ] [MAIN ] GMRE: Starting full primary refresh. [2025-09-16T18:39:41.177445] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-09. [2025-09-16T18:39:41.254027] [INFO ] [REMOTE ] GMRE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:39:41.283809] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T18:39:41.974937] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-07. [2025-09-16T18:39:42.774927] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-02. [2025-09-16T18:39:43.026435] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-05 (2.65 seconds). [2025-09-16T18:39:43.294870] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-09 (2.12 seconds). [2025-09-16T18:39:43.301329] [INFO ] [REMOTE ] GM: Full fetch complete (58408 records). [2025-09-16T18:39:43.301470] [DEBUG] [VALIDATOR ] GM: Validating primary data. [2025-09-16T18:39:43.580006] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-06. [2025-09-16T18:39:43.707514] [DEBUG] [VALIDATOR ] GM: All expected trading days are present. [2025-09-16T18:39:43.707687] [DEBUG] [VALIDATOR ] GM: Primary data validation passed. [2025-09-16T18:39:43.708510] [INFO ] [MAIN ] GM: Fetched primary data is valid. [2025-09-16T18:39:43.708576] [DEBUG] [LOCAL ] GM: Saving primary data. [2025-09-16T18:39:43.736737] [DEBUG] [LOCAL ] GM: Trimmed 58408 records to 40079 (keeping 150 days of data) [2025-09-16T18:39:43.738390] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T18:39:43.870952] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T18:39:44.029481] [DEBUG] [LOCAL ] GM: Primary data saved (40079 records). [2025-09-16T18:39:44.029611] [INFO ] [MAIN ] GM: Full history saved. [2025-09-16T18:39:44.029713] [INFO ] [MAIN ] Update Progress: 942/2369 symbols completed. GM took 26.6s. Estimated remaining: 2h 31m 0s. Rate: 9.5 symbols/min [2025-09-16T18:39:44.208476] [INFO ] [MAIN ] GNE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:39:44.224455] [DEBUG] [LOCAL ] GNE: Loaded 14550 existing records from primary storage. [2025-09-16T18:39:44.224594] [INFO ] [MAIN ] GNE: Loaded 14550 existing records. [2025-09-16T18:39:44.384889] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-03. [2025-09-16T18:39:44.414204] [WARN ] [MAIN ] GNE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:39:44.414484] [INFO ] [MAIN ] GNE: Wiping existing primary data before full refresh. [2025-09-16T18:39:44.414534] [INFO ] [LOCAL ] GNE: Starting deletion of primary data. [2025-09-16T18:39:44.415023] [DEBUG] [LOCAL ] GNE: Deleted primary data file. [2025-09-16T18:39:44.415076] [INFO ] [MAIN ] GNE: Starting full primary refresh. [2025-09-16T18:39:44.793257] [INFO ] [REMOTE ] GNE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:39:45.174901] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-02. [2025-09-16T18:39:45.862385] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T18:39:45.974877] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-04. [2025-09-16T18:39:46.122066] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-06 (2.54 seconds). [2025-09-16T18:39:46.146108] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-09-16T18:39:46.774905] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-03. [2025-09-16T18:39:47.476718] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T18:39:47.575737] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-05. [2025-09-16T18:39:47.577466] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T18:39:48.374983] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-04. [2025-09-16T18:39:49.067080] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-09-16T18:39:49.091408] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T18:39:49.174924] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-05. [2025-09-16T18:39:49.974902] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-07. [2025-09-16T18:39:50.113554] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-09-16T18:39:50.774912] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-06. [2025-09-16T18:39:51.574893] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-08. [2025-09-16T18:39:52.283396] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-09-16T18:39:52.374866] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-07. [2025-09-16T18:39:52.647032] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-07 (2.67 seconds). [2025-09-16T18:39:52.735508] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T18:39:53.174897] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-09. [2025-09-16T18:39:53.427270] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-09-16T18:39:53.974928] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-06. [2025-09-16T18:39:54.491405] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T18:39:54.497074] [INFO ] [REMOTE ] GMED: Full fetch complete (52540 records). [2025-09-16T18:39:54.497222] [DEBUG] [VALIDATOR ] GMED: Validating primary data. [2025-09-16T18:39:54.783811] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-08. [2025-09-16T18:39:54.875487] [DEBUG] [VALIDATOR ] GMED: All expected trading days are present. [2025-09-16T18:39:54.875566] [DEBUG] [VALIDATOR ] GMED: Primary data validation passed. [2025-09-16T18:39:54.876300] [INFO ] [MAIN ] GMED: Fetched primary data is valid. [2025-09-16T18:39:54.876347] [DEBUG] [LOCAL ] GMED: Saving primary data. [2025-09-16T18:39:54.877453] [DEBUG] [LOCAL ] GMED: Trimmed 52540 records to 35875 (keeping 150 days of data) [2025-09-16T18:39:55.090046] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T18:39:55.105426] [DEBUG] [LOCAL ] GMED: Primary data saved (35875 records). [2025-09-16T18:39:55.105525] [INFO ] [MAIN ] GMED: Full history saved. [2025-09-16T18:39:55.105611] [INFO ] [MAIN ] Update Progress: 943/2369 symbols completed. GMED took 26.6s. Estimated remaining: 2h 31m 1s. Rate: 9.4 symbols/min [2025-09-16T18:39:55.277477] [INFO ] [MAIN ] GNK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:39:55.306489] [DEBUG] [LOCAL ] GNK: Loaded 29366 existing records from primary storage. [2025-09-16T18:39:55.306624] [INFO ] [MAIN ] GNK: Loaded 29366 existing records. [2025-09-16T18:39:55.481396] [WARN ] [MAIN ] GNK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:39:55.481631] [INFO ] [MAIN ] GNK: Wiping existing primary data before full refresh. [2025-09-16T18:39:55.481699] [INFO ] [LOCAL ] GNK: Starting deletion of primary data. [2025-09-16T18:39:55.482381] [DEBUG] [LOCAL ] GNK: Deleted primary data file. [2025-09-16T18:39:55.482426] [INFO ] [MAIN ] GNK: Starting full primary refresh. [2025-09-16T18:39:55.580328] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-07. [2025-09-16T18:39:55.836053] [INFO ] [REMOTE ] GNK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:39:56.374952] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-08. [2025-09-16T18:39:56.932045] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T18:39:57.174890] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-08. [2025-09-16T18:39:57.338352] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-08 (2.55 seconds). [2025-09-16T18:39:57.445322] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T18:39:57.974886] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-09. [2025-09-16T18:39:58.215281] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-09-16T18:39:58.774911] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-02. [2025-09-16T18:39:59.398814] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T18:39:59.403670] [INFO ] [REMOTE ] GMRE: Full fetch complete (40378 records). [2025-09-16T18:39:59.403829] [DEBUG] [VALIDATOR ] GMRE: Validating primary data. [2025-09-16T18:39:59.581459] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-09. [2025-09-16T18:39:59.815475] [DEBUG] [VALIDATOR ] GMRE: All expected trading days are present. [2025-09-16T18:39:59.815557] [DEBUG] [VALIDATOR ] GMRE: Primary data validation passed. [2025-09-16T18:39:59.816187] [INFO ] [MAIN ] GMRE: Fetched primary data is valid. [2025-09-16T18:39:59.816229] [DEBUG] [LOCAL ] GMRE: Saving primary data. [2025-09-16T18:39:59.817220] [DEBUG] [LOCAL ] GMRE: Trimmed 40378 records to 28154 (keeping 150 days of data) [2025-09-16T18:39:59.977468] [DEBUG] [LOCAL ] GMRE: Primary data saved (28154 records). [2025-09-16T18:39:59.977612] [INFO ] [MAIN ] GMRE: Full history saved. [2025-09-16T18:39:59.977706] [INFO ] [MAIN ] Update Progress: 944/2369 symbols completed. GMRE took 19.3s. Estimated remaining: 2h 30m 53s. Rate: 9.4 symbols/min [2025-09-16T18:40:00.144612] [INFO ] [MAIN ] GNL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:00.175837] [DEBUG] [LOCAL ] GNL: Loaded 34319 existing records from primary storage. [2025-09-16T18:40:00.175994] [INFO ] [MAIN ] GNL: Loaded 34319 existing records. [2025-09-16T18:40:00.356327] [WARN ] [MAIN ] GNL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:40:00.356624] [INFO ] [MAIN ] GNL: Wiping existing primary data before full refresh. [2025-09-16T18:40:00.356779] [INFO ] [LOCAL ] GNL: Starting deletion of primary data. [2025-09-16T18:40:00.357540] [DEBUG] [LOCAL ] GNL: Deleted primary data file. [2025-09-16T18:40:00.357586] [INFO ] [MAIN ] GNL: Starting full primary refresh. [2025-09-16T18:40:00.380069] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-09. [2025-09-16T18:40:00.494485] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-09-16T18:40:00.765358] [INFO ] [REMOTE ] GNL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:01.174912] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-03. [2025-09-16T18:40:01.356271] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T18:40:01.358889] [INFO ] [REMOTE ] GNE: Full fetch complete (18158 records). [2025-09-16T18:40:01.358970] [DEBUG] [VALIDATOR ] GNE: Validating primary data. [2025-09-16T18:40:01.737868] [DEBUG] [VALIDATOR ] GNE: All expected trading days are present. [2025-09-16T18:40:01.737952] [DEBUG] [VALIDATOR ] GNE: Primary data validation passed. [2025-09-16T18:40:01.738383] [INFO ] [MAIN ] GNE: Fetched primary data is valid. [2025-09-16T18:40:01.738433] [DEBUG] [LOCAL ] GNE: Saving primary data. [2025-09-16T18:40:01.739032] [DEBUG] [LOCAL ] GNE: Trimmed 18158 records to 14670 (keeping 150 days of data) [2025-09-16T18:40:01.831824] [DEBUG] [LOCAL ] GNE: Primary data saved (14670 records). [2025-09-16T18:40:01.831985] [INFO ] [MAIN ] GNE: Full history saved. [2025-09-16T18:40:01.832075] [INFO ] [MAIN ] Update Progress: 945/2369 symbols completed. GNE took 17.8s. Estimated remaining: 2h 30m 39s. Rate: 9.5 symbols/min [2025-09-16T18:40:01.980519] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-02. [2025-09-16T18:40:02.008216] [INFO ] [MAIN ] GNRC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:02.045290] [DEBUG] [LOCAL ] GNRC: Loaded 33188 existing records from primary storage. [2025-09-16T18:40:02.045459] [INFO ] [MAIN ] GNRC: Loaded 33188 existing records. [2025-09-16T18:40:02.241613] [WARN ] [MAIN ] GNRC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:40:02.242100] [INFO ] [MAIN ] GNRC: Wiping existing primary data before full refresh. [2025-09-16T18:40:02.242172] [INFO ] [LOCAL ] GNRC: Starting deletion of primary data. [2025-09-16T18:40:02.243177] [DEBUG] [LOCAL ] GNRC: Deleted primary data file. [2025-09-16T18:40:02.243274] [INFO ] [MAIN ] GNRC: Starting full primary refresh. [2025-09-16T18:40:02.396293] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-09 (2.81 seconds). [2025-09-16T18:40:02.496523] [INFO ] [REMOTE ] GME: Full fetch complete (58436 records). [2025-09-16T18:40:02.526379] [DEBUG] [VALIDATOR ] GME: Validating primary data. [2025-09-16T18:40:02.751132] [INFO ] [REMOTE ] GNRC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:02.781200] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-02. [2025-09-16T18:40:03.039574] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-09-16T18:40:03.093268] [DEBUG] [VALIDATOR ] GME: All expected trading days are present. [2025-09-16T18:40:03.093736] [DEBUG] [VALIDATOR ] GME: Primary data validation passed. [2025-09-16T18:40:03.094832] [INFO ] [MAIN ] GME: Fetched primary data is valid. [2025-09-16T18:40:03.094906] [DEBUG] [LOCAL ] GME: Saving primary data. [2025-09-16T18:40:03.096178] [DEBUG] [LOCAL ] GME: Trimmed 58436 records to 40137 (keeping 150 days of data) [2025-09-16T18:40:03.356935] [DEBUG] [LOCAL ] GME: Primary data saved (40137 records). [2025-09-16T18:40:03.357100] [INFO ] [MAIN ] GME: Full history saved. [2025-09-16T18:40:03.357222] [INFO ] [MAIN ] Update Progress: 946/2369 symbols completed. GME took 39.0s. Estimated remaining: 2h 30m 26s. Rate: 9.5 symbols/min [2025-09-16T18:40:03.527208] [INFO ] [MAIN ] GNT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:03.533986] [DEBUG] [LOCAL ] GNT: Loaded 5815 existing records from primary storage. [2025-09-16T18:40:03.534100] [INFO ] [MAIN ] GNT: Loaded 5815 existing records. [2025-09-16T18:40:03.597148] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-04. [2025-09-16T18:40:03.711519] [WARN ] [MAIN ] GNT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:40:03.711983] [INFO ] [MAIN ] GNT: Wiping existing primary data before full refresh. [2025-09-16T18:40:03.712057] [INFO ] [LOCAL ] GNT: Starting deletion of primary data. [2025-09-16T18:40:03.712449] [DEBUG] [LOCAL ] GNT: Deleted primary data file. [2025-09-16T18:40:03.712502] [INFO ] [MAIN ] GNT: Starting full primary refresh. [2025-09-16T18:40:04.028515] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-09-16T18:40:04.108468] [INFO ] [REMOTE ] GNT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:04.374936] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-02. [2025-09-16T18:40:04.542122] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-09-16T18:40:05.174876] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-03. [2025-09-16T18:40:05.217363] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T18:40:05.244047] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T18:40:05.974904] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-05. [2025-09-16T18:40:06.774928] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-03. [2025-09-16T18:40:06.891450] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T18:40:07.479520] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-09-16T18:40:07.574856] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-06. [2025-09-16T18:40:08.374907] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-04. [2025-09-16T18:40:08.443719] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T18:40:09.002979] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T18:40:09.174888] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-03. [2025-09-16T18:40:09.779760] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-09-16T18:40:09.974896] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-04. [2025-09-16T18:40:10.522543] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-09-16T18:40:10.655398] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T18:40:10.774923] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-05. [2025-09-16T18:40:11.574879] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-07. [2025-09-16T18:40:11.714782] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-09-16T18:40:12.374887] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-04. [2025-09-16T18:40:13.174978] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-05. [2025-09-16T18:40:13.327493] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-09-16T18:40:13.737611] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-09-16T18:40:13.974901] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-08. [2025-09-16T18:40:14.774950] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-06. [2025-09-16T18:40:15.144717] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-09-16T18:40:15.403074] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-09-16T18:40:15.574919] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-09. [2025-09-16T18:40:16.374931] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-06. [2025-09-16T18:40:16.956571] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-09-16T18:40:16.961573] [INFO ] [REMOTE ] GNK: Full fetch complete (44551 records). [2025-09-16T18:40:16.961690] [DEBUG] [VALIDATOR ] GNK: Validating primary data. [2025-09-16T18:40:17.181966] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-05. [2025-09-16T18:40:17.350913] [DEBUG] [VALIDATOR ] GNK: All expected trading days are present. [2025-09-16T18:40:17.350989] [DEBUG] [VALIDATOR ] GNK: Primary data validation passed. [2025-09-16T18:40:17.351671] [INFO ] [MAIN ] GNK: Fetched primary data is valid. [2025-09-16T18:40:17.351724] [DEBUG] [LOCAL ] GNK: Saving primary data. [2025-09-16T18:40:17.352683] [DEBUG] [LOCAL ] GNK: Trimmed 44551 records to 29717 (keeping 150 days of data) [2025-09-16T18:40:17.471013] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-06 (2.70 seconds). [2025-09-16T18:40:17.547299] [DEBUG] [LOCAL ] GNK: Primary data saved (29717 records). [2025-09-16T18:40:17.547445] [INFO ] [MAIN ] GNK: Full history saved. [2025-09-16T18:40:17.547548] [INFO ] [MAIN ] Update Progress: 947/2369 symbols completed. GNK took 22.4s. Estimated remaining: 2h 30m 31s. Rate: 9.4 symbols/min [2025-09-16T18:40:17.726802] [INFO ] [MAIN ] GNTY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:17.727056] [DEBUG] [LOCAL ] GNTY: No existing data file found in primary storage. [2025-09-16T18:40:17.727136] [WARN ] [MAIN ] GNTY: No existing data found, performing full refresh instead. [2025-09-16T18:40:17.727779] [INFO ] [MAIN ] GNTY: Wiping existing primary data before full refresh. [2025-09-16T18:40:17.727862] [INFO ] [LOCAL ] GNTY: Starting deletion of primary data. [2025-09-16T18:40:17.728274] [DEBUG] [LOCAL ] GNTY: No primary data file found to delete. [2025-09-16T18:40:17.728408] [INFO ] [MAIN ] GNTY: Starting full primary refresh. [2025-09-16T18:40:17.922238] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-09-16T18:40:17.983086] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-07. [2025-09-16T18:40:18.143797] [INFO ] [REMOTE ] GNTY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:18.725295] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T18:40:18.774858] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-06. [2025-09-16T18:40:19.181333] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T18:40:19.574951] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-08. [2025-09-16T18:40:20.314131] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-09-16T18:40:20.374900] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-09. [2025-09-16T18:40:21.174881] [INFO ] [REMOTE ] GNTY: Fetching primary data for 2025-02. [2025-09-16T18:40:21.239190] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-06 (2.46 seconds). [2025-09-16T18:40:21.253588] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-09-16T18:40:21.255431] [INFO ] [REMOTE ] GNT: Full fetch complete (8293 records). [2025-09-16T18:40:21.255521] [DEBUG] [VALIDATOR ] GNT: Validating primary data. [2025-09-16T18:40:21.614883] [DEBUG] [VALIDATOR ] GNT: All expected trading days are present. [2025-09-16T18:40:21.614965] [DEBUG] [VALIDATOR ] GNT: Primary data validation passed. [2025-09-16T18:40:21.615169] [INFO ] [MAIN ] GNT: Fetched primary data is valid. [2025-09-16T18:40:21.615218] [DEBUG] [LOCAL ] GNT: Saving primary data. [2025-09-16T18:40:21.615707] [DEBUG] [LOCAL ] GNT: Trimmed 8293 records to 5867 (keeping 150 days of data) [2025-09-16T18:40:21.651407] [DEBUG] [LOCAL ] GNT: Primary data saved (5867 records). [2025-09-16T18:40:21.651517] [INFO ] [MAIN ] GNT: Full history saved. [2025-09-16T18:40:21.651626] [INFO ] [MAIN ] Update Progress: 948/2369 symbols completed. GNT took 18.3s. Estimated remaining: 2h 30m 21s. Rate: 9.5 symbols/min [2025-09-16T18:40:21.841305] [INFO ] [MAIN ] GNW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:21.846371] [INFO ] [REMOTE ] GNTY: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-09-16T18:40:21.885323] [DEBUG] [LOCAL ] GNW: Loaded 37886 existing records from primary storage. [2025-09-16T18:40:21.885465] [INFO ] [MAIN ] GNW: Loaded 37886 existing records. [2025-09-16T18:40:21.980004] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-07. [2025-09-16T18:40:22.073020] [WARN ] [MAIN ] GNW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:40:22.073275] [INFO ] [MAIN ] GNW: Wiping existing primary data before full refresh. [2025-09-16T18:40:22.073322] [INFO ] [LOCAL ] GNW: Starting deletion of primary data. [2025-09-16T18:40:22.074064] [DEBUG] [LOCAL ] GNW: Deleted primary data file. [2025-09-16T18:40:22.074118] [INFO ] [MAIN ] GNW: Starting full primary refresh. [2025-09-16T18:40:22.426910] [INFO ] [REMOTE ] GNW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:22.774933] [INFO ] [REMOTE ] GNTY: Fetching primary data for 2025-03. [2025-09-16T18:40:23.371724] [INFO ] [REMOTE ] GNTY: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-09-16T18:40:23.574883] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-02. [2025-09-16T18:40:23.715448] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T18:40:24.374899] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-07. [2025-09-16T18:40:25.174901] [INFO ] [REMOTE ] GNTY: Fetching primary data for 2025-04. [2025-09-16T18:40:25.259201] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T18:40:25.769021] [INFO ] [REMOTE ] GNTY: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-09-16T18:40:25.974882] [INFO ] [REMOTE ] GNTY: Fetching primary data for 2025-05. [2025-09-16T18:40:26.070110] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T18:40:26.375161] [INFO ] [REMOTE ] GNTY: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-09-16T18:40:26.774933] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-08. [2025-09-16T18:40:27.574928] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-03. [2025-09-16T18:40:28.375054] [INFO ] [REMOTE ] GNTY: Fetching primary data for 2025-06. [2025-09-16T18:40:28.532872] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-09-16T18:40:28.791221] [INFO ] [REMOTE ] GNTY: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-09-16T18:40:29.174962] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-09. [2025-09-16T18:40:29.249497] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T18:40:29.974985] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-04. [2025-09-16T18:40:30.432268] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T18:40:30.438699] [INFO ] [REMOTE ] GNL: Full fetch complete (51568 records). [2025-09-16T18:40:30.438885] [DEBUG] [VALIDATOR ] GNL: Validating primary data. [2025-09-16T18:40:30.785172] [INFO ] [REMOTE ] GNTY: Fetching primary data for 2025-07. [2025-09-16T18:40:30.834588] [DEBUG] [VALIDATOR ] GNL: All expected trading days are present. [2025-09-16T18:40:30.834690] [DEBUG] [VALIDATOR ] GNL: Primary data validation passed. [2025-09-16T18:40:30.835413] [INFO ] [MAIN ] GNL: Fetched primary data is valid. [2025-09-16T18:40:30.835461] [DEBUG] [LOCAL ] GNL: Saving primary data. [2025-09-16T18:40:30.837079] [DEBUG] [LOCAL ] GNL: Trimmed 51568 records to 34634 (keeping 150 days of data) [2025-09-16T18:40:31.045923] [DEBUG] [LOCAL ] GNL: Primary data saved (34634 records). [2025-09-16T18:40:31.046094] [INFO ] [MAIN ] GNL: Full history saved. [2025-09-16T18:40:31.046185] [INFO ] [MAIN ] Update Progress: 949/2369 symbols completed. GNL took 31.1s. Estimated remaining: 2h 30m 20s. Rate: 9.4 symbols/min [2025-09-16T18:40:31.222233] [INFO ] [MAIN ] GOF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:31.254469] [DEBUG] [LOCAL ] GOF: Loaded 30983 existing records from primary storage. [2025-09-16T18:40:31.254615] [INFO ] [MAIN ] GOF: Loaded 30983 existing records. [2025-09-16T18:40:31.437096] [WARN ] [MAIN ] GOF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:40:31.437336] [INFO ] [MAIN ] GOF: Wiping existing primary data before full refresh. [2025-09-16T18:40:31.437396] [INFO ] [LOCAL ] GOF: Starting deletion of primary data. [2025-09-16T18:40:31.438130] [DEBUG] [LOCAL ] GOF: Deleted primary data file. [2025-09-16T18:40:31.438184] [INFO ] [MAIN ] GOF: Starting full primary refresh. [2025-09-16T18:40:31.590269] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-08. [2025-09-16T18:40:31.747142] [INFO ] [REMOTE ] GNTY: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-09-16T18:40:31.809861] [INFO ] [REMOTE ] GOF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:31.844846] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-09-16T18:40:32.374899] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-02. [2025-09-16T18:40:33.174874] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-05. [2025-09-16T18:40:33.537345] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-09-16T18:40:33.950421] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T18:40:33.974855] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-03. [2025-09-16T18:40:34.774906] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-09. [2025-09-16T18:40:34.825344] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T18:40:35.575048] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-06. [2025-09-16T18:40:35.750315] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T18:40:35.936011] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-09-16T18:40:35.941446] [INFO ] [REMOTE ] GNRC: Full fetch complete (49251 records). [2025-09-16T18:40:35.941597] [DEBUG] [VALIDATOR ] GNRC: Validating primary data. [2025-09-16T18:40:36.343174] [DEBUG] [VALIDATOR ] GNRC: All expected trading days are present. [2025-09-16T18:40:36.343285] [DEBUG] [VALIDATOR ] GNRC: Primary data validation passed. [2025-09-16T18:40:36.344057] [INFO ] [MAIN ] GNRC: Fetched primary data is valid. [2025-09-16T18:40:36.344104] [DEBUG] [LOCAL ] GNRC: Saving primary data. [2025-09-16T18:40:36.345244] [DEBUG] [LOCAL ] GNRC: Trimmed 49251 records to 33506 (keeping 150 days of data) [2025-09-16T18:40:36.400175] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-04. [2025-09-16T18:40:36.552855] [DEBUG] [LOCAL ] GNRC: Primary data saved (33506 records). [2025-09-16T18:40:36.553007] [INFO ] [MAIN ] GNRC: Full history saved. [2025-09-16T18:40:36.553600] [INFO ] [MAIN ] Update Progress: 950/2369 symbols completed. GNRC took 34.7s. Estimated remaining: 2h 30m 12s. Rate: 9.4 symbols/min [2025-09-16T18:40:36.722624] [INFO ] [MAIN ] GOLF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:36.749172] [DEBUG] [LOCAL ] GOLF: Loaded 22665 existing records from primary storage. [2025-09-16T18:40:36.749337] [INFO ] [MAIN ] GOLF: Loaded 22665 existing records. [2025-09-16T18:40:36.926806] [WARN ] [MAIN ] GOLF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:40:36.927081] [INFO ] [MAIN ] GOLF: Wiping existing primary data before full refresh. [2025-09-16T18:40:36.927128] [INFO ] [LOCAL ] GOLF: Starting deletion of primary data. [2025-09-16T18:40:36.927787] [DEBUG] [LOCAL ] GOLF: Deleted primary data file. [2025-09-16T18:40:36.927839] [INFO ] [MAIN ] GOLF: Starting full primary refresh. [2025-09-16T18:40:37.180023] [INFO ] [REMOTE ] GNTY: Fetching primary data for 2025-08. [2025-09-16T18:40:37.289023] [INFO ] [REMOTE ] GOLF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:37.319255] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T18:40:37.794607] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T18:40:37.974909] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-02. [2025-09-16T18:40:38.317596] [INFO ] [REMOTE ] GNTY: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T18:40:38.774900] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-07. [2025-09-16T18:40:39.514029] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-09-16T18:40:39.574901] [INFO ] [REMOTE ] GNTY: Fetching primary data for 2025-09. [2025-09-16T18:40:40.327535] [INFO ] [REMOTE ] GNTY: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-09-16T18:40:40.329198] [INFO ] [REMOTE ] GNTY: Full fetch complete (7308 records). [2025-09-16T18:40:40.329275] [DEBUG] [VALIDATOR ] GNTY: Validating primary data. [2025-09-16T18:40:40.387208] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-05. [2025-09-16T18:40:40.701936] [DEBUG] [VALIDATOR ] GNTY: All expected trading days are present. [2025-09-16T18:40:40.702066] [DEBUG] [VALIDATOR ] GNTY: Primary data validation passed. [2025-09-16T18:40:40.702258] [INFO ] [MAIN ] GNTY: Fetched primary data is valid. [2025-09-16T18:40:40.702405] [DEBUG] [LOCAL ] GNTY: Saving primary data. [2025-09-16T18:40:40.702978] [DEBUG] [LOCAL ] GNTY: Trimmed 7308 records to 5679 (keeping 150 days of data) [2025-09-16T18:40:40.762620] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-09-16T18:40:40.779515] [DEBUG] [LOCAL ] GNTY: Primary data saved (5679 records). [2025-09-16T18:40:40.779732] [INFO ] [MAIN ] GNTY: Full history saved. [2025-09-16T18:40:40.779854] [INFO ] [MAIN ] Update Progress: 951/2369 symbols completed. GNTY took 23.2s. Estimated remaining: 2h 30m 3s. Rate: 9.5 symbols/min [2025-09-16T18:40:40.952874] [INFO ] [MAIN ] GOOS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:40.993056] [DEBUG] [LOCAL ] GOOS: Loaded 33701 existing records from primary storage. [2025-09-16T18:40:40.993239] [INFO ] [MAIN ] GOOS: Loaded 33701 existing records. [2025-09-16T18:40:41.170816] [WARN ] [MAIN ] GOOS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:40:41.171085] [INFO ] [MAIN ] GOOS: Wiping existing primary data before full refresh. [2025-09-16T18:40:41.171131] [INFO ] [LOCAL ] GOOS: Starting deletion of primary data. [2025-09-16T18:40:41.171834] [DEBUG] [LOCAL ] GOOS: Deleted primary data file. [2025-09-16T18:40:41.171882] [INFO ] [MAIN ] GOOS: Starting full primary refresh. [2025-09-16T18:40:41.182937] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-08. [2025-09-16T18:40:41.578244] [INFO ] [REMOTE ] GOOS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:41.974938] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-02. [2025-09-16T18:40:42.122911] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T18:40:42.774907] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-06. [2025-09-16T18:40:43.040199] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-09-16T18:40:43.413291] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T18:40:43.574932] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-03. [2025-09-16T18:40:43.982906] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T18:40:44.374895] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-09. [2025-09-16T18:40:45.007406] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-09-16T18:40:45.174906] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-03. [2025-09-16T18:40:45.688579] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T18:40:45.694119] [INFO ] [REMOTE ] GNW: Full fetch complete (55383 records). [2025-09-16T18:40:45.694239] [DEBUG] [VALIDATOR ] GNW: Validating primary data. [2025-09-16T18:40:45.694973] [FATAL] [VALIDATOR ] GNW: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:40:45.695139] [FATAL] [MAIN ] GNW: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:40:45.695211] [WARN ] [MAIN ] GNW: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:40:45.695280] [INFO ] [LOCAL ] GNW: Starting deletion of primary data. [2025-09-16T18:40:45.695350] [DEBUG] [LOCAL ] GNW: No primary data file found to delete. [2025-09-16T18:40:45.695388] [INFO ] [LOCAL ] GNW: Starting deletion of derivative data. [2025-09-16T18:40:45.695795] [DEBUG] [LOCAL ] GNW: Deleted derivative data file. [2025-09-16T18:40:45.695886] [INFO ] [MAIN ] Update Progress: 952/2369 symbols completed. GNW took 24.0s. Estimated remaining: 2h 29m 54s. Rate: 9.5 symbols/min [2025-09-16T18:40:45.876941] [INFO ] [MAIN ] GOTU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:45.904397] [DEBUG] [LOCAL ] GOTU: Loaded 27638 existing records from primary storage. [2025-09-16T18:40:45.904535] [INFO ] [MAIN ] GOTU: Loaded 27638 existing records. [2025-09-16T18:40:45.980024] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-04. [2025-09-16T18:40:46.082243] [WARN ] [MAIN ] GOTU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:40:46.082494] [INFO ] [MAIN ] GOTU: Wiping existing primary data before full refresh. [2025-09-16T18:40:46.082540] [INFO ] [LOCAL ] GOTU: Starting deletion of primary data. [2025-09-16T18:40:46.083118] [DEBUG] [LOCAL ] GOTU: Deleted primary data file. [2025-09-16T18:40:46.083165] [INFO ] [MAIN ] GOTU: Starting full primary refresh. [2025-09-16T18:40:46.433636] [INFO ] [REMOTE ] GOTU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:46.633258] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T18:40:46.774922] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-02. [2025-09-16T18:40:47.419628] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-09-16T18:40:47.574908] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-05. [2025-09-16T18:40:48.129972] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T18:40:48.374896] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-03. [2025-09-16T18:40:48.843698] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T18:40:49.174932] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-04. [2025-09-16T18:40:49.679463] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T18:40:49.974909] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-04. [2025-09-16T18:40:50.695078] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T18:40:50.774875] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-05. [2025-09-16T18:40:51.579974] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-06. [2025-09-16T18:40:51.592222] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T18:40:51.888451] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-09-16T18:40:52.374923] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-06. [2025-09-16T18:40:52.775636] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T18:40:53.174851] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-07. [2025-09-16T18:40:53.324631] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-09-16T18:40:53.974951] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-07. [2025-09-16T18:40:54.774907] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-05. [2025-09-16T18:40:55.253157] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-09-16T18:40:55.283893] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T18:40:55.574904] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-08. [2025-09-16T18:40:56.031405] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-09-16T18:40:56.374862] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-07. [2025-09-16T18:40:56.842371] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-09-16T18:40:57.174916] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-06. [2025-09-16T18:40:57.974941] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-09. [2025-09-16T18:40:58.074559] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T18:40:58.660482] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T18:40:58.774893] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-08. [2025-09-16T18:40:58.794755] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-09-16T18:40:58.798097] [INFO ] [REMOTE ] GOLF: Full fetch complete (35584 records). [2025-09-16T18:40:58.798204] [DEBUG] [VALIDATOR ] GOLF: Validating primary data. [2025-09-16T18:40:58.798820] [FATAL] [VALIDATOR ] GOLF: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:40:58.798900] [FATAL] [MAIN ] GOLF: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:40:58.798937] [WARN ] [MAIN ] GOLF: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:40:58.798984] [INFO ] [LOCAL ] GOLF: Starting deletion of primary data. [2025-09-16T18:40:58.799041] [DEBUG] [LOCAL ] GOLF: No primary data file found to delete. [2025-09-16T18:40:58.799076] [INFO ] [LOCAL ] GOLF: Starting deletion of derivative data. [2025-09-16T18:40:58.799391] [DEBUG] [LOCAL ] GOLF: Deleted derivative data file. [2025-09-16T18:40:58.799480] [INFO ] [MAIN ] Update Progress: 953/2369 symbols completed. GOLF took 22.2s. Estimated remaining: 2h 29m 58s. Rate: 9.4 symbols/min [2025-09-16T18:40:58.972337] [INFO ] [MAIN ] GPC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:40:59.010749] [DEBUG] [LOCAL ] GPC: Loaded 33780 existing records from primary storage. [2025-09-16T18:40:59.010889] [INFO ] [MAIN ] GPC: Loaded 33780 existing records. [2025-09-16T18:40:59.206268] [WARN ] [MAIN ] GPC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:40:59.206525] [INFO ] [MAIN ] GPC: Wiping existing primary data before full refresh. [2025-09-16T18:40:59.206573] [INFO ] [LOCAL ] GPC: Starting deletion of primary data. [2025-09-16T18:40:59.207567] [DEBUG] [LOCAL ] GPC: Deleted primary data file. [2025-09-16T18:40:59.207618] [INFO ] [MAIN ] GPC: Starting full primary refresh. [2025-09-16T18:40:59.557542] [INFO ] [REMOTE ] GPC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:40:59.574862] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-02. [2025-09-16T18:41:00.374868] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-08. [2025-09-16T18:41:00.443813] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:41:01.174936] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-09. [2025-09-16T18:41:01.832067] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-02 (2.26 seconds). [2025-09-16T18:41:01.883914] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T18:41:01.974863] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-09. [2025-09-16T18:41:02.581625] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T18:41:02.588013] [INFO ] [REMOTE ] GOOS: Full fetch complete (49741 records). [2025-09-16T18:41:02.588357] [DEBUG] [VALIDATOR ] GOOS: Validating primary data. [2025-09-16T18:41:02.589506] [FATAL] [VALIDATOR ] GOOS: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:41:02.589610] [FATAL] [MAIN ] GOOS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:41:02.589676] [WARN ] [MAIN ] GOOS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:41:02.589736] [INFO ] [LOCAL ] GOOS: Starting deletion of primary data. [2025-09-16T18:41:02.590056] [DEBUG] [LOCAL ] GOOS: No primary data file found to delete. [2025-09-16T18:41:02.590312] [INFO ] [LOCAL ] GOOS: Starting deletion of derivative data. [2025-09-16T18:41:02.590607] [DEBUG] [LOCAL ] GOOS: Deleted derivative data file. [2025-09-16T18:41:02.590837] [INFO ] [MAIN ] Update Progress: 954/2369 symbols completed. GOOS took 21.8s. Estimated remaining: 2h 29m 48s. Rate: 9.4 symbols/min [2025-09-16T18:41:02.768502] [INFO ] [MAIN ] GPI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:02.774912] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-07. [2025-09-16T18:41:02.781290] [DEBUG] [LOCAL ] GPI: Loaded 10095 existing records from primary storage. [2025-09-16T18:41:02.781450] [INFO ] [MAIN ] GPI: Loaded 10095 existing records. [2025-09-16T18:41:02.957379] [WARN ] [MAIN ] GPI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:41:02.957614] [INFO ] [MAIN ] GPI: Wiping existing primary data before full refresh. [2025-09-16T18:41:02.957690] [INFO ] [LOCAL ] GPI: Starting deletion of primary data. [2025-09-16T18:41:02.958054] [DEBUG] [LOCAL ] GPI: Deleted primary data file. [2025-09-16T18:41:02.958098] [INFO ] [MAIN ] GPI: Starting full primary refresh. [2025-09-16T18:41:03.304048] [INFO ] [REMOTE ] GPI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:03.574938] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-03. [2025-09-16T18:41:03.901439] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-09-16T18:41:03.906552] [INFO ] [REMOTE ] GOF: Full fetch complete (46828 records). [2025-09-16T18:41:03.906633] [DEBUG] [VALIDATOR ] GOF: Validating primary data. [2025-09-16T18:41:04.302501] [DEBUG] [VALIDATOR ] GOF: All expected trading days are present. [2025-09-16T18:41:04.302674] [DEBUG] [VALIDATOR ] GOF: Primary data validation passed. [2025-09-16T18:41:04.303616] [INFO ] [MAIN ] GOF: Fetched primary data is valid. [2025-09-16T18:41:04.304041] [DEBUG] [LOCAL ] GOF: Saving primary data. [2025-09-16T18:41:04.304768] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-09-16T18:41:04.313210] [DEBUG] [LOCAL ] GOF: Trimmed 46828 records to 31297 (keeping 150 days of data) [2025-09-16T18:41:04.377121] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-08. [2025-09-16T18:41:04.505120] [DEBUG] [LOCAL ] GOF: Primary data saved (31297 records). [2025-09-16T18:41:04.505246] [INFO ] [MAIN ] GOF: Full history saved. [2025-09-16T18:41:04.505330] [INFO ] [MAIN ] Update Progress: 955/2369 symbols completed. GOF took 33.5s. Estimated remaining: 2h 29m 35s. Rate: 9.5 symbols/min [2025-09-16T18:41:04.673211] [INFO ] [MAIN ] GPJA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:04.677510] [DEBUG] [LOCAL ] GPJA: Loaded 3082 existing records from primary storage. [2025-09-16T18:41:04.677587] [INFO ] [MAIN ] GPJA: Loaded 3082 existing records. [2025-09-16T18:41:04.854277] [WARN ] [MAIN ] GPJA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:41:04.854601] [INFO ] [MAIN ] GPJA: Wiping existing primary data before full refresh. [2025-09-16T18:41:04.854688] [INFO ] [LOCAL ] GPJA: Starting deletion of primary data. [2025-09-16T18:41:04.854889] [DEBUG] [LOCAL ] GPJA: Deleted primary data file. [2025-09-16T18:41:04.854933] [INFO ] [MAIN ] GPJA: Starting full primary refresh. [2025-09-16T18:41:05.181819] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-02. [2025-09-16T18:41:05.210077] [INFO ] [REMOTE ] GPJA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:05.334841] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T18:41:05.710790] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-09-16T18:41:05.970196] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T18:41:05.974834] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-03. [2025-09-16T18:41:06.774954] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-02. [2025-09-16T18:41:06.938815] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-09-16T18:41:07.567606] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T18:41:07.574835] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-03. [2025-09-16T18:41:07.965781] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-09-16T18:41:08.374901] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-04. [2025-09-16T18:41:08.828383] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-09-16T18:41:09.174922] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-09. [2025-09-16T18:41:09.974907] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-05. [2025-09-16T18:41:10.659532] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-09-16T18:41:10.673592] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:41:10.678516] [INFO ] [REMOTE ] GOTU: Full fetch complete (44002 records). [2025-09-16T18:41:10.678696] [DEBUG] [VALIDATOR ] GOTU: Validating primary data. [2025-09-16T18:41:10.780924] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-06. [2025-09-16T18:41:11.067492] [DEBUG] [VALIDATOR ] GOTU: All expected trading days are present. [2025-09-16T18:41:11.067593] [DEBUG] [VALIDATOR ] GOTU: Primary data validation passed. [2025-09-16T18:41:11.068318] [INFO ] [MAIN ] GOTU: Fetched primary data is valid. [2025-09-16T18:41:11.068366] [DEBUG] [LOCAL ] GOTU: Saving primary data. [2025-09-16T18:41:11.069506] [DEBUG] [LOCAL ] GOTU: Trimmed 44002 records to 27891 (keeping 150 days of data) [2025-09-16T18:41:11.241796] [DEBUG] [LOCAL ] GOTU: Primary data saved (27891 records). [2025-09-16T18:41:11.241951] [INFO ] [MAIN ] GOTU: Full history saved. [2025-09-16T18:41:11.242045] [INFO ] [MAIN ] Update Progress: 956/2369 symbols completed. GOTU took 25.5s. Estimated remaining: 2h 29m 29s. Rate: 9.5 symbols/min [2025-09-16T18:41:11.426902] [INFO ] [MAIN ] GPK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:11.466199] [DEBUG] [LOCAL ] GPK: Loaded 39013 existing records from primary storage. [2025-09-16T18:41:11.466365] [INFO ] [MAIN ] GPK: Loaded 39013 existing records. [2025-09-16T18:41:11.577726] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-04. [2025-09-16T18:41:11.579854] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-09-16T18:41:11.662212] [WARN ] [MAIN ] GPK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:41:11.662506] [INFO ] [MAIN ] GPK: Wiping existing primary data before full refresh. [2025-09-16T18:41:11.662555] [INFO ] [LOCAL ] GPK: Starting deletion of primary data. [2025-09-16T18:41:11.663371] [DEBUG] [LOCAL ] GPK: Deleted primary data file. [2025-09-16T18:41:11.663428] [INFO ] [MAIN ] GPK: Starting full primary refresh. [2025-09-16T18:41:12.029077] [INFO ] [REMOTE ] GPK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:12.374904] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-02. [2025-09-16T18:41:13.174950] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-04. [2025-09-16T18:41:13.681160] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-09-16T18:41:13.974908] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-05. [2025-09-16T18:41:14.100985] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T18:41:14.311906] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-09-16T18:41:14.774901] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-07. [2025-09-16T18:41:15.404938] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-09-16T18:41:15.530395] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T18:41:15.574872] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-06. [2025-09-16T18:41:16.374900] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-03. [2025-09-16T18:41:17.171886] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-09-16T18:41:17.180535] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-08. [2025-09-16T18:41:17.926035] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T18:41:17.974903] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-07. [2025-09-16T18:41:18.097457] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T18:41:18.774912] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-04. [2025-09-16T18:41:19.574909] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-09. [2025-09-16T18:41:19.625541] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T18:41:20.146466] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-09-16T18:41:20.147634] [INFO ] [REMOTE ] GPJA: Full fetch complete (4355 records). [2025-09-16T18:41:20.147730] [DEBUG] [VALIDATOR ] GPJA: Validating primary data. [2025-09-16T18:41:20.379290] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-05. [2025-09-16T18:41:20.508785] [DEBUG] [VALIDATOR ] GPJA: All expected trading days are present. [2025-09-16T18:41:20.508945] [DEBUG] [VALIDATOR ] GPJA: Primary data validation passed. [2025-09-16T18:41:20.509167] [INFO ] [MAIN ] GPJA: Fetched primary data is valid. [2025-09-16T18:41:20.509231] [DEBUG] [LOCAL ] GPJA: Saving primary data. [2025-09-16T18:41:20.509736] [DEBUG] [LOCAL ] GPJA: Trimmed 4355 records to 3120 (keeping 150 days of data) [2025-09-16T18:41:20.529303] [DEBUG] [LOCAL ] GPJA: Primary data saved (3120 records). [2025-09-16T18:41:20.529395] [INFO ] [MAIN ] GPJA: Full history saved. [2025-09-16T18:41:20.529576] [INFO ] [MAIN ] Update Progress: 957/2369 symbols completed. GPJA took 16.0s. Estimated remaining: 2h 29m 27s. Rate: 9.4 symbols/min [2025-09-16T18:41:20.558653] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-09-16T18:41:21.175033] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-08. [2025-09-16T18:41:21.497928] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T18:41:21.974990] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-06. [2025-09-16T18:41:22.719255] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T18:41:22.774849] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-09. [2025-09-16T18:41:23.111832] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:41:23.574869] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-07. [2025-09-16T18:41:24.180431] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T18:41:24.185718] [INFO ] [REMOTE ] GPC: Full fetch complete (50903 records). [2025-09-16T18:41:24.185855] [DEBUG] [VALIDATOR ] GPC: Validating primary data. [2025-09-16T18:41:24.186531] [FATAL] [VALIDATOR ] GPC: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:41:24.186610] [FATAL] [MAIN ] GPC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:41:24.186665] [WARN ] [MAIN ] GPC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:41:24.186712] [INFO ] [LOCAL ] GPC: Starting deletion of primary data. [2025-09-16T18:41:24.186793] [DEBUG] [LOCAL ] GPC: No primary data file found to delete. [2025-09-16T18:41:24.186832] [INFO ] [LOCAL ] GPC: Starting deletion of derivative data. [2025-09-16T18:41:24.187146] [DEBUG] [LOCAL ] GPC: Deleted derivative data file. [2025-09-16T18:41:24.188200] [INFO ] [MAIN ] Update Progress: 958/2369 symbols completed. GPC took 25.4s. Estimated remaining: 2h 29m 17s. Rate: 9.5 symbols/min [2025-09-16T18:41:24.374869] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-05. [2025-09-16T18:41:24.734409] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-09-16T18:41:25.174908] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-08. [2025-09-16T18:41:25.889859] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-09-16T18:41:25.974904] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-09. [2025-09-16T18:41:26.249881] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-09-16T18:41:26.774931] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-06. [2025-09-16T18:41:26.914801] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-09-16T18:41:26.917598] [INFO ] [REMOTE ] GPI: Full fetch complete (16146 records). [2025-09-16T18:41:26.917722] [DEBUG] [VALIDATOR ] GPI: Validating primary data. [2025-09-16T18:41:27.289257] [DEBUG] [VALIDATOR ] GPI: All expected trading days are present. [2025-09-16T18:41:27.289343] [DEBUG] [VALIDATOR ] GPI: Primary data validation passed. [2025-09-16T18:41:27.289759] [INFO ] [MAIN ] GPI: Fetched primary data is valid. [2025-09-16T18:41:27.289807] [DEBUG] [LOCAL ] GPI: Saving primary data. [2025-09-16T18:41:27.290403] [DEBUG] [LOCAL ] GPI: Trimmed 16146 records to 10230 (keeping 150 days of data) [2025-09-16T18:41:27.357834] [DEBUG] [LOCAL ] GPI: Primary data saved (10230 records). [2025-09-16T18:41:27.357988] [INFO ] [MAIN ] GPI: Full history saved. [2025-09-16T18:41:27.358077] [INFO ] [MAIN ] Update Progress: 959/2369 symbols completed. GPI took 24.8s. Estimated remaining: 2h 29m 6s. Rate: 9.5 symbols/min [2025-09-16T18:41:28.465958] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T18:41:28.476795] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-07. [2025-09-16T18:41:30.352357] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T18:41:30.364035] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-08. [2025-09-16T18:41:32.185705] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T18:41:32.198667] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-09. [2025-09-16T18:41:33.899359] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T18:41:33.907428] [INFO ] [REMOTE ] GPK: Full fetch complete (57080 records). [2025-09-16T18:41:33.907822] [DEBUG] [VALIDATOR ] GPK: Validating primary data. [2025-09-16T18:41:34.358816] [DEBUG] [VALIDATOR ] GPK: All expected trading days are present. [2025-09-16T18:41:34.358944] [DEBUG] [VALIDATOR ] GPK: Primary data validation passed. [2025-09-16T18:41:34.360172] [INFO ] [MAIN ] GPK: Fetched primary data is valid. [2025-09-16T18:41:34.360247] [DEBUG] [LOCAL ] GPK: Saving primary data. [2025-09-16T18:41:34.362302] [DEBUG] [LOCAL ] GPK: Trimmed 57080 records to 39401 (keeping 150 days of data) [2025-09-16T18:41:34.637397] [DEBUG] [LOCAL ] GPK: Primary data saved (39401 records). [2025-09-16T18:41:34.637573] [INFO ] [MAIN ] GPK: Full history saved. [2025-09-16T18:41:34.637727] [INFO ] [MAIN ] Update Progress: 960/2369 symbols completed. GPK took 23.4s. Estimated remaining: 2h 29m 1s. Rate: 9.5 symbols/min [2025-09-16T18:41:35.425816] [INFO ] [MAIN ] GPRK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:35.443121] [INFO ] [MAIN ] GPN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:35.421515] [INFO ] [MAIN ] GPOR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:35.469905] [INFO ] [MAIN ] GPMT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:35.507828] [DEBUG] [LOCAL ] GPRK: Loaded 28378 existing records from primary storage. [2025-09-16T18:41:35.508151] [INFO ] [MAIN ] GPRK: Loaded 28378 existing records. [2025-09-16T18:41:35.620092] [DEBUG] [LOCAL ] GPOR: Loaded 16284 existing records from primary storage. [2025-09-16T18:41:35.619962] [DEBUG] [LOCAL ] GPMT: Loaded 20224 existing records from primary storage. [2025-09-16T18:41:35.648400] [INFO ] [MAIN ] GPOR: Loaded 16284 existing records. [2025-09-16T18:41:35.663306] [INFO ] [MAIN ] GPMT: Loaded 20224 existing records. [2025-09-16T18:41:35.777598] [DEBUG] [LOCAL ] GPN: Loaded 38444 existing records from primary storage. [2025-09-16T18:41:35.853332] [WARN ] [MAIN ] GPRK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:41:35.847670] [INFO ] [MAIN ] GPN: Loaded 38444 existing records. [2025-09-16T18:41:35.952842] [INFO ] [MAIN ] GPRK: Wiping existing primary data before full refresh. [2025-09-16T18:41:35.997455] [INFO ] [LOCAL ] GPRK: Starting deletion of primary data. [2025-09-16T18:41:36.016629] [DEBUG] [LOCAL ] GPRK: Deleted primary data file. [2025-09-16T18:41:36.022214] [INFO ] [MAIN ] GPRK: Starting full primary refresh. [2025-09-16T18:41:36.315296] [WARN ] [MAIN ] GPMT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:41:36.438725] [WARN ] [MAIN ] GPN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:41:36.460225] [INFO ] [MAIN ] GPN: Wiping existing primary data before full refresh. [2025-09-16T18:41:36.460521] [INFO ] [LOCAL ] GPN: Starting deletion of primary data. [2025-09-16T18:41:36.460404] [WARN ] [MAIN ] GPOR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:41:36.465950] [INFO ] [MAIN ] GPMT: Wiping existing primary data before full refresh. [2025-09-16T18:41:36.502056] [INFO ] [MAIN ] GPOR: Wiping existing primary data before full refresh. [2025-09-16T18:41:36.501938] [DEBUG] [LOCAL ] GPN: Deleted primary data file. [2025-09-16T18:41:36.519425] [INFO ] [LOCAL ] GPOR: Starting deletion of primary data. [2025-09-16T18:41:36.520860] [INFO ] [LOCAL ] GPMT: Starting deletion of primary data. [2025-09-16T18:41:36.550776] [INFO ] [MAIN ] GPN: Starting full primary refresh. [2025-09-16T18:41:36.569225] [DEBUG] [LOCAL ] GPOR: Deleted primary data file. [2025-09-16T18:41:36.590306] [INFO ] [MAIN ] GPOR: Starting full primary refresh. [2025-09-16T18:41:36.605588] [DEBUG] [LOCAL ] GPMT: Deleted primary data file. [2025-09-16T18:41:36.736134] [INFO ] [MAIN ] GPMT: Starting full primary refresh. [2025-09-16T18:41:36.974327] [INFO ] [REMOTE ] GPRK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:37.008090] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-02. [2025-09-16T18:41:37.698368] [INFO ] [REMOTE ] GPN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:37.731606] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-02. [2025-09-16T18:41:37.772733] [INFO ] [REMOTE ] GPOR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:37.777210] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-02. [2025-09-16T18:41:37.858391] [INFO ] [REMOTE ] GPMT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:37.858778] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-02. [2025-09-16T18:41:38.958369] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-09-16T18:41:38.963291] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-03. [2025-09-16T18:41:39.113600] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T18:41:39.118612] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-03. [2025-09-16T18:41:39.144610] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-02 (2.14 seconds). [2025-09-16T18:41:39.152398] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-03. [2025-09-16T18:41:39.762095] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-02 (2.03 seconds). [2025-09-16T18:41:39.772033] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-03. [2025-09-16T18:41:39.815766] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-09-16T18:41:39.821362] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-04. [2025-09-16T18:41:40.274614] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T18:41:40.289898] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-04. [2025-09-16T18:41:40.305683] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T18:41:40.314781] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-04. [2025-09-16T18:41:41.002188] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T18:41:41.007579] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-05. [2025-09-16T18:41:41.627547] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-09-16T18:41:41.639029] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-04. [2025-09-16T18:41:41.781915] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T18:41:41.789940] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-05. [2025-09-16T18:41:41.874053] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T18:41:41.883548] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-05. [2025-09-16T18:41:42.019754] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T18:41:42.025057] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-06. [2025-09-16T18:41:42.848621] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-09-16T18:41:42.855422] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-06. [2025-09-16T18:41:43.074495] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T18:41:43.247129] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-09-16T18:41:43.341293] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T18:41:43.574936] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-07. [2025-09-16T18:41:43.779830] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-09-16T18:41:44.374931] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-05. [2025-09-16T18:41:44.648067] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-09-16T18:41:45.174942] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-08. [2025-09-16T18:41:45.974903] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-07. [2025-09-16T18:41:46.187010] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T18:41:46.378903] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-09-16T18:41:46.774902] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-06. [2025-09-16T18:41:47.086197] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-09-16T18:41:47.574934] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-06. [2025-09-16T18:41:48.380000] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-08. [2025-09-16T18:41:48.396568] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T18:41:49.009897] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T18:41:49.174911] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-09. [2025-09-16T18:41:49.548127] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T18:41:49.974940] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-09. [2025-09-16T18:41:50.417051] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T18:41:50.420699] [INFO ] [REMOTE ] GPOR: Full fetch complete (24992 records). [2025-09-16T18:41:50.420842] [DEBUG] [VALIDATOR ] GPOR: Validating primary data. [2025-09-16T18:41:50.787251] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-07. [2025-09-16T18:41:50.800181] [DEBUG] [VALIDATOR ] GPOR: All expected trading days are present. [2025-09-16T18:41:50.800262] [DEBUG] [VALIDATOR ] GPOR: Primary data validation passed. [2025-09-16T18:41:50.800857] [INFO ] [MAIN ] GPOR: Fetched primary data is valid. [2025-09-16T18:41:50.800928] [DEBUG] [LOCAL ] GPOR: Saving primary data. [2025-09-16T18:41:50.801941] [DEBUG] [LOCAL ] GPOR: Trimmed 24992 records to 16532 (keeping 150 days of data) [2025-09-16T18:41:50.910513] [DEBUG] [LOCAL ] GPOR: Primary data saved (16532 records). [2025-09-16T18:41:50.910700] [INFO ] [MAIN ] GPOR: Full history saved. [2025-09-16T18:41:50.910793] [INFO ] [MAIN ] Update Progress: 961/2369 symbols completed. GPOR took 16.3s. Estimated remaining: 2h 29m 9s. Rate: 9.4 symbols/min [2025-09-16T18:41:51.046085] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-09-16T18:41:51.110361] [INFO ] [MAIN ] GRBK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:51.110460] [INFO ] [REMOTE ] GPMT: Full fetch complete (29124 records). [2025-09-16T18:41:51.110679] [DEBUG] [VALIDATOR ] GPMT: Validating primary data. [2025-09-16T18:41:51.315447] [DEBUG] [LOCAL ] GRBK: Loaded 19511 existing records from primary storage. [2025-09-16T18:41:51.375475] [INFO ] [MAIN ] GRBK: Loaded 19511 existing records. [2025-09-16T18:41:51.597283] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-07. [2025-09-16T18:41:51.630534] [DEBUG] [VALIDATOR ] GPMT: All expected trading days are present. [2025-09-16T18:41:51.666576] [DEBUG] [VALIDATOR ] GPMT: Primary data validation passed. [2025-09-16T18:41:51.687563] [INFO ] [MAIN ] GPMT: Fetched primary data is valid. [2025-09-16T18:41:51.686972] [WARN ] [MAIN ] GRBK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:41:51.687689] [DEBUG] [LOCAL ] GPMT: Saving primary data. [2025-09-16T18:41:51.687812] [INFO ] [MAIN ] GRBK: Wiping existing primary data before full refresh. [2025-09-16T18:41:51.687965] [INFO ] [LOCAL ] GRBK: Starting deletion of primary data. [2025-09-16T18:41:51.688533] [DEBUG] [LOCAL ] GRBK: Deleted primary data file. [2025-09-16T18:41:51.688495] [DEBUG] [LOCAL ] GPMT: Trimmed 29124 records to 20304 (keeping 150 days of data) [2025-09-16T18:41:51.688610] [INFO ] [MAIN ] GRBK: Starting full primary refresh. [2025-09-16T18:41:52.169513] [INFO ] [REMOTE ] GRBK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:52.170084] [DEBUG] [LOCAL ] GPMT: Primary data saved (20304 records). [2025-09-16T18:41:52.170538] [INFO ] [MAIN ] GPMT: Full history saved. [2025-09-16T18:41:52.170630] [INFO ] [MAIN ] Update Progress: 962/2369 symbols completed. GPMT took 17.5s. Estimated remaining: 2h 28m 55s. Rate: 9.4 symbols/min [2025-09-16T18:41:52.349733] [INFO ] [MAIN ] GRC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:52.376789] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-02. [2025-09-16T18:41:52.380060] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T18:41:52.383604] [DEBUG] [LOCAL ] GRC: Loaded 6266 existing records from primary storage. [2025-09-16T18:41:52.391691] [INFO ] [MAIN ] GRC: Loaded 6266 existing records. [2025-09-16T18:41:52.575687] [WARN ] [MAIN ] GRC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:41:52.575972] [INFO ] [MAIN ] GRC: Wiping existing primary data before full refresh. [2025-09-16T18:41:52.576021] [INFO ] [LOCAL ] GRC: Starting deletion of primary data. [2025-09-16T18:41:52.576547] [DEBUG] [LOCAL ] GRC: Deleted primary data file. [2025-09-16T18:41:52.576594] [INFO ] [MAIN ] GRC: Starting full primary refresh. [2025-09-16T18:41:52.964172] [INFO ] [REMOTE ] GRC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:53.174959] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-08. [2025-09-16T18:41:53.472739] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T18:41:53.960127] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T18:41:53.974842] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-03. [2025-09-16T18:41:54.265822] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-09-16T18:41:54.774879] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-09. [2025-09-16T18:41:55.243558] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T18:41:55.574905] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-08. [2025-09-16T18:41:56.061845] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T18:41:56.066246] [INFO ] [REMOTE ] GPRK: Full fetch complete (42571 records). [2025-09-16T18:41:56.066340] [DEBUG] [VALIDATOR ] GPRK: Validating primary data. [2025-09-16T18:41:56.380038] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-02. [2025-09-16T18:41:56.461971] [DEBUG] [VALIDATOR ] GPRK: All expected trading days are present. [2025-09-16T18:41:56.462062] [DEBUG] [VALIDATOR ] GPRK: Primary data validation passed. [2025-09-16T18:41:56.462765] [INFO ] [MAIN ] GPRK: Fetched primary data is valid. [2025-09-16T18:41:56.462818] [DEBUG] [LOCAL ] GPRK: Saving primary data. [2025-09-16T18:41:56.463807] [DEBUG] [LOCAL ] GPRK: Trimmed 42571 records to 28709 (keeping 150 days of data) [2025-09-16T18:41:56.632142] [DEBUG] [LOCAL ] GPRK: Primary data saved (28709 records). [2025-09-16T18:41:56.632285] [INFO ] [MAIN ] GPRK: Full history saved. [2025-09-16T18:41:56.632385] [INFO ] [MAIN ] Update Progress: 963/2369 symbols completed. GPRK took 22.0s. Estimated remaining: 2h 28m 46s. Rate: 9.5 symbols/min [2025-09-16T18:41:56.801374] [INFO ] [MAIN ] GRDN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:41:56.801554] [DEBUG] [LOCAL ] GRDN: No existing data file found in primary storage. [2025-09-16T18:41:56.801597] [WARN ] [MAIN ] GRDN: No existing data found, performing full refresh instead. [2025-09-16T18:41:56.801745] [INFO ] [MAIN ] GRDN: Wiping existing primary data before full refresh. [2025-09-16T18:41:56.801788] [INFO ] [LOCAL ] GRDN: Starting deletion of primary data. [2025-09-16T18:41:56.801829] [DEBUG] [LOCAL ] GRDN: No primary data file found to delete. [2025-09-16T18:41:56.801862] [INFO ] [MAIN ] GRDN: Starting full primary refresh. [2025-09-16T18:41:57.164089] [INFO ] [REMOTE ] GRDN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:41:57.174877] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-02. [2025-09-16T18:41:57.308259] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-09-16T18:41:57.325329] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-09-16T18:41:57.974898] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-04. [2025-09-16T18:41:58.112039] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-09-16T18:41:58.774914] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-03. [2025-09-16T18:41:59.291912] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T18:41:59.441212] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-09-16T18:41:59.574946] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-05. [2025-09-16T18:42:00.374916] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-09. [2025-09-16T18:42:00.688342] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-09-16T18:42:01.174933] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-03. [2025-09-16T18:42:01.974949] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-04. [2025-09-16T18:42:02.001501] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T18:42:02.661323] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-09 (2.29 seconds). [2025-09-16T18:42:02.666871] [INFO ] [REMOTE ] GPN: Full fetch complete (56262 records). [2025-09-16T18:42:02.666967] [DEBUG] [VALIDATOR ] GPN: Validating primary data. [2025-09-16T18:42:02.667681] [FATAL] [VALIDATOR ] GPN: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:42:02.667761] [FATAL] [MAIN ] GPN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:42:02.667798] [WARN ] [MAIN ] GPN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:42:02.667841] [INFO ] [LOCAL ] GPN: Starting deletion of primary data. [2025-09-16T18:42:02.667899] [DEBUG] [LOCAL ] GPN: No primary data file found to delete. [2025-09-16T18:42:02.667934] [INFO ] [LOCAL ] GPN: Starting deletion of derivative data. [2025-09-16T18:42:02.668199] [DEBUG] [LOCAL ] GPN: Deleted derivative data file. [2025-09-16T18:42:02.669163] [INFO ] [MAIN ] Update Progress: 964/2369 symbols completed. GPN took 28.0s. Estimated remaining: 2h 28m 39s. Rate: 9.5 symbols/min [2025-09-16T18:42:02.780035] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-06. [2025-09-16T18:42:02.847452] [INFO ] [MAIN ] GRMN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:42:02.881634] [DEBUG] [LOCAL ] GRMN: Loaded 29826 existing records from primary storage. [2025-09-16T18:42:02.881838] [INFO ] [MAIN ] GRMN: Loaded 29826 existing records. [2025-09-16T18:42:03.059716] [WARN ] [MAIN ] GRMN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:42:03.060025] [INFO ] [MAIN ] GRMN: Wiping existing primary data before full refresh. [2025-09-16T18:42:03.065442] [INFO ] [LOCAL ] GRMN: Starting deletion of primary data. [2025-09-16T18:42:03.070102] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-09-16T18:42:03.073299] [DEBUG] [LOCAL ] GRMN: Deleted primary data file. [2025-09-16T18:42:03.073738] [INFO ] [MAIN ] GRMN: Starting full primary refresh. [2025-09-16T18:42:03.423691] [INFO ] [REMOTE ] GRMN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:42:03.574920] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-02. [2025-09-16T18:42:04.187732] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T18:42:04.374896] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-07. [2025-09-16T18:42:05.031458] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-09-16T18:42:05.174921] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-04. [2025-09-16T18:42:05.299801] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-09-16T18:42:05.974904] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-05. [2025-09-16T18:42:06.092771] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-09-16T18:42:06.774922] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-05. [2025-09-16T18:42:07.039092] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-09-16T18:42:07.575115] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-06. [2025-09-16T18:42:07.684844] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-09-16T18:42:08.374938] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-08. [2025-09-16T18:42:08.748430] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-09-16T18:42:09.174914] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-06. [2025-09-16T18:42:09.281225] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-09-16T18:42:09.917908] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-09-16T18:42:09.974917] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-07. [2025-09-16T18:42:10.774902] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-03. [2025-09-16T18:42:10.786114] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-09-16T18:42:11.574938] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-08. [2025-09-16T18:42:12.374914] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-09. [2025-09-16T18:42:12.418503] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T18:42:12.446029] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-09-16T18:42:13.174863] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-04. [2025-09-16T18:42:13.421489] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T18:42:13.424744] [INFO ] [REMOTE ] GRBK: Full fetch complete (30101 records). [2025-09-16T18:42:13.424855] [DEBUG] [VALIDATOR ] GRBK: Validating primary data. [2025-09-16T18:42:13.789903] [DEBUG] [VALIDATOR ] GRBK: All expected trading days are present. [2025-09-16T18:42:13.789987] [DEBUG] [VALIDATOR ] GRBK: Primary data validation passed. [2025-09-16T18:42:13.790577] [INFO ] [MAIN ] GRBK: Fetched primary data is valid. [2025-09-16T18:42:13.790658] [DEBUG] [LOCAL ] GRBK: Saving primary data. [2025-09-16T18:42:13.791443] [DEBUG] [LOCAL ] GRBK: Trimmed 30101 records to 19673 (keeping 150 days of data) [2025-09-16T18:42:13.904849] [DEBUG] [LOCAL ] GRBK: Primary data saved (19673 records). [2025-09-16T18:42:13.904997] [INFO ] [MAIN ] GRBK: Full history saved. [2025-09-16T18:42:13.905086] [INFO ] [MAIN ] Update Progress: 965/2369 symbols completed. GRBK took 23.0s. Estimated remaining: 2h 28m 40s. Rate: 9.4 symbols/min [2025-09-16T18:42:13.980145] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-07. [2025-09-16T18:42:14.079779] [INFO ] [MAIN ] GRND: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:42:14.114728] [DEBUG] [LOCAL ] GRND: Loaded 36943 existing records from primary storage. [2025-09-16T18:42:14.114878] [INFO ] [MAIN ] GRND: Loaded 36943 existing records. [2025-09-16T18:42:14.290024] [WARN ] [MAIN ] GRND: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:42:14.290285] [INFO ] [MAIN ] GRND: Wiping existing primary data before full refresh. [2025-09-16T18:42:14.290331] [INFO ] [LOCAL ] GRND: Starting deletion of primary data. [2025-09-16T18:42:14.291402] [DEBUG] [LOCAL ] GRND: Deleted primary data file. [2025-09-16T18:42:14.291454] [INFO ] [MAIN ] GRND: Starting full primary refresh. [2025-09-16T18:42:14.642980] [INFO ] [REMOTE ] GRND: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:42:14.678141] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T18:42:14.718503] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-09-16T18:42:14.774911] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-08. [2025-09-16T18:42:15.574904] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-02. [2025-09-16T18:42:15.845832] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T18:42:16.374916] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-09. [2025-09-16T18:42:17.107176] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T18:42:17.174914] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-03. [2025-09-16T18:42:17.469821] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T18:42:17.473751] [INFO ] [REMOTE ] GRDN: Full fetch complete (19681 records). [2025-09-16T18:42:17.473825] [DEBUG] [VALIDATOR ] GRDN: Validating primary data. [2025-09-16T18:42:17.837451] [DEBUG] [VALIDATOR ] GRDN: All expected trading days are present. [2025-09-16T18:42:17.837531] [DEBUG] [VALIDATOR ] GRDN: Primary data validation passed. [2025-09-16T18:42:17.837973] [INFO ] [MAIN ] GRDN: Fetched primary data is valid. [2025-09-16T18:42:17.838019] [DEBUG] [LOCAL ] GRDN: Saving primary data. [2025-09-16T18:42:17.838677] [DEBUG] [LOCAL ] GRDN: Trimmed 19681 records to 15220 (keeping 150 days of data) [2025-09-16T18:42:17.929106] [DEBUG] [LOCAL ] GRDN: Primary data saved (15220 records). [2025-09-16T18:42:17.929250] [INFO ] [MAIN ] GRDN: Full history saved. [2025-09-16T18:42:17.929345] [INFO ] [MAIN ] Update Progress: 966/2369 symbols completed. GRDN took 21.3s. Estimated remaining: 2h 28m 30s. Rate: 9.4 symbols/min [2025-09-16T18:42:17.982157] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-09. [2025-09-16T18:42:18.112945] [INFO ] [MAIN ] GRNT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:42:18.135919] [DEBUG] [LOCAL ] GRNT: Loaded 24663 existing records from primary storage. [2025-09-16T18:42:18.136068] [INFO ] [MAIN ] GRNT: Loaded 24663 existing records. [2025-09-16T18:42:18.313594] [WARN ] [MAIN ] GRNT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:42:18.313863] [INFO ] [MAIN ] GRNT: Wiping existing primary data before full refresh. [2025-09-16T18:42:18.313972] [INFO ] [LOCAL ] GRNT: Starting deletion of primary data. [2025-09-16T18:42:18.314849] [DEBUG] [LOCAL ] GRNT: Deleted primary data file. [2025-09-16T18:42:18.314900] [INFO ] [MAIN ] GRNT: Starting full primary refresh. [2025-09-16T18:42:18.672244] [INFO ] [REMOTE ] GRNT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:42:18.774904] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-02. [2025-09-16T18:42:18.868793] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T18:42:18.870887] [INFO ] [REMOTE ] GRC: Full fetch complete (9653 records). [2025-09-16T18:42:18.871062] [DEBUG] [VALIDATOR ] GRC: Validating primary data. [2025-09-16T18:42:19.073596] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-09-16T18:42:19.284206] [DEBUG] [VALIDATOR ] GRC: All expected trading days are present. [2025-09-16T18:42:19.284285] [DEBUG] [VALIDATOR ] GRC: Primary data validation passed. [2025-09-16T18:42:19.284526] [INFO ] [MAIN ] GRC: Fetched primary data is valid. [2025-09-16T18:42:19.284565] [DEBUG] [LOCAL ] GRC: Saving primary data. [2025-09-16T18:42:19.285014] [DEBUG] [LOCAL ] GRC: Trimmed 9653 records to 6338 (keeping 150 days of data) [2025-09-16T18:42:19.323689] [DEBUG] [LOCAL ] GRC: Primary data saved (6338 records). [2025-09-16T18:42:19.323794] [INFO ] [MAIN ] GRC: Full history saved. [2025-09-16T18:42:19.323880] [INFO ] [MAIN ] Update Progress: 967/2369 symbols completed. GRC took 27.2s. Estimated remaining: 2h 28m 17s. Rate: 9.5 symbols/min [2025-09-16T18:42:19.491536] [INFO ] [MAIN ] GROV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:42:19.500032] [DEBUG] [LOCAL ] GROV: Loaded 6809 existing records from primary storage. [2025-09-16T18:42:19.500139] [INFO ] [MAIN ] GROV: Loaded 6809 existing records. [2025-09-16T18:42:19.580125] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-04. [2025-09-16T18:42:19.682036] [WARN ] [MAIN ] GROV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:42:19.682514] [INFO ] [MAIN ] GROV: Wiping existing primary data before full refresh. [2025-09-16T18:42:19.682565] [INFO ] [LOCAL ] GROV: Starting deletion of primary data. [2025-09-16T18:42:19.682831] [DEBUG] [LOCAL ] GROV: Deleted primary data file. [2025-09-16T18:42:19.682874] [INFO ] [MAIN ] GROV: Starting full primary refresh. [2025-09-16T18:42:20.037984] [INFO ] [REMOTE ] GROV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:42:20.096507] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-09-16T18:42:20.374925] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-02. [2025-09-16T18:42:21.083854] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T18:42:21.168345] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T18:42:21.174849] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-03. [2025-09-16T18:42:21.964261] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T18:42:21.974846] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-04. [2025-09-16T18:42:22.634718] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-09-16T18:42:22.774926] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-03. [2025-09-16T18:42:23.574919] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-05. [2025-09-16T18:42:23.994932] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-09-16T18:42:24.374924] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-04. [2025-09-16T18:42:24.979288] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T18:42:25.174903] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-05. [2025-09-16T18:42:25.527212] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T18:42:25.974908] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-05. [2025-09-16T18:42:26.000774] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T18:42:26.774903] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-06. [2025-09-16T18:42:27.263423] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-09-16T18:42:27.522871] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-09-16T18:42:27.574856] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-07. [2025-09-16T18:42:28.257706] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-09-16T18:42:28.374861] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-08. [2025-09-16T18:42:29.174932] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-06. [2025-09-16T18:42:29.194949] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-09-16T18:42:29.974915] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-05. [2025-09-16T18:42:30.769817] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T18:42:30.776907] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-09. [2025-09-16T18:42:31.493278] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-09-16T18:42:31.494532] [INFO ] [REMOTE ] GROV: Full fetch complete (10067 records). [2025-09-16T18:42:31.494604] [DEBUG] [VALIDATOR ] GROV: Validating primary data. [2025-09-16T18:42:31.575476] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-06. [2025-09-16T18:42:31.863079] [DEBUG] [VALIDATOR ] GROV: All expected trading days are present. [2025-09-16T18:42:31.863241] [DEBUG] [VALIDATOR ] GROV: Primary data validation passed. [2025-09-16T18:42:31.863548] [INFO ] [MAIN ] GROV: Fetched primary data is valid. [2025-09-16T18:42:31.863591] [DEBUG] [LOCAL ] GROV: Saving primary data. [2025-09-16T18:42:31.864169] [DEBUG] [LOCAL ] GROV: Trimmed 10067 records to 6863 (keeping 150 days of data) [2025-09-16T18:42:31.923418] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-09-16T18:42:31.945706] [DEBUG] [LOCAL ] GROV: Primary data saved (6863 records). [2025-09-16T18:42:31.945857] [INFO ] [MAIN ] GROV: Full history saved. [2025-09-16T18:42:31.945975] [INFO ] [MAIN ] Update Progress: 968/2369 symbols completed. GROV took 12.6s. Estimated remaining: 2h 28m 19s. Rate: 9.4 symbols/min [2025-09-16T18:42:32.121572] [INFO ] [MAIN ] GRX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:42:32.126475] [DEBUG] [LOCAL ] GRX: Loaded 3278 existing records from primary storage. [2025-09-16T18:42:32.126575] [INFO ] [MAIN ] GRX: Loaded 3278 existing records. [2025-09-16T18:42:32.307382] [WARN ] [MAIN ] GRX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:42:32.307623] [INFO ] [MAIN ] GRX: Wiping existing primary data before full refresh. [2025-09-16T18:42:32.307701] [INFO ] [LOCAL ] GRX: Starting deletion of primary data. [2025-09-16T18:42:32.308003] [DEBUG] [LOCAL ] GRX: Deleted primary data file. [2025-09-16T18:42:32.308050] [INFO ] [MAIN ] GRX: Starting full primary refresh. [2025-09-16T18:42:32.383570] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-06. [2025-09-16T18:42:32.678694] [INFO ] [REMOTE ] GRX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:42:33.174916] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-07. [2025-09-16T18:42:33.439997] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-09-16T18:42:33.976487] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-02. [2025-09-16T18:42:33.983965] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-09-16T18:42:34.370295] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T18:42:34.605542] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-09-16T18:42:34.774964] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-07. [2025-09-16T18:42:35.574904] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-03. [2025-09-16T18:42:36.137157] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-09-16T18:42:36.374880] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-08. [2025-09-16T18:42:36.413003] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T18:42:37.174898] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-07. [2025-09-16T18:42:37.811440] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T18:42:37.974897] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-09. [2025-09-16T18:42:38.774907] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-04. [2025-09-16T18:42:38.922634] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-09-16T18:42:39.115424] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-09-16T18:42:39.249998] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-09-16T18:42:39.254278] [INFO ] [REMOTE ] GRNT: Full fetch complete (36547 records). [2025-09-16T18:42:39.254374] [DEBUG] [VALIDATOR ] GRNT: Validating primary data. [2025-09-16T18:42:39.579980] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-05. [2025-09-16T18:42:39.637509] [DEBUG] [VALIDATOR ] GRNT: All expected trading days are present. [2025-09-16T18:42:39.637586] [DEBUG] [VALIDATOR ] GRNT: Primary data validation passed. [2025-09-16T18:42:39.638313] [INFO ] [MAIN ] GRNT: Fetched primary data is valid. [2025-09-16T18:42:39.638364] [DEBUG] [LOCAL ] GRNT: Saving primary data. [2025-09-16T18:42:39.639471] [DEBUG] [LOCAL ] GRNT: Trimmed 36547 records to 24876 (keeping 150 days of data) [2025-09-16T18:42:39.789930] [DEBUG] [LOCAL ] GRNT: Primary data saved (24876 records). [2025-09-16T18:42:39.790115] [INFO ] [MAIN ] GRNT: Full history saved. [2025-09-16T18:42:39.790225] [INFO ] [MAIN ] Update Progress: 969/2369 symbols completed. GRNT took 21.9s. Estimated remaining: 2h 28m 15s. Rate: 9.4 symbols/min [2025-09-16T18:42:39.967704] [INFO ] [MAIN ] GS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:42:40.010081] [DEBUG] [LOCAL ] GS: Loaded 38263 existing records from primary storage. [2025-09-16T18:42:40.010245] [INFO ] [MAIN ] GS: Loaded 38263 existing records. [2025-09-16T18:42:40.190500] [WARN ] [MAIN ] GS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:42:40.190780] [INFO ] [MAIN ] GS: Wiping existing primary data before full refresh. [2025-09-16T18:42:40.190905] [INFO ] [LOCAL ] GS: Starting deletion of primary data. [2025-09-16T18:42:40.194471] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-09-16T18:42:40.195610] [DEBUG] [LOCAL ] GS: Deleted primary data file. [2025-09-16T18:42:40.196076] [INFO ] [MAIN ] GS: Starting full primary refresh. [2025-09-16T18:42:40.379997] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-08. [2025-09-16T18:42:40.554351] [INFO ] [REMOTE ] GS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:42:41.174900] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-02. [2025-09-16T18:42:41.974913] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-08. [2025-09-16T18:42:42.146075] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T18:42:42.774957] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-06. [2025-09-16T18:42:43.202976] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-09-16T18:42:43.251365] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-02 (2.08 seconds). [2025-09-16T18:42:43.432249] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-09-16T18:42:43.574950] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-09. [2025-09-16T18:42:44.374877] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-09. [2025-09-16T18:42:45.183731] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-07. [2025-09-16T18:42:45.185709] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T18:42:45.190850] [INFO ] [REMOTE ] GRMN: Full fetch complete (45739 records). [2025-09-16T18:42:45.190939] [DEBUG] [VALIDATOR ] GRMN: Validating primary data. [2025-09-16T18:42:45.586710] [DEBUG] [VALIDATOR ] GRMN: All expected trading days are present. [2025-09-16T18:42:45.586811] [DEBUG] [VALIDATOR ] GRMN: Primary data validation passed. [2025-09-16T18:42:45.587582] [INFO ] [MAIN ] GRMN: Fetched primary data is valid. [2025-09-16T18:42:45.587682] [DEBUG] [LOCAL ] GRMN: Saving primary data. [2025-09-16T18:42:45.588817] [DEBUG] [LOCAL ] GRMN: Trimmed 45739 records to 30105 (keeping 150 days of data) [2025-09-16T18:42:45.788663] [DEBUG] [LOCAL ] GRMN: Primary data saved (30105 records). [2025-09-16T18:42:45.788805] [INFO ] [MAIN ] GRMN: Full history saved. [2025-09-16T18:42:45.789505] [INFO ] [MAIN ] Update Progress: 970/2369 symbols completed. GRMN took 43.1s. Estimated remaining: 2h 28m 8s. Rate: 9.4 symbols/min [2025-09-16T18:42:45.963332] [INFO ] [MAIN ] GSBD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:42:45.963951] [DEBUG] [LOCAL ] GSBD: No existing data file found in primary storage. [2025-09-16T18:42:45.964123] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-09-16T18:42:45.964229] [WARN ] [MAIN ] GSBD: No existing data found, performing full refresh instead. [2025-09-16T18:42:45.966123] [INFO ] [MAIN ] GSBD: Wiping existing primary data before full refresh. [2025-09-16T18:42:45.966486] [INFO ] [LOCAL ] GSBD: Starting deletion of primary data. [2025-09-16T18:42:45.966535] [DEBUG] [LOCAL ] GSBD: No primary data file found to delete. [2025-09-16T18:42:45.966578] [INFO ] [MAIN ] GSBD: Starting full primary refresh. [2025-09-16T18:42:45.979954] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-08. [2025-09-16T18:42:46.349904] [INFO ] [REMOTE ] GSBD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:42:46.365957] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-09-16T18:42:46.372922] [INFO ] [REMOTE ] GRND: Full fetch complete (54214 records). [2025-09-16T18:42:46.373034] [DEBUG] [VALIDATOR ] GRND: Validating primary data. [2025-09-16T18:42:46.725774] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T18:42:46.788236] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-09. [2025-09-16T18:42:46.806480] [DEBUG] [VALIDATOR ] GRND: All expected trading days are present. [2025-09-16T18:42:46.806546] [DEBUG] [VALIDATOR ] GRND: Primary data validation passed. [2025-09-16T18:42:46.807344] [INFO ] [MAIN ] GRND: Fetched primary data is valid. [2025-09-16T18:42:46.807391] [DEBUG] [LOCAL ] GRND: Saving primary data. [2025-09-16T18:42:46.808755] [DEBUG] [LOCAL ] GRND: Trimmed 54214 records to 37247 (keeping 150 days of data) [2025-09-16T18:42:47.031160] [DEBUG] [LOCAL ] GRND: Primary data saved (37247 records). [2025-09-16T18:42:47.031306] [INFO ] [MAIN ] GRND: Full history saved. [2025-09-16T18:42:47.031418] [INFO ] [MAIN ] Update Progress: 971/2369 symbols completed. GRND took 33.1s. Estimated remaining: 2h 27m 55s. Rate: 9.5 symbols/min [2025-09-16T18:42:47.200176] [INFO ] [MAIN ] GSK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:42:47.238232] [DEBUG] [LOCAL ] GSK: Loaded 39497 existing records from primary storage. [2025-09-16T18:42:47.238391] [INFO ] [MAIN ] GSK: Loaded 39497 existing records. [2025-09-16T18:42:47.418113] [WARN ] [MAIN ] GSK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:42:47.418385] [INFO ] [MAIN ] GSK: Wiping existing primary data before full refresh. [2025-09-16T18:42:47.418434] [INFO ] [LOCAL ] GSK: Starting deletion of primary data. [2025-09-16T18:42:47.419355] [DEBUG] [LOCAL ] GSK: Deleted primary data file. [2025-09-16T18:42:47.419407] [INFO ] [MAIN ] GSK: Starting full primary refresh. [2025-09-16T18:42:47.524970] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-09-16T18:42:47.552094] [INFO ] [REMOTE ] GRX: Full fetch complete (4544 records). [2025-09-16T18:42:47.585400] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-02. [2025-09-16T18:42:47.575681] [DEBUG] [VALIDATOR ] GRX: Validating primary data. [2025-09-16T18:42:47.967259] [INFO ] [REMOTE ] GSK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:42:48.147317] [DEBUG] [VALIDATOR ] GRX: All expected trading days are present. [2025-09-16T18:42:48.147396] [DEBUG] [VALIDATOR ] GRX: Primary data validation passed. [2025-09-16T18:42:48.147533] [INFO ] [MAIN ] GRX: Fetched primary data is valid. [2025-09-16T18:42:48.147572] [DEBUG] [LOCAL ] GRX: Saving primary data. [2025-09-16T18:42:48.147955] [DEBUG] [LOCAL ] GRX: Trimmed 4544 records to 3340 (keeping 150 days of data) [2025-09-16T18:42:48.168203] [DEBUG] [LOCAL ] GRX: Primary data saved (3340 records). [2025-09-16T18:42:48.168266] [INFO ] [MAIN ] GRX: Full history saved. [2025-09-16T18:42:48.168353] [INFO ] [MAIN ] Update Progress: 972/2369 symbols completed. GRX took 16.2s. Estimated remaining: 2h 27m 41s. Rate: 9.5 symbols/min [2025-09-16T18:42:48.338151] [INFO ] [MAIN ] GSL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:42:48.362566] [DEBUG] [LOCAL ] GSL: Loaded 24348 existing records from primary storage. [2025-09-16T18:42:48.362714] [INFO ] [MAIN ] GSL: Loaded 24348 existing records. [2025-09-16T18:42:48.383107] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-02. [2025-09-16T18:42:48.539375] [WARN ] [MAIN ] GSL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:42:48.539629] [INFO ] [MAIN ] GSL: Wiping existing primary data before full refresh. [2025-09-16T18:42:48.539711] [INFO ] [LOCAL ] GSL: Starting deletion of primary data. [2025-09-16T18:42:48.540403] [DEBUG] [LOCAL ] GSL: Deleted primary data file. [2025-09-16T18:42:48.540464] [INFO ] [MAIN ] GSL: Starting full primary refresh. [2025-09-16T18:42:48.902872] [INFO ] [REMOTE ] GSL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:42:49.116962] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T18:42:49.174874] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-03. [2025-09-16T18:42:49.974914] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-02. [2025-09-16T18:42:50.281388] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-09-16T18:42:50.774907] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-03. [2025-09-16T18:42:50.817311] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T18:42:51.394581] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T18:42:51.574901] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-04. [2025-09-16T18:42:52.374906] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-03. [2025-09-16T18:42:52.538776] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T18:42:53.168265] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-09-16T18:42:53.177150] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-03. [2025-09-16T18:42:53.974904] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-04. [2025-09-16T18:42:54.511900] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T18:42:54.635634] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-03 (2.26 seconds). [2025-09-16T18:42:54.774901] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-04. [2025-09-16T18:42:55.574896] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-04. [2025-09-16T18:42:55.911902] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-09-16T18:42:56.374897] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-05. [2025-09-16T18:42:56.995281] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-04 (2.22 seconds). [2025-09-16T18:42:57.031437] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T18:42:57.174942] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-05. [2025-09-16T18:42:57.940340] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T18:42:57.974856] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-06. [2025-09-16T18:42:58.597033] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-09-16T18:42:58.774911] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-05. [2025-09-16T18:42:59.574936] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-06. [2025-09-16T18:42:59.755730] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-09-16T18:43:00.374878] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-07. [2025-09-16T18:43:00.616328] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T18:43:01.065443] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T18:43:01.174863] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-07. [2025-09-16T18:43:01.974925] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-06. [2025-09-16T18:43:02.019173] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T18:43:02.774935] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-05. [2025-09-16T18:43:02.956191] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-09-16T18:43:03.390601] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T18:43:03.574961] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-07. [2025-09-16T18:43:04.374936] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-08. [2025-09-16T18:43:04.842464] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-09-16T18:43:05.090180] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-09-16T18:43:05.174868] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-06. [2025-09-16T18:43:05.974864] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-08. [2025-09-16T18:43:06.076959] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-09-16T18:43:06.775009] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-09. [2025-09-16T18:43:07.005670] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T18:43:07.356263] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T18:43:07.574941] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-09. [2025-09-16T18:43:08.294926] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T18:43:08.303351] [INFO ] [REMOTE ] GSK: Full fetch complete (58180 records). [2025-09-16T18:43:08.303542] [DEBUG] [VALIDATOR ] GSK: Validating primary data. [2025-09-16T18:43:08.379990] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-08. [2025-09-16T18:43:08.689911] [DEBUG] [VALIDATOR ] GSK: All expected trading days are present. [2025-09-16T18:43:08.690000] [DEBUG] [VALIDATOR ] GSK: Primary data validation passed. [2025-09-16T18:43:08.690888] [INFO ] [MAIN ] GSK: Fetched primary data is valid. [2025-09-16T18:43:08.690934] [DEBUG] [LOCAL ] GSK: Saving primary data. [2025-09-16T18:43:08.692309] [DEBUG] [LOCAL ] GSK: Trimmed 58180 records to 39887 (keeping 150 days of data) [2025-09-16T18:43:08.936732] [DEBUG] [LOCAL ] GSK: Primary data saved (39887 records). [2025-09-16T18:43:08.936974] [INFO ] [MAIN ] GSK: Full history saved. [2025-09-16T18:43:08.937104] [INFO ] [MAIN ] Update Progress: 973/2369 symbols completed. GSK took 21.9s. Estimated remaining: 2h 27m 55s. Rate: 9.4 symbols/min [2025-09-16T18:43:09.052308] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T18:43:09.104085] [INFO ] [REMOTE ] GSBD: Full fetch complete (42570 records). [2025-09-16T18:43:09.127570] [INFO ] [MAIN ] GTES: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:43:09.127740] [DEBUG] [VALIDATOR ] GSBD: Validating primary data. [2025-09-16T18:43:09.177565] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-07. [2025-09-16T18:43:09.419032] [DEBUG] [LOCAL ] GTES: Loaded 37017 existing records from primary storage. [2025-09-16T18:43:09.458226] [INFO ] [MAIN ] GTES: Loaded 37017 existing records. [2025-09-16T18:43:09.721588] [DEBUG] [VALIDATOR ] GSBD: All expected trading days are present. [2025-09-16T18:43:09.746328] [DEBUG] [VALIDATOR ] GSBD: Primary data validation passed. [2025-09-16T18:43:09.750454] [INFO ] [MAIN ] GSBD: Fetched primary data is valid. [2025-09-16T18:43:09.749708] [WARN ] [MAIN ] GTES: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:43:09.750723] [DEBUG] [LOCAL ] GSBD: Saving primary data. [2025-09-16T18:43:09.750844] [INFO ] [MAIN ] GTES: Wiping existing primary data before full refresh. [2025-09-16T18:43:09.751102] [INFO ] [LOCAL ] GTES: Starting deletion of primary data. [2025-09-16T18:43:09.752145] [DEBUG] [LOCAL ] GSBD: Trimmed 42570 records to 28138 (keeping 150 days of data) [2025-09-16T18:43:09.752199] [DEBUG] [LOCAL ] GTES: Deleted primary data file. [2025-09-16T18:43:09.752333] [INFO ] [MAIN ] GTES: Starting full primary refresh. [2025-09-16T18:43:10.244509] [INFO ] [REMOTE ] GTES: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:43:10.256825] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-02. [2025-09-16T18:43:10.258451] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-09-16T18:43:10.323866] [DEBUG] [LOCAL ] GSBD: Primary data saved (28138 records). [2025-09-16T18:43:10.324036] [INFO ] [MAIN ] GSBD: Full history saved. [2025-09-16T18:43:10.324117] [INFO ] [MAIN ] Update Progress: 974/2369 symbols completed. GSBD took 24.5s. Estimated remaining: 2h 27m 42s. Rate: 9.4 symbols/min [2025-09-16T18:43:10.494242] [INFO ] [MAIN ] GTLS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:43:10.528887] [DEBUG] [LOCAL ] GTLS: Loaded 31961 existing records from primary storage. [2025-09-16T18:43:10.529048] [INFO ] [MAIN ] GTLS: Loaded 31961 existing records. [2025-09-16T18:43:10.707154] [WARN ] [MAIN ] GTLS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:43:10.707455] [INFO ] [MAIN ] GTLS: Wiping existing primary data before full refresh. [2025-09-16T18:43:10.707519] [INFO ] [LOCAL ] GTLS: Starting deletion of primary data. [2025-09-16T18:43:10.708286] [DEBUG] [LOCAL ] GTLS: Deleted primary data file. [2025-09-16T18:43:10.708360] [INFO ] [MAIN ] GTLS: Starting full primary refresh. [2025-09-16T18:43:10.785468] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-09. [2025-09-16T18:43:11.062238] [INFO ] [REMOTE ] GTLS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:43:11.219794] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-09-16T18:43:11.574920] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-08. [2025-09-16T18:43:11.678063] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T18:43:12.208668] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T18:43:12.212860] [INFO ] [REMOTE ] GSL: Full fetch complete (37068 records). [2025-09-16T18:43:12.212943] [DEBUG] [VALIDATOR ] GSL: Validating primary data. [2025-09-16T18:43:12.378733] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-03. [2025-09-16T18:43:12.597832] [DEBUG] [VALIDATOR ] GSL: All expected trading days are present. [2025-09-16T18:43:12.597914] [DEBUG] [VALIDATOR ] GSL: Primary data validation passed. [2025-09-16T18:43:12.598579] [INFO ] [MAIN ] GSL: Fetched primary data is valid. [2025-09-16T18:43:12.598622] [DEBUG] [LOCAL ] GSL: Saving primary data. [2025-09-16T18:43:12.599428] [DEBUG] [LOCAL ] GSL: Trimmed 37068 records to 24570 (keeping 150 days of data) [2025-09-16T18:43:12.753802] [DEBUG] [LOCAL ] GSL: Primary data saved (24570 records). [2025-09-16T18:43:12.753964] [INFO ] [MAIN ] GSL: Full history saved. [2025-09-16T18:43:12.754055] [INFO ] [MAIN ] Update Progress: 975/2369 symbols completed. GSL took 24.6s. Estimated remaining: 2h 27m 30s. Rate: 9.5 symbols/min [2025-09-16T18:43:12.928858] [INFO ] [MAIN ] GTN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:43:12.964181] [DEBUG] [LOCAL ] GTN: Loaded 33382 existing records from primary storage. [2025-09-16T18:43:12.964345] [INFO ] [MAIN ] GTN: Loaded 33382 existing records. [2025-09-16T18:43:13.148578] [WARN ] [MAIN ] GTN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:43:13.148892] [INFO ] [MAIN ] GTN: Wiping existing primary data before full refresh. [2025-09-16T18:43:13.148943] [INFO ] [LOCAL ] GTN: Starting deletion of primary data. [2025-09-16T18:43:13.149724] [DEBUG] [LOCAL ] GTN: Deleted primary data file. [2025-09-16T18:43:13.149775] [INFO ] [MAIN ] GTN: Starting full primary refresh. [2025-09-16T18:43:13.181046] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-02. [2025-09-16T18:43:13.521543] [INFO ] [REMOTE ] GTN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:43:13.552277] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-09-16T18:43:13.930587] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T18:43:13.974845] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-04. [2025-09-16T18:43:14.771571] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T18:43:14.778891] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-02. [2025-09-16T18:43:15.574912] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-09. [2025-09-16T18:43:15.706995] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T18:43:16.374923] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-05. [2025-09-16T18:43:16.560620] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T18:43:17.174919] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-03. [2025-09-16T18:43:17.710256] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-09-16T18:43:17.718698] [INFO ] [REMOTE ] GS: Full fetch complete (56879 records). [2025-09-16T18:43:17.718943] [DEBUG] [VALIDATOR ] GS: Validating primary data. [2025-09-16T18:43:17.985464] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-03. [2025-09-16T18:43:18.064067] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-09-16T18:43:18.152185] [DEBUG] [VALIDATOR ] GS: All expected trading days are present. [2025-09-16T18:43:18.152387] [DEBUG] [VALIDATOR ] GS: Primary data validation passed. [2025-09-16T18:43:18.153276] [INFO ] [MAIN ] GS: Fetched primary data is valid. [2025-09-16T18:43:18.153341] [DEBUG] [LOCAL ] GS: Saving primary data. [2025-09-16T18:43:18.154723] [DEBUG] [LOCAL ] GS: Trimmed 56879 records to 38615 (keeping 150 days of data) [2025-09-16T18:43:18.413738] [DEBUG] [LOCAL ] GS: Primary data saved (38615 records). [2025-09-16T18:43:18.413890] [INFO ] [MAIN ] GS: Full history saved. [2025-09-16T18:43:18.413991] [INFO ] [MAIN ] Update Progress: 976/2369 symbols completed. GS took 38.6s. Estimated remaining: 2h 27m 22s. Rate: 9.5 symbols/min [2025-09-16T18:43:18.593919] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T18:43:18.619596] [INFO ] [MAIN ] GTY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:43:18.641000] [DEBUG] [LOCAL ] GTY: Loaded 21698 existing records from primary storage. [2025-09-16T18:43:18.641143] [INFO ] [MAIN ] GTY: Loaded 21698 existing records. [2025-09-16T18:43:18.787452] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-06. [2025-09-16T18:43:18.823813] [WARN ] [MAIN ] GTY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:43:18.824038] [INFO ] [MAIN ] GTY: Wiping existing primary data before full refresh. [2025-09-16T18:43:18.824084] [INFO ] [LOCAL ] GTY: Starting deletion of primary data. [2025-09-16T18:43:18.824712] [DEBUG] [LOCAL ] GTY: Deleted primary data file. [2025-09-16T18:43:18.824773] [INFO ] [MAIN ] GTY: Starting full primary refresh. [2025-09-16T18:43:19.174936] [INFO ] [REMOTE ] GTY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:43:19.574891] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-04. [2025-09-16T18:43:19.671728] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-09-16T18:43:20.374895] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-04. [2025-09-16T18:43:20.442404] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T18:43:20.908744] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T18:43:21.174916] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-02. [2025-09-16T18:43:21.974903] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-05. [2025-09-16T18:43:22.013291] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T18:43:22.558258] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T18:43:22.774907] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-05. [2025-09-16T18:43:23.234162] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-09-16T18:43:23.574954] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-03. [2025-09-16T18:43:24.331397] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T18:43:24.374947] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-06. [2025-09-16T18:43:24.934957] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T18:43:25.174950] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-07. [2025-09-16T18:43:25.974942] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-06. [2025-09-16T18:43:26.295398] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-09-16T18:43:26.774974] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-04. [2025-09-16T18:43:26.893504] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-09-16T18:43:27.498965] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T18:43:27.574955] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-08. [2025-09-16T18:43:27.702994] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T18:43:28.374977] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-05. [2025-09-16T18:43:28.747160] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T18:43:29.179795] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-07. [2025-09-16T18:43:29.182601] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-09-16T18:43:29.974901] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-06. [2025-09-16T18:43:30.774908] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-09. [2025-09-16T18:43:30.883616] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T18:43:30.955401] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T18:43:31.574913] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-07. [2025-09-16T18:43:32.320387] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-09-16T18:43:32.326698] [INFO ] [REMOTE ] GTES: Full fetch complete (55303 records). [2025-09-16T18:43:32.326847] [DEBUG] [VALIDATOR ] GTES: Validating primary data. [2025-09-16T18:43:32.385814] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-08. [2025-09-16T18:43:32.729555] [DEBUG] [VALIDATOR ] GTES: All expected trading days are present. [2025-09-16T18:43:32.729661] [DEBUG] [VALIDATOR ] GTES: Primary data validation passed. [2025-09-16T18:43:32.730412] [INFO ] [MAIN ] GTES: Fetched primary data is valid. [2025-09-16T18:43:32.730454] [DEBUG] [LOCAL ] GTES: Saving primary data. [2025-09-16T18:43:32.731670] [DEBUG] [LOCAL ] GTES: Trimmed 55303 records to 37387 (keeping 150 days of data) [2025-09-16T18:43:32.954552] [DEBUG] [LOCAL ] GTES: Primary data saved (37387 records). [2025-09-16T18:43:32.954721] [INFO ] [MAIN ] GTES: Full history saved. [2025-09-16T18:43:32.954811] [INFO ] [MAIN ] Update Progress: 977/2369 symbols completed. GTES took 24.0s. Estimated remaining: 2h 27m 28s. Rate: 9.4 symbols/min [2025-09-16T18:43:33.166146] [INFO ] [MAIN ] GUG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:43:33.176059] [DEBUG] [LOCAL ] GUG: Loaded 8754 existing records from primary storage. [2025-09-16T18:43:33.175843] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-07. [2025-09-16T18:43:33.176192] [INFO ] [MAIN ] GUG: Loaded 8754 existing records. [2025-09-16T18:43:33.371096] [WARN ] [MAIN ] GUG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:43:33.371493] [INFO ] [MAIN ] GUG: Wiping existing primary data before full refresh. [2025-09-16T18:43:33.392041] [INFO ] [LOCAL ] GUG: Starting deletion of primary data. [2025-09-16T18:43:33.395312] [DEBUG] [LOCAL ] GUG: Deleted primary data file. [2025-09-16T18:43:33.396415] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T18:43:33.396480] [INFO ] [MAIN ] GUG: Starting full primary refresh. [2025-09-16T18:43:33.790575] [INFO ] [REMOTE ] GUG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:43:33.935130] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T18:43:33.974910] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-09. [2025-09-16T18:43:34.339833] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-09-16T18:43:34.774909] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-08. [2025-09-16T18:43:35.276447] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T18:43:35.282239] [INFO ] [REMOTE ] GTN: Full fetch complete (49879 records). [2025-09-16T18:43:35.282411] [DEBUG] [VALIDATOR ] GTN: Validating primary data. [2025-09-16T18:43:35.580282] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-02. [2025-09-16T18:43:35.690993] [DEBUG] [VALIDATOR ] GTN: All expected trading days are present. [2025-09-16T18:43:35.691093] [DEBUG] [VALIDATOR ] GTN: Primary data validation passed. [2025-09-16T18:43:35.691939] [INFO ] [MAIN ] GTN: Fetched primary data is valid. [2025-09-16T18:43:35.691989] [DEBUG] [LOCAL ] GTN: Saving primary data. [2025-09-16T18:43:35.693354] [DEBUG] [LOCAL ] GTN: Trimmed 49879 records to 33708 (keeping 150 days of data) [2025-09-16T18:43:35.898551] [DEBUG] [LOCAL ] GTN: Primary data saved (33708 records). [2025-09-16T18:43:35.898715] [INFO ] [MAIN ] GTN: Full history saved. [2025-09-16T18:43:35.898803] [INFO ] [MAIN ] Update Progress: 978/2369 symbols completed. GTN took 23.1s. Estimated remaining: 2h 27m 16s. Rate: 9.4 symbols/min [2025-09-16T18:43:36.081088] [INFO ] [MAIN ] GUT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:43:36.097886] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-09-16T18:43:36.113850] [DEBUG] [LOCAL ] GUT: Loaded 11442 existing records from primary storage. [2025-09-16T18:43:36.114030] [INFO ] [MAIN ] GUT: Loaded 11442 existing records. [2025-09-16T18:43:36.303895] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-09-16T18:43:36.309259] [WARN ] [MAIN ] GUT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:43:36.312884] [INFO ] [MAIN ] GUT: Wiping existing primary data before full refresh. [2025-09-16T18:43:36.312939] [INFO ] [LOCAL ] GUT: Starting deletion of primary data. [2025-09-16T18:43:36.313372] [DEBUG] [LOCAL ] GUT: Deleted primary data file. [2025-09-16T18:43:36.313418] [INFO ] [MAIN ] GUT: Starting full primary refresh. [2025-09-16T18:43:36.380019] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-03. [2025-09-16T18:43:36.685855] [INFO ] [REMOTE ] GUT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:43:37.046015] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-09-16T18:43:37.174907] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-09. [2025-09-16T18:43:37.974861] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-08. [2025-09-16T18:43:38.034429] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T18:43:38.037811] [INFO ] [REMOTE ] GTY: Full fetch complete (32954 records). [2025-09-16T18:43:38.037888] [DEBUG] [VALIDATOR ] GTY: Validating primary data. [2025-09-16T18:43:38.409164] [DEBUG] [VALIDATOR ] GTY: All expected trading days are present. [2025-09-16T18:43:38.409243] [DEBUG] [VALIDATOR ] GTY: Primary data validation passed. [2025-09-16T18:43:38.409823] [INFO ] [MAIN ] GTY: Fetched primary data is valid. [2025-09-16T18:43:38.409869] [DEBUG] [LOCAL ] GTY: Saving primary data. [2025-09-16T18:43:38.410634] [DEBUG] [LOCAL ] GTY: Trimmed 32954 records to 21887 (keeping 150 days of data) [2025-09-16T18:43:38.549288] [DEBUG] [LOCAL ] GTY: Primary data saved (21887 records). [2025-09-16T18:43:38.549419] [INFO ] [MAIN ] GTY: Full history saved. [2025-09-16T18:43:38.549502] [INFO ] [MAIN ] Update Progress: 979/2369 symbols completed. GTY took 20.1s. Estimated remaining: 2h 27m 5s. Rate: 9.5 symbols/min [2025-09-16T18:43:38.721140] [INFO ] [MAIN ] GVA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:43:38.747683] [DEBUG] [LOCAL ] GVA: Loaded 23856 existing records from primary storage. [2025-09-16T18:43:38.747830] [INFO ] [MAIN ] GVA: Loaded 23856 existing records. [2025-09-16T18:43:38.784041] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-02. [2025-09-16T18:43:38.944859] [WARN ] [MAIN ] GVA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:43:38.945102] [INFO ] [MAIN ] GVA: Wiping existing primary data before full refresh. [2025-09-16T18:43:38.945151] [INFO ] [LOCAL ] GVA: Starting deletion of primary data. [2025-09-16T18:43:38.945795] [DEBUG] [LOCAL ] GVA: Deleted primary data file. [2025-09-16T18:43:38.945844] [INFO ] [MAIN ] GVA: Starting full primary refresh. [2025-09-16T18:43:39.302253] [INFO ] [REMOTE ] GVA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:43:39.574910] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-02. [2025-09-16T18:43:39.766949] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T18:43:40.005661] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T18:43:40.374888] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-09. [2025-09-16T18:43:41.093344] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T18:43:41.174847] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-03. [2025-09-16T18:43:41.951000] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-09-16T18:43:41.956429] [INFO ] [REMOTE ] GTLS: Full fetch complete (46467 records). [2025-09-16T18:43:41.956523] [DEBUG] [VALIDATOR ] GTLS: Validating primary data. [2025-09-16T18:43:41.979989] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-04. [2025-09-16T18:43:42.342084] [DEBUG] [VALIDATOR ] GTLS: All expected trading days are present. [2025-09-16T18:43:42.342180] [DEBUG] [VALIDATOR ] GTLS: Primary data validation passed. [2025-09-16T18:43:42.342889] [INFO ] [MAIN ] GTLS: Fetched primary data is valid. [2025-09-16T18:43:42.342937] [DEBUG] [LOCAL ] GTLS: Saving primary data. [2025-09-16T18:43:42.344294] [DEBUG] [LOCAL ] GTLS: Trimmed 46467 records to 32299 (keeping 150 days of data) [2025-09-16T18:43:42.483612] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T18:43:42.577485] [DEBUG] [LOCAL ] GTLS: Primary data saved (32299 records). [2025-09-16T18:43:42.577725] [INFO ] [MAIN ] GTLS: Full history saved. [2025-09-16T18:43:42.579050] [INFO ] [MAIN ] Update Progress: 980/2369 symbols completed. GTLS took 32.3s. Estimated remaining: 2h 26m 55s. Rate: 9.5 symbols/min [2025-09-16T18:43:42.751401] [INFO ] [MAIN ] GWH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:43:42.766907] [DEBUG] [LOCAL ] GWH: Loaded 12801 existing records from primary storage. [2025-09-16T18:43:42.767075] [INFO ] [MAIN ] GWH: Loaded 12801 existing records. [2025-09-16T18:43:42.780156] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-04. [2025-09-16T18:43:42.950692] [WARN ] [MAIN ] GWH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:43:42.951022] [INFO ] [MAIN ] GWH: Wiping existing primary data before full refresh. [2025-09-16T18:43:42.951068] [INFO ] [LOCAL ] GWH: Starting deletion of primary data. [2025-09-16T18:43:42.951597] [DEBUG] [LOCAL ] GWH: Deleted primary data file. [2025-09-16T18:43:42.951676] [INFO ] [MAIN ] GWH: Starting full primary refresh. [2025-09-16T18:43:43.081286] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-09-16T18:43:43.330712] [INFO ] [REMOTE ] GWH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:43:43.574936] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-02. [2025-09-16T18:43:44.159622] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-09-16T18:43:44.374910] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-05. [2025-09-16T18:43:44.519475] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-09-16T18:43:44.960251] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-09-16T18:43:45.174905] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-06. [2025-09-16T18:43:45.974906] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-03. [2025-09-16T18:43:46.046080] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-09-16T18:43:46.774883] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-05. [2025-09-16T18:43:46.933454] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-09-16T18:43:47.574946] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-03. [2025-09-16T18:43:47.923214] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T18:43:48.374935] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-06. [2025-09-16T18:43:48.423944] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-09-16T18:43:49.174937] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-04. [2025-09-16T18:43:49.741255] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T18:43:49.913808] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-09-16T18:43:49.974896] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-05. [2025-09-16T18:43:50.774869] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-07. [2025-09-16T18:43:50.988625] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T18:43:51.434486] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-09-16T18:43:51.574894] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-04. [2025-09-16T18:43:52.374873] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-06. [2025-09-16T18:43:52.695669] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T18:43:53.174874] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-08. [2025-09-16T18:43:53.457695] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T18:43:53.974937] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-07. [2025-09-16T18:43:54.079846] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-09-16T18:43:54.774916] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-05. [2025-09-16T18:43:55.574936] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-09. [2025-09-16T18:43:55.623361] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-09-16T18:43:55.654595] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-09-16T18:43:56.374898] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-08. [2025-09-16T18:43:56.506305] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T18:43:56.508224] [INFO ] [REMOTE ] GUG: Full fetch complete (13527 records). [2025-09-16T18:43:56.508298] [DEBUG] [VALIDATOR ] GUG: Validating primary data. [2025-09-16T18:43:56.871505] [DEBUG] [VALIDATOR ] GUG: All expected trading days are present. [2025-09-16T18:43:56.871727] [DEBUG] [VALIDATOR ] GUG: Primary data validation passed. [2025-09-16T18:43:56.872037] [INFO ] [MAIN ] GUG: Fetched primary data is valid. [2025-09-16T18:43:56.872077] [DEBUG] [LOCAL ] GUG: Saving primary data. [2025-09-16T18:43:56.872564] [DEBUG] [LOCAL ] GUG: Trimmed 13527 records to 8867 (keeping 150 days of data) [2025-09-16T18:43:56.928586] [DEBUG] [LOCAL ] GUG: Primary data saved (8867 records). [2025-09-16T18:43:56.928767] [INFO ] [MAIN ] GUG: Full history saved. [2025-09-16T18:43:56.928861] [INFO ] [MAIN ] Update Progress: 981/2369 symbols completed. GUG took 24.0s. Estimated remaining: 2h 27m 0s. Rate: 9.4 symbols/min [2025-09-16T18:43:57.106477] [INFO ] [MAIN ] GWRE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:43:57.136369] [DEBUG] [LOCAL ] GWRE: Loaded 29708 existing records from primary storage. [2025-09-16T18:43:57.136527] [INFO ] [MAIN ] GWRE: Loaded 29708 existing records. [2025-09-16T18:43:57.180059] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-06. [2025-09-16T18:43:57.324595] [WARN ] [MAIN ] GWRE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:43:57.324901] [INFO ] [MAIN ] GWRE: Wiping existing primary data before full refresh. [2025-09-16T18:43:57.324949] [INFO ] [LOCAL ] GWRE: Starting deletion of primary data. [2025-09-16T18:43:57.325761] [DEBUG] [LOCAL ] GWRE: Deleted primary data file. [2025-09-16T18:43:57.325816] [INFO ] [MAIN ] GWRE: Starting full primary refresh. [2025-09-16T18:43:57.705920] [INFO ] [REMOTE ] GWRE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:43:57.827337] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T18:43:57.974898] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-09. [2025-09-16T18:43:58.298214] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T18:43:58.774925] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-02. [2025-09-16T18:43:59.296156] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T18:43:59.299938] [INFO ] [REMOTE ] GVA: Full fetch complete (37257 records). [2025-09-16T18:43:59.300017] [DEBUG] [VALIDATOR ] GVA: Validating primary data. [2025-09-16T18:43:59.581744] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-07. [2025-09-16T18:43:59.670331] [DEBUG] [VALIDATOR ] GVA: All expected trading days are present. [2025-09-16T18:43:59.670414] [DEBUG] [VALIDATOR ] GVA: Primary data validation passed. [2025-09-16T18:43:59.671034] [INFO ] [MAIN ] GVA: Fetched primary data is valid. [2025-09-16T18:43:59.671078] [DEBUG] [LOCAL ] GVA: Saving primary data. [2025-09-16T18:43:59.671947] [DEBUG] [LOCAL ] GVA: Trimmed 37257 records to 24076 (keeping 150 days of data) [2025-09-16T18:43:59.814957] [DEBUG] [LOCAL ] GVA: Primary data saved (24076 records). [2025-09-16T18:43:59.815089] [INFO ] [MAIN ] GVA: Full history saved. [2025-09-16T18:43:59.815170] [INFO ] [MAIN ] Update Progress: 982/2369 symbols completed. GVA took 21.3s. Estimated remaining: 2h 26m 49s. Rate: 9.4 symbols/min [2025-09-16T18:43:59.998997] [INFO ] [MAIN ] GWW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:00.013904] [DEBUG] [LOCAL ] GWW: Loaded 13875 existing records from primary storage. [2025-09-16T18:44:00.014052] [INFO ] [MAIN ] GWW: Loaded 13875 existing records. [2025-09-16T18:44:00.188858] [WARN ] [MAIN ] GWW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:44:00.189104] [INFO ] [MAIN ] GWW: Wiping existing primary data before full refresh. [2025-09-16T18:44:00.189151] [INFO ] [LOCAL ] GWW: Starting deletion of primary data. [2025-09-16T18:44:00.189696] [DEBUG] [LOCAL ] GWW: Deleted primary data file. [2025-09-16T18:44:00.189754] [INFO ] [MAIN ] GWW: Starting full primary refresh. [2025-09-16T18:44:00.379984] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-07. [2025-09-16T18:44:00.554726] [INFO ] [REMOTE ] GWW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:00.578820] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-09-16T18:44:00.751559] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T18:44:01.174910] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-08. [2025-09-16T18:44:01.815695] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T18:44:01.974938] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-08. [2025-09-16T18:44:02.160033] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T18:44:02.774902] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-03. [2025-09-16T18:44:02.898457] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-09-16T18:44:03.574899] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-09. [2025-09-16T18:44:04.374908] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-02. [2025-09-16T18:44:04.447282] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T18:44:04.628398] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-09-16T18:44:04.630937] [INFO ] [REMOTE ] GUT: Full fetch complete (16986 records). [2025-09-16T18:44:04.631015] [DEBUG] [VALIDATOR ] GUT: Validating primary data. [2025-09-16T18:44:05.018797] [DEBUG] [VALIDATOR ] GUT: All expected trading days are present. [2025-09-16T18:44:05.018967] [DEBUG] [VALIDATOR ] GUT: Primary data validation passed. [2025-09-16T18:44:05.019631] [INFO ] [MAIN ] GUT: Fetched primary data is valid. [2025-09-16T18:44:05.019716] [DEBUG] [LOCAL ] GUT: Saving primary data. [2025-09-16T18:44:05.021233] [DEBUG] [LOCAL ] GUT: Trimmed 16986 records to 11583 (keeping 150 days of data) [2025-09-16T18:44:05.096255] [DEBUG] [LOCAL ] GUT: Primary data saved (11583 records). [2025-09-16T18:44:05.096471] [INFO ] [MAIN ] GUT: Full history saved. [2025-09-16T18:44:05.096625] [INFO ] [MAIN ] Update Progress: 983/2369 symbols completed. GUT took 29.2s. Estimated remaining: 2h 26m 41s. Rate: 9.4 symbols/min [2025-09-16T18:44:05.180045] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-04. [2025-09-16T18:44:05.280354] [INFO ] [MAIN ] GXO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:05.317747] [DEBUG] [LOCAL ] GXO: Loaded 34160 existing records from primary storage. [2025-09-16T18:44:05.317966] [INFO ] [MAIN ] GXO: Loaded 34160 existing records. [2025-09-16T18:44:05.507910] [WARN ] [MAIN ] GXO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:44:05.508242] [INFO ] [MAIN ] GXO: Wiping existing primary data before full refresh. [2025-09-16T18:44:05.508294] [INFO ] [LOCAL ] GXO: Starting deletion of primary data. [2025-09-16T18:44:05.509259] [DEBUG] [LOCAL ] GXO: Deleted primary data file. [2025-09-16T18:44:05.509316] [INFO ] [MAIN ] GXO: Starting full primary refresh. [2025-09-16T18:44:05.847386] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T18:44:05.920562] [INFO ] [REMOTE ] GXO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:05.974918] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-03. [2025-09-16T18:44:06.724286] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-09-16T18:44:06.774867] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-05. [2025-09-16T18:44:07.206708] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-09-16T18:44:07.575002] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-04. [2025-09-16T18:44:08.311880] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T18:44:08.374948] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-09. [2025-09-16T18:44:08.680371] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-09-16T18:44:09.174914] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-06. [2025-09-16T18:44:09.318771] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-09-16T18:44:09.321075] [INFO ] [REMOTE ] GWH: Full fetch complete (16285 records). [2025-09-16T18:44:09.321177] [DEBUG] [VALIDATOR ] GWH: Validating primary data. [2025-09-16T18:44:09.694033] [DEBUG] [VALIDATOR ] GWH: All expected trading days are present. [2025-09-16T18:44:09.694129] [DEBUG] [VALIDATOR ] GWH: Primary data validation passed. [2025-09-16T18:44:09.694553] [INFO ] [MAIN ] GWH: Fetched primary data is valid. [2025-09-16T18:44:09.694611] [DEBUG] [LOCAL ] GWH: Saving primary data. [2025-09-16T18:44:09.695492] [DEBUG] [LOCAL ] GWH: Trimmed 16285 records to 12976 (keeping 150 days of data) [2025-09-16T18:44:09.770888] [DEBUG] [LOCAL ] GWH: Primary data saved (12976 records). [2025-09-16T18:44:09.771034] [INFO ] [MAIN ] GWH: Full history saved. [2025-09-16T18:44:09.771118] [INFO ] [MAIN ] Update Progress: 984/2369 symbols completed. GWH took 27.2s. Estimated remaining: 2h 26m 32s. Rate: 9.5 symbols/min [2025-09-16T18:44:09.948697] [INFO ] [MAIN ] GYLD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:09.951694] [DEBUG] [LOCAL ] GYLD: Loaded 1104 existing records from primary storage. [2025-09-16T18:44:09.951764] [INFO ] [MAIN ] GYLD: Loaded 1104 existing records. [2025-09-16T18:44:09.982352] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-02. [2025-09-16T18:44:10.136942] [WARN ] [MAIN ] GYLD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:44:10.137224] [INFO ] [MAIN ] GYLD: Wiping existing primary data before full refresh. [2025-09-16T18:44:10.137273] [INFO ] [LOCAL ] GYLD: Starting deletion of primary data. [2025-09-16T18:44:10.137466] [DEBUG] [LOCAL ] GYLD: Deleted primary data file. [2025-09-16T18:44:10.137510] [INFO ] [MAIN ] GYLD: Starting full primary refresh. [2025-09-16T18:44:10.508931] [INFO ] [REMOTE ] GYLD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:10.563050] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T18:44:10.774959] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-02. [2025-09-16T18:44:11.238193] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-09-16T18:44:11.574908] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-03. [2025-09-16T18:44:11.773826] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-09-16T18:44:12.040288] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-09-16T18:44:12.374926] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-04. [2025-09-16T18:44:12.857284] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-09-16T18:44:13.174912] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-05. [2025-09-16T18:44:13.535627] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-09-16T18:44:13.974941] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-07. [2025-09-16T18:44:14.774897] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-03. [2025-09-16T18:44:15.305591] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-09-16T18:44:15.574911] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-08. [2025-09-16T18:44:16.374937] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-05. [2025-09-16T18:44:16.751208] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-09-16T18:44:17.062901] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T18:44:17.174915] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-06. [2025-09-16T18:44:17.193755] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T18:44:17.610214] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-09-16T18:44:17.974904] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-06. [2025-09-16T18:44:18.774933] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-09. [2025-09-16T18:44:18.942238] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-09-16T18:44:19.574971] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-07. [2025-09-16T18:44:20.070783] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-09-16T18:44:20.347813] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T18:44:20.356007] [INFO ] [REMOTE ] GWRE: Full fetch complete (45395 records). [2025-09-16T18:44:20.356240] [DEBUG] [VALIDATOR ] GWRE: Validating primary data. [2025-09-16T18:44:20.380128] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-04. [2025-09-16T18:44:20.772612] [DEBUG] [VALIDATOR ] GWRE: All expected trading days are present. [2025-09-16T18:44:20.772740] [DEBUG] [VALIDATOR ] GWRE: Primary data validation passed. [2025-09-16T18:44:20.773553] [INFO ] [MAIN ] GWRE: Fetched primary data is valid. [2025-09-16T18:44:20.773612] [DEBUG] [LOCAL ] GWRE: Saving primary data. [2025-09-16T18:44:20.775003] [DEBUG] [LOCAL ] GWRE: Trimmed 45395 records to 30089 (keeping 150 days of data) [2025-09-16T18:44:20.960959] [DEBUG] [LOCAL ] GWRE: Primary data saved (30089 records). [2025-09-16T18:44:20.961116] [INFO ] [MAIN ] GWRE: Full history saved. [2025-09-16T18:44:20.961227] [INFO ] [MAIN ] Update Progress: 985/2369 symbols completed. GWRE took 24.0s. Estimated remaining: 2h 26m 33s. Rate: 9.4 symbols/min [2025-09-16T18:44:21.145847] [INFO ] [MAIN ] H: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:21.179371] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-07. [2025-09-16T18:44:21.180045] [DEBUG] [LOCAL ] H: Loaded 32936 existing records from primary storage. [2025-09-16T18:44:21.180952] [INFO ] [MAIN ] H: Loaded 32936 existing records. [2025-09-16T18:44:21.372586] [WARN ] [MAIN ] H: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:44:21.372890] [INFO ] [MAIN ] H: Wiping existing primary data before full refresh. [2025-09-16T18:44:21.372950] [INFO ] [LOCAL ] H: Starting deletion of primary data. [2025-09-16T18:44:21.373830] [DEBUG] [LOCAL ] H: Deleted primary data file. [2025-09-16T18:44:21.373897] [INFO ] [MAIN ] H: Starting full primary refresh. [2025-09-16T18:44:21.783523] [INFO ] [REMOTE ] H: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:21.974937] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-08. [2025-09-16T18:44:22.149672] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T18:44:22.350212] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-09-16T18:44:22.562305] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-09-16T18:44:22.774902] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-09. [2025-09-16T18:44:23.246011] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-09-16T18:44:23.246905] [INFO ] [REMOTE ] GYLD: Full fetch complete (1630 records). [2025-09-16T18:44:23.246985] [DEBUG] [VALIDATOR ] GYLD: Validating primary data. [2025-09-16T18:44:23.584506] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-08. [2025-09-16T18:44:23.616512] [DEBUG] [VALIDATOR ] GYLD: All expected trading days are present. [2025-09-16T18:44:23.616609] [DEBUG] [VALIDATOR ] GYLD: Primary data validation passed. [2025-09-16T18:44:23.616739] [INFO ] [MAIN ] GYLD: Fetched primary data is valid. [2025-09-16T18:44:23.616791] [DEBUG] [LOCAL ] GYLD: Saving primary data. [2025-09-16T18:44:23.617157] [DEBUG] [LOCAL ] GYLD: Trimmed 1630 records to 1117 (keeping 150 days of data) [2025-09-16T18:44:23.625006] [DEBUG] [LOCAL ] GYLD: Primary data saved (1117 records). [2025-09-16T18:44:23.625098] [INFO ] [MAIN ] GYLD: Full history saved. [2025-09-16T18:44:23.625199] [INFO ] [MAIN ] Update Progress: 986/2369 symbols completed. GYLD took 13.9s. Estimated remaining: 2h 26m 21s. Rate: 9.4 symbols/min [2025-09-16T18:44:23.806519] [INFO ] [MAIN ] HAE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:23.834068] [DEBUG] [LOCAL ] HAE: Loaded 27848 existing records from primary storage. [2025-09-16T18:44:23.834218] [INFO ] [MAIN ] HAE: Loaded 27848 existing records. [2025-09-16T18:44:24.029009] [WARN ] [MAIN ] HAE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:44:24.029270] [INFO ] [MAIN ] HAE: Wiping existing primary data before full refresh. [2025-09-16T18:44:24.029315] [INFO ] [LOCAL ] HAE: Starting deletion of primary data. [2025-09-16T18:44:24.030003] [DEBUG] [LOCAL ] HAE: Deleted primary data file. [2025-09-16T18:44:24.030052] [INFO ] [MAIN ] HAE: Starting full primary refresh. [2025-09-16T18:44:24.380117] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-05. [2025-09-16T18:44:24.410940] [INFO ] [REMOTE ] HAE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:24.819968] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T18:44:25.174982] [INFO ] [REMOTE ] H: Fetching primary data for 2025-02. [2025-09-16T18:44:25.974910] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-09. [2025-09-16T18:44:26.035759] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T18:44:26.675243] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-09-16T18:44:26.774869] [INFO ] [REMOTE ] H: Fetching primary data for 2025-03. [2025-09-16T18:44:27.102490] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T18:44:27.105508] [INFO ] [REMOTE ] GWW: Full fetch complete (21835 records). [2025-09-16T18:44:27.105605] [DEBUG] [VALIDATOR ] GWW: Validating primary data. [2025-09-16T18:44:27.490829] [DEBUG] [VALIDATOR ] GWW: All expected trading days are present. [2025-09-16T18:44:27.490913] [DEBUG] [VALIDATOR ] GWW: Primary data validation passed. [2025-09-16T18:44:27.491379] [INFO ] [MAIN ] GWW: Fetched primary data is valid. [2025-09-16T18:44:27.491421] [DEBUG] [LOCAL ] GWW: Saving primary data. [2025-09-16T18:44:27.492214] [DEBUG] [LOCAL ] GWW: Trimmed 21835 records to 14090 (keeping 150 days of data) [2025-09-16T18:44:27.575386] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-06. [2025-09-16T18:44:27.589340] [DEBUG] [LOCAL ] GWW: Primary data saved (14090 records). [2025-09-16T18:44:27.589442] [INFO ] [MAIN ] GWW: Full history saved. [2025-09-16T18:44:27.589535] [INFO ] [MAIN ] Update Progress: 987/2369 symbols completed. GWW took 27.8s. Estimated remaining: 2h 26m 12s. Rate: 9.5 symbols/min [2025-09-16T18:44:27.761795] [INFO ] [MAIN ] HAFN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:27.793942] [DEBUG] [LOCAL ] HAFN: Loaded 31519 existing records from primary storage. [2025-09-16T18:44:27.794097] [INFO ] [MAIN ] HAFN: Loaded 31519 existing records. [2025-09-16T18:44:27.974418] [WARN ] [MAIN ] HAFN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:44:27.974694] [INFO ] [MAIN ] HAFN: Wiping existing primary data before full refresh. [2025-09-16T18:44:27.974742] [INFO ] [LOCAL ] HAFN: Starting deletion of primary data. [2025-09-16T18:44:27.975558] [DEBUG] [LOCAL ] HAFN: Deleted primary data file. [2025-09-16T18:44:27.975606] [INFO ] [MAIN ] HAFN: Starting full primary refresh. [2025-09-16T18:44:28.339225] [INFO ] [REMOTE ] HAFN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:28.374880] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-02. [2025-09-16T18:44:28.444839] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T18:44:29.173570] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-09-16T18:44:29.177040] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-02. [2025-09-16T18:44:29.974901] [INFO ] [REMOTE ] H: Fetching primary data for 2025-04. [2025-09-16T18:44:30.236785] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-09-16T18:44:30.774889] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-03. [2025-09-16T18:44:30.895436] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-09-16T18:44:31.574922] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-03. [2025-09-16T18:44:31.605724] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-09-16T18:44:31.940214] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T18:44:32.374907] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-04. [2025-09-16T18:44:33.070899] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-09-16T18:44:33.174884] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-04. [2025-09-16T18:44:33.606037] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T18:44:33.974915] [INFO ] [REMOTE ] H: Fetching primary data for 2025-05. [2025-09-16T18:44:34.678338] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T18:44:34.774892] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-05. [2025-09-16T18:44:35.516953] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T18:44:35.574860] [INFO ] [REMOTE ] H: Fetching primary data for 2025-06. [2025-09-16T18:44:36.380222] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-05. [2025-09-16T18:44:36.386188] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T18:44:36.851836] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T18:44:37.174902] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-07. [2025-09-16T18:44:37.803604] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-09-16T18:44:37.974902] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-06. [2025-09-16T18:44:38.332975] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-09-16T18:44:38.774920] [INFO ] [REMOTE ] H: Fetching primary data for 2025-07. [2025-09-16T18:44:39.103901] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-09-16T18:44:39.574910] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-08. [2025-09-16T18:44:40.318992] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-09-16T18:44:40.374898] [INFO ] [REMOTE ] H: Fetching primary data for 2025-08. [2025-09-16T18:44:40.761702] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-09-16T18:44:41.174900] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-07. [2025-09-16T18:44:41.426553] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T18:44:41.974898] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-09. [2025-09-16T18:44:42.635840] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T18:44:42.774899] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-06. [2025-09-16T18:44:43.531846] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T18:44:43.537348] [INFO ] [REMOTE ] GXO: Full fetch complete (52490 records). [2025-09-16T18:44:43.537495] [DEBUG] [VALIDATOR ] GXO: Validating primary data. [2025-09-16T18:44:43.580031] [INFO ] [REMOTE ] H: Fetching primary data for 2025-09. [2025-09-16T18:44:43.923460] [DEBUG] [VALIDATOR ] GXO: All expected trading days are present. [2025-09-16T18:44:43.923539] [DEBUG] [VALIDATOR ] GXO: Primary data validation passed. [2025-09-16T18:44:43.924300] [INFO ] [MAIN ] GXO: Fetched primary data is valid. [2025-09-16T18:44:43.924355] [DEBUG] [LOCAL ] GXO: Saving primary data. [2025-09-16T18:44:43.926401] [DEBUG] [LOCAL ] GXO: Trimmed 52490 records to 34488 (keeping 150 days of data) [2025-09-16T18:44:44.128918] [DEBUG] [LOCAL ] GXO: Primary data saved (34488 records). [2025-09-16T18:44:44.129057] [INFO ] [MAIN ] GXO: Full history saved. [2025-09-16T18:44:44.129159] [INFO ] [MAIN ] Update Progress: 988/2369 symbols completed. GXO took 39.0s. Estimated remaining: 2h 26m 19s. Rate: 9.4 symbols/min [2025-09-16T18:44:44.305702] [INFO ] [MAIN ] HAL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:44.346922] [DEBUG] [LOCAL ] HAL: Loaded 39678 existing records from primary storage. [2025-09-16T18:44:44.347088] [INFO ] [MAIN ] HAL: Loaded 39678 existing records. [2025-09-16T18:44:44.379941] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-08. [2025-09-16T18:44:44.528724] [WARN ] [MAIN ] HAL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:44:44.529088] [INFO ] [MAIN ] HAL: Wiping existing primary data before full refresh. [2025-09-16T18:44:44.529336] [INFO ] [LOCAL ] HAL: Starting deletion of primary data. [2025-09-16T18:44:44.538849] [DEBUG] [LOCAL ] HAL: Deleted primary data file. [2025-09-16T18:44:44.552470] [INFO ] [MAIN ] HAL: Starting full primary refresh. [2025-09-16T18:44:44.602858] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T18:44:44.919407] [INFO ] [REMOTE ] HAL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:45.174889] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-02. [2025-09-16T18:44:45.301924] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-09-16T18:44:45.307513] [INFO ] [REMOTE ] H: Full fetch complete (49142 records). [2025-09-16T18:44:45.307621] [DEBUG] [VALIDATOR ] H: Validating primary data. [2025-09-16T18:44:45.693260] [DEBUG] [VALIDATOR ] H: All expected trading days are present. [2025-09-16T18:44:45.693369] [DEBUG] [VALIDATOR ] H: Primary data validation passed. [2025-09-16T18:44:45.694203] [INFO ] [MAIN ] H: Fetched primary data is valid. [2025-09-16T18:44:45.694272] [DEBUG] [LOCAL ] H: Saving primary data. [2025-09-16T18:44:45.696348] [DEBUG] [LOCAL ] H: Trimmed 49142 records to 33240 (keeping 150 days of data) [2025-09-16T18:44:45.917472] [DEBUG] [LOCAL ] H: Primary data saved (33240 records). [2025-09-16T18:44:45.917734] [INFO ] [MAIN ] H: Full history saved. [2025-09-16T18:44:45.917929] [INFO ] [MAIN ] Update Progress: 989/2369 symbols completed. H took 25.0s. Estimated remaining: 2h 26m 7s. Rate: 9.4 symbols/min [2025-09-16T18:44:45.975008] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-07. [2025-09-16T18:44:46.052136] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:44:46.132174] [INFO ] [MAIN ] HASI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:46.171390] [DEBUG] [LOCAL ] HASI: Loaded 35139 existing records from primary storage. [2025-09-16T18:44:46.171547] [INFO ] [MAIN ] HASI: Loaded 35139 existing records. [2025-09-16T18:44:46.356749] [WARN ] [MAIN ] HASI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:44:46.357040] [INFO ] [MAIN ] HASI: Wiping existing primary data before full refresh. [2025-09-16T18:44:46.357088] [INFO ] [LOCAL ] HASI: Starting deletion of primary data. [2025-09-16T18:44:46.357869] [DEBUG] [LOCAL ] HASI: Deleted primary data file. [2025-09-16T18:44:46.357928] [INFO ] [MAIN ] HASI: Starting full primary refresh. [2025-09-16T18:44:46.723276] [INFO ] [REMOTE ] HASI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:46.774899] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-09. [2025-09-16T18:44:46.998926] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-09-16T18:44:47.397415] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-09-16T18:44:47.574910] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-08. [2025-09-16T18:44:48.374958] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-02. [2025-09-16T18:44:48.413595] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T18:44:48.419273] [INFO ] [REMOTE ] HAE: Full fetch complete (43903 records). [2025-09-16T18:44:48.419428] [DEBUG] [VALIDATOR ] HAE: Validating primary data. [2025-09-16T18:44:48.799087] [DEBUG] [VALIDATOR ] HAE: All expected trading days are present. [2025-09-16T18:44:48.799184] [DEBUG] [VALIDATOR ] HAE: Primary data validation passed. [2025-09-16T18:44:48.799850] [INFO ] [MAIN ] HAE: Fetched primary data is valid. [2025-09-16T18:44:48.799894] [DEBUG] [LOCAL ] HAE: Saving primary data. [2025-09-16T18:44:48.800941] [DEBUG] [LOCAL ] HAE: Trimmed 43903 records to 28192 (keeping 150 days of data) [2025-09-16T18:44:48.968362] [DEBUG] [LOCAL ] HAE: Primary data saved (28192 records). [2025-09-16T18:44:48.968501] [INFO ] [MAIN ] HAE: Full history saved. [2025-09-16T18:44:48.968602] [INFO ] [MAIN ] Update Progress: 990/2369 symbols completed. HAE took 25.3s. Estimated remaining: 2h 25m 56s. Rate: 9.4 symbols/min [2025-09-16T18:44:49.139175] [INFO ] [MAIN ] HAYW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:49.168057] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T18:44:49.176752] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-03. [2025-09-16T18:44:49.207803] [DEBUG] [LOCAL ] HAYW: Loaded 35753 existing records from primary storage. [2025-09-16T18:44:49.207952] [INFO ] [MAIN ] HAYW: Loaded 35753 existing records. [2025-09-16T18:44:49.385991] [WARN ] [MAIN ] HAYW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:44:49.386229] [INFO ] [MAIN ] HAYW: Wiping existing primary data before full refresh. [2025-09-16T18:44:49.386286] [INFO ] [LOCAL ] HAYW: Starting deletion of primary data. [2025-09-16T18:44:49.387078] [DEBUG] [LOCAL ] HAYW: Deleted primary data file. [2025-09-16T18:44:49.387135] [INFO ] [MAIN ] HAYW: Starting full primary refresh. [2025-09-16T18:44:49.736936] [INFO ] [REMOTE ] HAYW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:49.959873] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T18:44:49.974846] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-03. [2025-09-16T18:44:50.777210] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-09. [2025-09-16T18:44:50.783324] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T18:44:51.398555] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T18:44:51.574885] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-04. [2025-09-16T18:44:52.281664] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:44:52.286713] [INFO ] [REMOTE ] HAFN: Full fetch complete (48287 records). [2025-09-16T18:44:52.286814] [DEBUG] [VALIDATOR ] HAFN: Validating primary data. [2025-09-16T18:44:52.381608] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-02. [2025-09-16T18:44:52.669397] [DEBUG] [VALIDATOR ] HAFN: All expected trading days are present. [2025-09-16T18:44:52.669481] [DEBUG] [VALIDATOR ] HAFN: Primary data validation passed. [2025-09-16T18:44:52.670175] [INFO ] [MAIN ] HAFN: Fetched primary data is valid. [2025-09-16T18:44:52.670219] [DEBUG] [LOCAL ] HAFN: Saving primary data. [2025-09-16T18:44:52.671321] [DEBUG] [LOCAL ] HAFN: Trimmed 48287 records to 31774 (keeping 150 days of data) [2025-09-16T18:44:52.861630] [DEBUG] [LOCAL ] HAFN: Primary data saved (31774 records). [2025-09-16T18:44:52.861779] [INFO ] [MAIN ] HAFN: Full history saved. [2025-09-16T18:44:52.861871] [INFO ] [MAIN ] Update Progress: 991/2369 symbols completed. HAFN took 25.3s. Estimated remaining: 2h 25m 46s. Rate: 9.5 symbols/min [2025-09-16T18:44:53.030669] [INFO ] [MAIN ] HBB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:44:53.030840] [DEBUG] [LOCAL ] HBB: No existing data file found in primary storage. [2025-09-16T18:44:53.030879] [WARN ] [MAIN ] HBB: No existing data found, performing full refresh instead. [2025-09-16T18:44:53.030994] [INFO ] [MAIN ] HBB: Wiping existing primary data before full refresh. [2025-09-16T18:44:53.031033] [INFO ] [LOCAL ] HBB: Starting deletion of primary data. [2025-09-16T18:44:53.031074] [DEBUG] [LOCAL ] HBB: No primary data file found to delete. [2025-09-16T18:44:53.031107] [INFO ] [MAIN ] HBB: Starting full primary refresh. [2025-09-16T18:44:53.179955] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-04. [2025-09-16T18:44:53.383114] [INFO ] [REMOTE ] HBB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:44:53.591341] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-09-16T18:44:53.974906] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-05. [2025-09-16T18:44:54.240584] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-09-16T18:44:54.774914] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-02. [2025-09-16T18:44:54.940789] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T18:44:55.251051] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-09-16T18:44:55.574879] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-03. [2025-09-16T18:44:55.822322] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-09-16T18:44:56.271947] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-09-16T18:44:56.374889] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-03. [2025-09-16T18:44:57.174919] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-06. [2025-09-16T18:44:57.974880] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-05. [2025-09-16T18:44:58.032784] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-09-16T18:44:58.774913] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-04. [2025-09-16T18:44:59.318608] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-06 (2.14 seconds). [2025-09-16T18:44:59.368185] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T18:44:59.574907] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-07. [2025-09-16T18:45:00.240921] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T18:45:00.374994] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-04. [2025-09-16T18:45:01.174964] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-06. [2025-09-16T18:45:01.401023] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T18:45:01.651037] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-09-16T18:45:01.974988] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-05. [2025-09-16T18:45:02.774985] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-05. [2025-09-16T18:45:02.792358] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T18:45:02.974193] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-09-16T18:45:03.574901] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-07. [2025-09-16T18:45:04.374956] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-08. [2025-09-16T18:45:04.637031] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-09-16T18:45:05.174924] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-06. [2025-09-16T18:45:05.386261] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T18:45:05.948229] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T18:45:05.974870] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-09. [2025-09-16T18:45:06.779987] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-08. [2025-09-16T18:45:06.796165] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T18:45:07.574909] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-07. [2025-09-16T18:45:07.854017] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-09-16T18:45:07.860463] [INFO ] [REMOTE ] HAL: Full fetch complete (58398 records). [2025-09-16T18:45:07.860588] [DEBUG] [VALIDATOR ] HAL: Validating primary data. [2025-09-16T18:45:08.273104] [DEBUG] [VALIDATOR ] HAL: All expected trading days are present. [2025-09-16T18:45:08.273201] [DEBUG] [VALIDATOR ] HAL: Primary data validation passed. [2025-09-16T18:45:08.274023] [INFO ] [MAIN ] HAL: Fetched primary data is valid. [2025-09-16T18:45:08.274181] [DEBUG] [LOCAL ] HAL: Saving primary data. [2025-09-16T18:45:08.275753] [DEBUG] [LOCAL ] HAL: Trimmed 58398 records to 40068 (keeping 150 days of data) [2025-09-16T18:45:08.374894] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-06. [2025-09-16T18:45:08.408558] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-09-16T18:45:08.592754] [DEBUG] [LOCAL ] HAL: Primary data saved (40068 records). [2025-09-16T18:45:08.592953] [INFO ] [MAIN ] HAL: Full history saved. [2025-09-16T18:45:08.593105] [INFO ] [MAIN ] Update Progress: 992/2369 symbols completed. HAL took 24.5s. Estimated remaining: 2h 25m 53s. Rate: 9.4 symbols/min [2025-09-16T18:45:08.786795] [INFO ] [MAIN ] HBI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:08.842778] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-09-16T18:45:08.868893] [DEBUG] [LOCAL ] HBI: Loaded 39082 existing records from primary storage. [2025-09-16T18:45:08.869076] [INFO ] [MAIN ] HBI: Loaded 39082 existing records. [2025-09-16T18:45:09.057519] [WARN ] [MAIN ] HBI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:45:09.057929] [INFO ] [MAIN ] HBI: Wiping existing primary data before full refresh. [2025-09-16T18:45:09.057987] [INFO ] [LOCAL ] HBI: Starting deletion of primary data. [2025-09-16T18:45:09.058937] [DEBUG] [LOCAL ] HBI: Deleted primary data file. [2025-09-16T18:45:09.059002] [INFO ] [MAIN ] HBI: Starting full primary refresh. [2025-09-16T18:45:09.163717] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-09-16T18:45:09.175078] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-08. [2025-09-16T18:45:09.447121] [INFO ] [REMOTE ] HBI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:09.974937] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-07. [2025-09-16T18:45:10.602811] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-09-16T18:45:10.636798] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-09-16T18:45:10.774928] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-08. [2025-09-16T18:45:11.490189] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-09-16T18:45:11.574920] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-09. [2025-09-16T18:45:12.182957] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-09-16T18:45:12.184341] [INFO ] [REMOTE ] HBB: Full fetch complete (7576 records). [2025-09-16T18:45:12.184446] [DEBUG] [VALIDATOR ] HBB: Validating primary data. [2025-09-16T18:45:12.383268] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-02. [2025-09-16T18:45:12.555167] [DEBUG] [VALIDATOR ] HBB: All expected trading days are present. [2025-09-16T18:45:12.555246] [DEBUG] [VALIDATOR ] HBB: Primary data validation passed. [2025-09-16T18:45:12.555436] [INFO ] [MAIN ] HBB: Fetched primary data is valid. [2025-09-16T18:45:12.555475] [DEBUG] [LOCAL ] HBB: Saving primary data. [2025-09-16T18:45:12.555899] [DEBUG] [LOCAL ] HBB: Trimmed 7576 records to 4754 (keeping 150 days of data) [2025-09-16T18:45:12.585564] [DEBUG] [LOCAL ] HBB: Primary data saved (4754 records). [2025-09-16T18:45:12.585678] [INFO ] [MAIN ] HBB: Full history saved. [2025-09-16T18:45:12.585753] [INFO ] [MAIN ] Update Progress: 993/2369 symbols completed. HBB took 19.7s. Estimated remaining: 2h 25m 43s. Rate: 9.4 symbols/min [2025-09-16T18:45:12.774627] [INFO ] [MAIN ] HBM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:12.817798] [DEBUG] [LOCAL ] HBM: Loaded 38996 existing records from primary storage. [2025-09-16T18:45:12.817954] [INFO ] [MAIN ] HBM: Loaded 38996 existing records. [2025-09-16T18:45:13.015020] [WARN ] [MAIN ] HBM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:45:13.015292] [INFO ] [MAIN ] HBM: Wiping existing primary data before full refresh. [2025-09-16T18:45:13.015339] [INFO ] [LOCAL ] HBM: Starting deletion of primary data. [2025-09-16T18:45:13.016342] [DEBUG] [LOCAL ] HBM: Deleted primary data file. [2025-09-16T18:45:13.016399] [INFO ] [MAIN ] HBM: Starting full primary refresh. [2025-09-16T18:45:13.176597] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-09. [2025-09-16T18:45:13.382239] [INFO ] [REMOTE ] HBM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:13.977747] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-09. [2025-09-16T18:45:14.619182] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-09-16T18:45:14.774992] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-03. [2025-09-16T18:45:14.988629] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-09-16T18:45:14.995599] [INFO ] [REMOTE ] HAYW: Full fetch complete (52687 records). [2025-09-16T18:45:14.995809] [DEBUG] [VALIDATOR ] HAYW: Validating primary data. [2025-09-16T18:45:14.996811] [FATAL] [VALIDATOR ] HAYW: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:45:14.996901] [FATAL] [MAIN ] HAYW: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:45:14.996958] [WARN ] [MAIN ] HAYW: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:45:14.997028] [INFO ] [LOCAL ] HAYW: Starting deletion of primary data. [2025-09-16T18:45:14.997113] [DEBUG] [LOCAL ] HAYW: No primary data file found to delete. [2025-09-16T18:45:14.997163] [INFO ] [LOCAL ] HAYW: Starting deletion of derivative data. [2025-09-16T18:45:14.997558] [DEBUG] [LOCAL ] HAYW: Deleted derivative data file. [2025-09-16T18:45:14.999203] [INFO ] [MAIN ] Update Progress: 994/2369 symbols completed. HAYW took 26.0s. Estimated remaining: 2h 25m 31s. Rate: 9.4 symbols/min [2025-09-16T18:45:15.185371] [INFO ] [MAIN ] HCA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:15.200941] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-09-16T18:45:15.206399] [INFO ] [REMOTE ] HASI: Full fetch complete (51041 records). [2025-09-16T18:45:15.206517] [DEBUG] [VALIDATOR ] HASI: Validating primary data. [2025-09-16T18:45:15.207424] [FATAL] [VALIDATOR ] HASI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:45:15.207519] [FATAL] [MAIN ] HASI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:45:15.207558] [WARN ] [MAIN ] HASI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:45:15.207606] [INFO ] [LOCAL ] HASI: Starting deletion of primary data. [2025-09-16T18:45:15.207714] [DEBUG] [LOCAL ] HASI: No primary data file found to delete. [2025-09-16T18:45:15.207756] [INFO ] [LOCAL ] HASI: Starting deletion of derivative data. [2025-09-16T18:45:15.208016] [DEBUG] [LOCAL ] HASI: Deleted derivative data file. [2025-09-16T18:45:15.208107] [INFO ] [MAIN ] Update Progress: 995/2369 symbols completed. HASI took 29.3s. Estimated remaining: 2h 25m 16s. Rate: 9.5 symbols/min [2025-09-16T18:45:15.399493] [INFO ] [MAIN ] HCC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:15.410817] [DEBUG] [LOCAL ] HCA: Loaded 34822 existing records from primary storage. [2025-09-16T18:45:15.410967] [INFO ] [MAIN ] HCA: Loaded 34822 existing records. [2025-09-16T18:45:15.583453] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-02. [2025-09-16T18:45:15.611985] [WARN ] [MAIN ] HCA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:45:15.613117] [INFO ] [MAIN ] HCA: Wiping existing primary data before full refresh. [2025-09-16T18:45:15.613177] [INFO ] [LOCAL ] HCA: Starting deletion of primary data. [2025-09-16T18:45:15.614125] [DEBUG] [LOCAL ] HCA: Deleted primary data file. [2025-09-16T18:45:15.614224] [INFO ] [MAIN ] HCA: Starting full primary refresh. [2025-09-16T18:45:15.732318] [DEBUG] [LOCAL ] HCC: Loaded 33376 existing records from primary storage. [2025-09-16T18:45:15.748057] [INFO ] [MAIN ] HCC: Loaded 33376 existing records. [2025-09-16T18:45:16.165908] [WARN ] [MAIN ] HCC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:45:16.197174] [INFO ] [MAIN ] HCC: Wiping existing primary data before full refresh. [2025-09-16T18:45:16.221290] [INFO ] [LOCAL ] HCC: Starting deletion of primary data. [2025-09-16T18:45:16.221103] [INFO ] [REMOTE ] HCA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:16.222161] [DEBUG] [LOCAL ] HCC: Deleted primary data file. [2025-09-16T18:45:16.222282] [INFO ] [MAIN ] HCC: Starting full primary refresh. [2025-09-16T18:45:16.377155] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-02. [2025-09-16T18:45:16.570664] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T18:45:16.706379] [INFO ] [REMOTE ] HCC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:17.174934] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-04. [2025-09-16T18:45:17.574487] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-09-16T18:45:17.974959] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-03. [2025-09-16T18:45:18.445179] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-09-16T18:45:18.558870] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-09-16T18:45:18.774943] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-03. [2025-09-16T18:45:19.386593] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-09-16T18:45:19.574919] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-04. [2025-09-16T18:45:20.374906] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-02. [2025-09-16T18:45:20.529477] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T18:45:21.174907] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-04. [2025-09-16T18:45:21.404322] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T18:45:21.974926] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-05. [2025-09-16T18:45:22.370148] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-09-16T18:45:22.774919] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-05. [2025-09-16T18:45:23.242249] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-09-16T18:45:23.574963] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-05. [2025-09-16T18:45:23.635212] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T18:45:24.374917] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-06. [2025-09-16T18:45:24.440314] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-09-16T18:45:25.174905] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-06. [2025-09-16T18:45:25.346122] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-09-16T18:45:25.974928] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-06. [2025-09-16T18:45:26.015060] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T18:45:26.774914] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-07. [2025-09-16T18:45:27.002488] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T18:45:27.567052] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T18:45:27.576569] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-03. [2025-09-16T18:45:28.374916] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-07. [2025-09-16T18:45:28.804887] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-09-16T18:45:29.174878] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-08. [2025-09-16T18:45:29.263032] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-09-16T18:45:29.779287] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-09-16T18:45:29.974871] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-08. [2025-09-16T18:45:30.774926] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-07. [2025-09-16T18:45:30.932568] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-09-16T18:45:31.575045] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-04. [2025-09-16T18:45:31.788323] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-09-16T18:45:32.374927] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-09. [2025-09-16T18:45:32.688399] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T18:45:33.174902] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-09. [2025-09-16T18:45:33.365126] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T18:45:33.974915] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-08. [2025-09-16T18:45:34.048889] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-09-16T18:45:34.055251] [INFO ] [REMOTE ] HBI: Full fetch complete (57723 records). [2025-09-16T18:45:34.055370] [DEBUG] [VALIDATOR ] HBI: Validating primary data. [2025-09-16T18:45:34.462672] [DEBUG] [VALIDATOR ] HBI: All expected trading days are present. [2025-09-16T18:45:34.462784] [DEBUG] [VALIDATOR ] HBI: Primary data validation passed. [2025-09-16T18:45:34.463557] [INFO ] [MAIN ] HBI: Fetched primary data is valid. [2025-09-16T18:45:34.463601] [DEBUG] [LOCAL ] HBI: Saving primary data. [2025-09-16T18:45:34.464831] [DEBUG] [LOCAL ] HBI: Trimmed 57723 records to 39471 (keeping 150 days of data) [2025-09-16T18:45:34.693148] [DEBUG] [LOCAL ] HBI: Primary data saved (39471 records). [2025-09-16T18:45:34.693310] [INFO ] [MAIN ] HBI: Full history saved. [2025-09-16T18:45:34.693430] [INFO ] [MAIN ] Update Progress: 996/2369 symbols completed. HBI took 26.1s. Estimated remaining: 2h 25m 28s. Rate: 9.4 symbols/min [2025-09-16T18:45:34.785805] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-05. [2025-09-16T18:45:34.868622] [INFO ] [MAIN ] HCI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:34.886197] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T18:45:34.894617] [INFO ] [REMOTE ] HBM: Full fetch complete (57559 records). [2025-09-16T18:45:34.894768] [DEBUG] [VALIDATOR ] HBM: Validating primary data. [2025-09-16T18:45:34.980579] [DEBUG] [LOCAL ] HCI: Loaded 10345 existing records from primary storage. [2025-09-16T18:45:35.004194] [INFO ] [MAIN ] HCI: Loaded 10345 existing records. [2025-09-16T18:45:35.347742] [WARN ] [MAIN ] HCI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:45:35.386875] [INFO ] [MAIN ] HCI: Wiping existing primary data before full refresh. [2025-09-16T18:45:35.399411] [INFO ] [LOCAL ] HCI: Starting deletion of primary data. [2025-09-16T18:45:35.405159] [DEBUG] [LOCAL ] HCI: Deleted primary data file. [2025-09-16T18:45:35.415418] [INFO ] [MAIN ] HCI: Starting full primary refresh. [2025-09-16T18:45:35.596733] [DEBUG] [VALIDATOR ] HBM: All expected trading days are present. [2025-09-16T18:45:35.617761] [DEBUG] [VALIDATOR ] HBM: Primary data validation passed. [2025-09-16T18:45:35.697537] [INFO ] [MAIN ] HBM: Fetched primary data is valid. [2025-09-16T18:45:35.732866] [DEBUG] [LOCAL ] HBM: Saving primary data. [2025-09-16T18:45:35.788627] [DEBUG] [LOCAL ] HBM: Trimmed 57559 records to 39386 (keeping 150 days of data) [2025-09-16T18:45:35.926145] [INFO ] [REMOTE ] HCI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:35.952832] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-02. [2025-09-16T18:45:35.989621] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-09-16T18:45:36.141816] [DEBUG] [LOCAL ] HBM: Primary data saved (39386 records). [2025-09-16T18:45:36.141958] [INFO ] [MAIN ] HBM: Full history saved. [2025-09-16T18:45:36.142060] [INFO ] [MAIN ] Update Progress: 997/2369 symbols completed. HBM took 23.6s. Estimated remaining: 2h 25m 15s. Rate: 9.4 symbols/min [2025-09-16T18:45:36.374884] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-09. [2025-09-16T18:45:36.452474] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-09-16T18:45:36.789481] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T18:45:37.174903] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-06. [2025-09-16T18:45:37.974893] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-03. [2025-09-16T18:45:38.123046] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-09-16T18:45:38.128393] [INFO ] [REMOTE ] HCA: Full fetch complete (51787 records). [2025-09-16T18:45:38.128485] [DEBUG] [VALIDATOR ] HCA: Validating primary data. [2025-09-16T18:45:38.515629] [DEBUG] [VALIDATOR ] HCA: All expected trading days are present. [2025-09-16T18:45:38.515770] [DEBUG] [VALIDATOR ] HCA: Primary data validation passed. [2025-09-16T18:45:38.516551] [INFO ] [MAIN ] HCA: Fetched primary data is valid. [2025-09-16T18:45:38.516596] [DEBUG] [LOCAL ] HCA: Saving primary data. [2025-09-16T18:45:38.517686] [DEBUG] [LOCAL ] HCA: Trimmed 51787 records to 35145 (keeping 150 days of data) [2025-09-16T18:45:38.740594] [DEBUG] [LOCAL ] HCA: Primary data saved (35145 records). [2025-09-16T18:45:38.740755] [INFO ] [MAIN ] HCA: Full history saved. [2025-09-16T18:45:38.741424] [INFO ] [MAIN ] Update Progress: 998/2369 symbols completed. HCA took 23.7s. Estimated remaining: 2h 25m 4s. Rate: 9.5 symbols/min [2025-09-16T18:45:38.765041] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T18:45:38.774832] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-07. [2025-09-16T18:45:38.827070] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-09-16T18:45:39.574951] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-04. [2025-09-16T18:45:40.256043] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T18:45:40.374889] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-08. [2025-09-16T18:45:40.558805] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-09-16T18:45:41.174898] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-05. [2025-09-16T18:45:41.903491] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T18:45:41.974952] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-09. [2025-09-16T18:45:42.065857] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-09-16T18:45:42.774945] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-06. [2025-09-16T18:45:43.194040] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-09-16T18:45:43.199354] [INFO ] [REMOTE ] HCC: Full fetch complete (49520 records). [2025-09-16T18:45:43.199456] [DEBUG] [VALIDATOR ] HCC: Validating primary data. [2025-09-16T18:45:43.590953] [DEBUG] [VALIDATOR ] HCC: All expected trading days are present. [2025-09-16T18:45:43.591049] [DEBUG] [VALIDATOR ] HCC: Primary data validation passed. [2025-09-16T18:45:43.591793] [INFO ] [MAIN ] HCC: Fetched primary data is valid. [2025-09-16T18:45:43.591841] [DEBUG] [LOCAL ] HCC: Saving primary data. [2025-09-16T18:45:43.593111] [DEBUG] [LOCAL ] HCC: Trimmed 49520 records to 33661 (keeping 150 days of data) [2025-09-16T18:45:43.804141] [DEBUG] [LOCAL ] HCC: Primary data saved (33661 records). [2025-09-16T18:45:43.804304] [INFO ] [MAIN ] HCC: Full history saved. [2025-09-16T18:45:43.804411] [INFO ] [MAIN ] Update Progress: 999/2369 symbols completed. HCC took 28.6s. Estimated remaining: 2h 24m 55s. Rate: 9.5 symbols/min [2025-09-16T18:45:43.814417] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-09-16T18:45:43.818409] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-07. [2025-09-16T18:45:44.955036] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T18:45:44.959137] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-08. [2025-09-16T18:45:45.958118] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-09-16T18:45:45.961483] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-09. [2025-09-16T18:45:46.695914] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T18:45:46.697916] [INFO ] [REMOTE ] HCI: Full fetch complete (14902 records). [2025-09-16T18:45:46.697984] [DEBUG] [VALIDATOR ] HCI: Validating primary data. [2025-09-16T18:45:46.698376] [FATAL] [VALIDATOR ] HCI: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:45:46.698467] [FATAL] [MAIN ] HCI: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:45:46.698504] [WARN ] [MAIN ] HCI: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:45:46.698540] [INFO ] [LOCAL ] HCI: Starting deletion of primary data. [2025-09-16T18:45:46.698595] [DEBUG] [LOCAL ] HCI: No primary data file found to delete. [2025-09-16T18:45:46.698631] [INFO ] [LOCAL ] HCI: Starting deletion of derivative data. [2025-09-16T18:45:46.698991] [DEBUG] [LOCAL ] HCI: Deleted derivative data file. [2025-09-16T18:45:46.699068] [INFO ] [MAIN ] Update Progress: 1000/2369 symbols completed. HCI took 12.0s. Estimated remaining: 2h 24m 44s. Rate: 9.5 symbols/min [2025-09-16T18:45:47.204153] [INFO ] [MAIN ] HD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:47.412904] [INFO ] [MAIN ] HCXY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:47.416591] [INFO ] [MAIN ] HDB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:47.417905] [INFO ] [MAIN ] HE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:47.418462] [DEBUG] [LOCAL ] HCXY: No existing data file found in primary storage. [2025-09-16T18:45:47.418729] [WARN ] [MAIN ] HCXY: No existing data found, performing full refresh instead. [2025-09-16T18:45:47.422140] [INFO ] [MAIN ] HCXY: Wiping existing primary data before full refresh. [2025-09-16T18:45:47.422289] [INFO ] [LOCAL ] HCXY: Starting deletion of primary data. [2025-09-16T18:45:47.422350] [DEBUG] [LOCAL ] HCXY: No primary data file found to delete. [2025-09-16T18:45:47.422394] [INFO ] [MAIN ] HCXY: Starting full primary refresh. [2025-09-16T18:45:47.750628] [DEBUG] [LOCAL ] HD: Loaded 39510 existing records from primary storage. [2025-09-16T18:45:47.783202] [DEBUG] [LOCAL ] HE: Loaded 35907 existing records from primary storage. [2025-09-16T18:45:47.781209] [INFO ] [MAIN ] HD: Loaded 39510 existing records. [2025-09-16T18:45:47.794015] [INFO ] [MAIN ] HE: Loaded 35907 existing records. [2025-09-16T18:45:47.864016] [DEBUG] [LOCAL ] HDB: Loaded 38022 existing records from primary storage. [2025-09-16T18:45:47.981468] [INFO ] [REMOTE ] HCXY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:48.000828] [INFO ] [MAIN ] HDB: Loaded 38022 existing records. [2025-09-16T18:45:48.016538] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-02. [2025-09-16T18:45:48.309093] [WARN ] [MAIN ] HD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:45:48.369931] [INFO ] [MAIN ] HD: Wiping existing primary data before full refresh. [2025-09-16T18:45:48.338881] [WARN ] [MAIN ] HE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:45:48.386777] [INFO ] [LOCAL ] HD: Starting deletion of primary data. [2025-09-16T18:45:48.407615] [INFO ] [MAIN ] HE: Wiping existing primary data before full refresh. [2025-09-16T18:45:48.407495] [WARN ] [MAIN ] HDB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:45:48.407777] [INFO ] [LOCAL ] HE: Starting deletion of primary data. [2025-09-16T18:45:48.407954] [INFO ] [MAIN ] HDB: Wiping existing primary data before full refresh. [2025-09-16T18:45:48.408004] [INFO ] [LOCAL ] HDB: Starting deletion of primary data. [2025-09-16T18:45:48.408603] [DEBUG] [LOCAL ] HE: Deleted primary data file. [2025-09-16T18:45:48.408675] [INFO ] [MAIN ] HE: Starting full primary refresh. [2025-09-16T18:45:48.419707] [DEBUG] [LOCAL ] HDB: Deleted primary data file. [2025-09-16T18:45:48.419670] [DEBUG] [LOCAL ] HD: Deleted primary data file. [2025-09-16T18:45:48.438994] [INFO ] [MAIN ] HDB: Starting full primary refresh. [2025-09-16T18:45:48.462977] [INFO ] [MAIN ] HD: Starting full primary refresh. [2025-09-16T18:45:49.065280] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-09-16T18:45:49.113736] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-03. [2025-09-16T18:45:49.478301] [INFO ] [REMOTE ] HDB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:49.490788] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-02. [2025-09-16T18:45:49.490389] [INFO ] [REMOTE ] HE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:49.542891] [INFO ] [REMOTE ] HD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:49.544214] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-02. [2025-09-16T18:45:49.543897] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-02. [2025-09-16T18:45:49.641874] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-09-16T18:45:49.974918] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-04. [2025-09-16T18:45:50.804038] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-09-16T18:45:50.804890] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-05. [2025-09-16T18:45:51.121451] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-09-16T18:45:51.575017] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-06. [2025-09-16T18:45:51.716029] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-02 (2.23 seconds). [2025-09-16T18:45:51.795353] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-02 (2.25 seconds). [2025-09-16T18:45:51.831320] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-02 (2.29 seconds). [2025-09-16T18:45:51.840280] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-09-16T18:45:52.374951] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-03. [2025-09-16T18:45:53.174965] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-03. [2025-09-16T18:45:53.974922] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-07. [2025-09-16T18:45:54.215914] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-07 (0.24 seconds). [2025-09-16T18:45:54.542672] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-03 (2.17 seconds). [2025-09-16T18:45:54.774937] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-08. [2025-09-16T18:45:55.031269] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-09-16T18:45:55.112245] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-09-16T18:45:55.574926] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-09. [2025-09-16T18:45:56.134272] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T18:45:56.134951] [INFO ] [REMOTE ] HCXY: Full fetch complete (385 records). [2025-09-16T18:45:56.135017] [DEBUG] [VALIDATOR ] HCXY: Validating primary data. [2025-09-16T18:45:56.310518] [WARN ] [VALIDATOR ] HCXY: Validation failed - insufficient historical data. Have 119 unique days, need 150 days [2025-09-16T18:45:56.310754] [FATAL] [MAIN ] HCXY: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:45:56.310826] [WARN ] [MAIN ] HCXY: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:45:56.310901] [INFO ] [LOCAL ] HCXY: Starting deletion of primary data. [2025-09-16T18:45:56.310970] [DEBUG] [LOCAL ] HCXY: No primary data file found to delete. [2025-09-16T18:45:56.311007] [INFO ] [LOCAL ] HCXY: Starting deletion of derivative data. [2025-09-16T18:45:56.311049] [DEBUG] [LOCAL ] HCXY: No derivative data file found to delete. [2025-09-16T18:45:56.311121] [INFO ] [MAIN ] Update Progress: 1001/2369 symbols completed. HCXY took 9.6s. Estimated remaining: 2h 24m 43s. Rate: 9.5 symbols/min [2025-09-16T18:45:56.380066] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-04. [2025-09-16T18:45:56.483850] [INFO ] [MAIN ] HEI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:45:56.506535] [DEBUG] [LOCAL ] HEI: Loaded 23374 existing records from primary storage. [2025-09-16T18:45:56.506721] [INFO ] [MAIN ] HEI: Loaded 23374 existing records. [2025-09-16T18:45:56.681137] [WARN ] [MAIN ] HEI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:45:56.681396] [INFO ] [MAIN ] HEI: Wiping existing primary data before full refresh. [2025-09-16T18:45:56.681446] [INFO ] [LOCAL ] HEI: Starting deletion of primary data. [2025-09-16T18:45:56.682008] [DEBUG] [LOCAL ] HEI: Deleted primary data file. [2025-09-16T18:45:56.682056] [INFO ] [MAIN ] HEI: Starting full primary refresh. [2025-09-16T18:45:57.051422] [INFO ] [REMOTE ] HEI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:45:57.174888] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-02. [2025-09-16T18:45:57.974925] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-04. [2025-09-16T18:45:58.201989] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T18:45:58.357807] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-09-16T18:45:58.775005] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-05. [2025-09-16T18:45:59.574910] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-03. [2025-09-16T18:46:00.269942] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-09-16T18:46:00.374923] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-03. [2025-09-16T18:46:00.487734] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-09-16T18:46:01.174917] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-06. [2025-09-16T18:46:01.662957] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-09-16T18:46:01.974953] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-05. [2025-09-16T18:46:02.133482] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T18:46:02.774955] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-04. [2025-09-16T18:46:02.867089] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T18:46:03.574876] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-04. [2025-09-16T18:46:04.176048] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-05 (2.20 seconds). [2025-09-16T18:46:04.374938] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-06. [2025-09-16T18:46:04.990410] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-04 (2.22 seconds). [2025-09-16T18:46:05.099826] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T18:46:05.174941] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-05. [2025-09-16T18:46:05.974913] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-07. [2025-09-16T18:46:06.420744] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-06 (2.05 seconds). [2025-09-16T18:46:06.774929] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-07. [2025-09-16T18:46:06.970270] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-09-16T18:46:07.574886] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-05. [2025-09-16T18:46:07.701897] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T18:46:08.374950] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-06. [2025-09-16T18:46:09.028883] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-09-16T18:46:09.067109] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T18:46:09.174938] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-08. [2025-09-16T18:46:09.798565] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T18:46:09.974905] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-07. [2025-09-16T18:46:10.774907] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-08. [2025-09-16T18:46:11.176844] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-09-16T18:46:11.534276] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-09-16T18:46:11.574882] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-08. [2025-09-16T18:46:12.060994] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T18:46:12.374913] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-09. [2025-09-16T18:46:13.174886] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-06. [2025-09-16T18:46:13.291394] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-09-16T18:46:13.793196] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T18:46:13.802546] [INFO ] [REMOTE ] HE: Full fetch complete (53920 records). [2025-09-16T18:46:13.802780] [DEBUG] [VALIDATOR ] HE: Validating primary data. [2025-09-16T18:46:13.977181] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-09. [2025-09-16T18:46:14.182979] [DEBUG] [VALIDATOR ] HE: All expected trading days are present. [2025-09-16T18:46:14.183068] [DEBUG] [VALIDATOR ] HE: Primary data validation passed. [2025-09-16T18:46:14.183827] [INFO ] [MAIN ] HE: Fetched primary data is valid. [2025-09-16T18:46:14.183895] [DEBUG] [LOCAL ] HE: Saving primary data. [2025-09-16T18:46:14.185163] [DEBUG] [LOCAL ] HE: Trimmed 53920 records to 36245 (keeping 150 days of data) [2025-09-16T18:46:14.413327] [DEBUG] [LOCAL ] HE: Primary data saved (36245 records). [2025-09-16T18:46:14.413518] [INFO ] [MAIN ] HE: Full history saved. [2025-09-16T18:46:14.413660] [INFO ] [MAIN ] Update Progress: 1002/2369 symbols completed. HE took 27.7s. Estimated remaining: 2h 24m 52s. Rate: 9.4 symbols/min [2025-09-16T18:46:14.592453] [INFO ] [MAIN ] HEQ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:46:14.598940] [DEBUG] [LOCAL ] HEQ: Loaded 4271 existing records from primary storage. [2025-09-16T18:46:14.599087] [INFO ] [MAIN ] HEQ: Loaded 4271 existing records. [2025-09-16T18:46:14.783047] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-09. [2025-09-16T18:46:14.782866] [WARN ] [MAIN ] HEQ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:46:14.784575] [INFO ] [MAIN ] HEQ: Wiping existing primary data before full refresh. [2025-09-16T18:46:14.784716] [INFO ] [LOCAL ] HEQ: Starting deletion of primary data. [2025-09-16T18:46:14.785163] [DEBUG] [LOCAL ] HEQ: Deleted primary data file. [2025-09-16T18:46:14.785319] [INFO ] [MAIN ] HEQ: Starting full primary refresh. [2025-09-16T18:46:15.109153] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-09-16T18:46:15.206571] [INFO ] [REMOTE ] HEQ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:46:15.574918] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-02. [2025-09-16T18:46:16.255177] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-09 (2.28 seconds). [2025-09-16T18:46:16.261537] [INFO ] [REMOTE ] HD: Full fetch complete (58182 records). [2025-09-16T18:46:16.261661] [DEBUG] [VALIDATOR ] HD: Validating primary data. [2025-09-16T18:46:16.380023] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-07. [2025-09-16T18:46:16.487420] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-09-16T18:46:16.653522] [DEBUG] [VALIDATOR ] HD: All expected trading days are present. [2025-09-16T18:46:16.653608] [DEBUG] [VALIDATOR ] HD: Primary data validation passed. [2025-09-16T18:46:16.654426] [INFO ] [MAIN ] HD: Fetched primary data is valid. [2025-09-16T18:46:16.654478] [DEBUG] [LOCAL ] HD: Saving primary data. [2025-09-16T18:46:16.671429] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-09-16T18:46:16.677986] [DEBUG] [LOCAL ] HD: Trimmed 58182 records to 39898 (keeping 150 days of data) [2025-09-16T18:46:16.679321] [INFO ] [REMOTE ] HDB: Full fetch complete (56505 records). [2025-09-16T18:46:16.679409] [DEBUG] [VALIDATOR ] HDB: Validating primary data. [2025-09-16T18:46:17.169176] [DEBUG] [VALIDATOR ] HDB: All expected trading days are present. [2025-09-16T18:46:17.169678] [DEBUG] [VALIDATOR ] HDB: Primary data validation passed. [2025-09-16T18:46:17.171569] [INFO ] [MAIN ] HDB: Fetched primary data is valid. [2025-09-16T18:46:17.171870] [DEBUG] [LOCAL ] HDB: Saving primary data. [2025-09-16T18:46:17.176986] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-03. [2025-09-16T18:46:17.177832] [DEBUG] [LOCAL ] HDB: Trimmed 56505 records to 38408 (keeping 150 days of data) [2025-09-16T18:46:17.558878] [DEBUG] [LOCAL ] HDB: Primary data saved (38408 records). [2025-09-16T18:46:17.559002] [INFO ] [MAIN ] HDB: Full history saved. [2025-09-16T18:46:17.559985] [INFO ] [MAIN ] Update Progress: 1003/2369 symbols completed. HDB took 30.9s. Estimated remaining: 2h 24m 41s. Rate: 9.4 symbols/min [2025-09-16T18:46:17.558798] [DEBUG] [LOCAL ] HD: Primary data saved (39898 records). [2025-09-16T18:46:17.571623] [INFO ] [MAIN ] HD: Full history saved. [2025-09-16T18:46:17.603443] [INFO ] [MAIN ] Update Progress: 1004/2369 symbols completed. HD took 30.9s. Estimated remaining: 2h 24m 27s. Rate: 9.5 symbols/min [2025-09-16T18:46:17.850247] [INFO ] [MAIN ] HESM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:46:17.903466] [INFO ] [MAIN ] HF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:46:17.903996] [DEBUG] [LOCAL ] HF: No existing data file found in primary storage. [2025-09-16T18:46:17.904043] [WARN ] [MAIN ] HF: No existing data found, performing full refresh instead. [2025-09-16T18:46:17.904486] [INFO ] [MAIN ] HF: Wiping existing primary data before full refresh. [2025-09-16T18:46:17.904606] [INFO ] [LOCAL ] HF: Starting deletion of primary data. [2025-09-16T18:46:17.904841] [DEBUG] [LOCAL ] HF: No primary data file found to delete. [2025-09-16T18:46:17.904881] [INFO ] [MAIN ] HF: Starting full primary refresh. [2025-09-16T18:46:18.012825] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-09-16T18:46:18.037746] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-04. [2025-09-16T18:46:18.139823] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T18:46:18.222827] [DEBUG] [LOCAL ] HESM: Loaded 35628 existing records from primary storage. [2025-09-16T18:46:18.244870] [INFO ] [MAIN ] HESM: Loaded 35628 existing records. [2025-09-16T18:46:18.473038] [INFO ] [REMOTE ] HF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:46:18.529989] [WARN ] [MAIN ] HESM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:46:18.530156] [INFO ] [MAIN ] HESM: Wiping existing primary data before full refresh. [2025-09-16T18:46:18.530200] [INFO ] [LOCAL ] HESM: Starting deletion of primary data. [2025-09-16T18:46:18.530961] [DEBUG] [LOCAL ] HESM: Deleted primary data file. [2025-09-16T18:46:18.531011] [INFO ] [MAIN ] HESM: Starting full primary refresh. [2025-09-16T18:46:18.780039] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-02. [2025-09-16T18:46:18.896043] [INFO ] [REMOTE ] HESM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:46:18.959178] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-09-16T18:46:19.282433] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-09-16T18:46:19.574907] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-02. [2025-09-16T18:46:20.374939] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-08. [2025-09-16T18:46:21.136612] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T18:46:21.174890] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-03. [2025-09-16T18:46:21.422319] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T18:46:21.974943] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-03. [2025-09-16T18:46:22.209952] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-03 (0.24 seconds). [2025-09-16T18:46:22.773266] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T18:46:22.782552] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-09. [2025-09-16T18:46:23.574900] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-04. [2025-09-16T18:46:23.846777] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-04 (0.27 seconds). [2025-09-16T18:46:24.155958] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-09-16T18:46:24.159791] [INFO ] [REMOTE ] HEI: Full fetch complete (36409 records). [2025-09-16T18:46:24.159922] [DEBUG] [VALIDATOR ] HEI: Validating primary data. [2025-09-16T18:46:24.383614] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-04. [2025-09-16T18:46:24.569404] [DEBUG] [VALIDATOR ] HEI: All expected trading days are present. [2025-09-16T18:46:24.569486] [DEBUG] [VALIDATOR ] HEI: Primary data validation passed. [2025-09-16T18:46:24.570186] [INFO ] [MAIN ] HEI: Fetched primary data is valid. [2025-09-16T18:46:24.570252] [DEBUG] [LOCAL ] HEI: Saving primary data. [2025-09-16T18:46:24.571473] [DEBUG] [LOCAL ] HEI: Trimmed 36409 records to 23607 (keeping 150 days of data) [2025-09-16T18:46:24.728049] [DEBUG] [LOCAL ] HEI: Primary data saved (23607 records). [2025-09-16T18:46:24.728197] [INFO ] [MAIN ] HEI: Full history saved. [2025-09-16T18:46:24.728300] [INFO ] [MAIN ] Update Progress: 1005/2369 symbols completed. HEI took 28.4s. Estimated remaining: 2h 24m 21s. Rate: 9.4 symbols/min [2025-09-16T18:46:24.896596] [INFO ] [MAIN ] HFND: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:46:24.896827] [DEBUG] [LOCAL ] HFND: No existing data file found in primary storage. [2025-09-16T18:46:24.896868] [WARN ] [MAIN ] HFND: No existing data found, performing full refresh instead. [2025-09-16T18:46:24.897034] [INFO ] [MAIN ] HFND: Wiping existing primary data before full refresh. [2025-09-16T18:46:24.897075] [INFO ] [LOCAL ] HFND: Starting deletion of primary data. [2025-09-16T18:46:24.897116] [DEBUG] [LOCAL ] HFND: No primary data file found to delete. [2025-09-16T18:46:24.897148] [INFO ] [MAIN ] HFND: Starting full primary refresh. [2025-09-16T18:46:25.180041] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-05. [2025-09-16T18:46:25.262455] [INFO ] [REMOTE ] HFND: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:46:25.690128] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-09-16T18:46:25.981136] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-02. [2025-09-16T18:46:25.987459] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T18:46:26.482136] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-09-16T18:46:26.774989] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-06. [2025-09-16T18:46:27.444819] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-09-16T18:46:27.574944] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-05. [2025-09-16T18:46:28.374960] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-07. [2025-09-16T18:46:28.960883] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-09-16T18:46:29.174890] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-05. [2025-09-16T18:46:29.218718] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T18:46:29.408303] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-05 (0.23 seconds). [2025-09-16T18:46:29.974927] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-06. [2025-09-16T18:46:30.229224] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-06 (0.25 seconds). [2025-09-16T18:46:30.774911] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-07. [2025-09-16T18:46:31.010717] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-07 (0.24 seconds). [2025-09-16T18:46:31.574901] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-03. [2025-09-16T18:46:32.009221] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-09-16T18:46:32.374925] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-04. [2025-09-16T18:46:32.813491] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-09-16T18:46:33.174977] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-06. [2025-09-16T18:46:33.974936] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-08. [2025-09-16T18:46:34.616597] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-09-16T18:46:34.737023] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T18:46:34.774843] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-07. [2025-09-16T18:46:35.574897] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-09. [2025-09-16T18:46:36.374904] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-08. [2025-09-16T18:46:36.404931] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-09-16T18:46:36.406791] [INFO ] [REMOTE ] HEQ: Full fetch complete (6532 records). [2025-09-16T18:46:36.406890] [DEBUG] [VALIDATOR ] HEQ: Validating primary data. [2025-09-16T18:46:36.620739] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-08 (0.25 seconds). [2025-09-16T18:46:36.804927] [DEBUG] [VALIDATOR ] HEQ: All expected trading days are present. [2025-09-16T18:46:36.805129] [DEBUG] [VALIDATOR ] HEQ: Primary data validation passed. [2025-09-16T18:46:36.805389] [INFO ] [MAIN ] HEQ: Fetched primary data is valid. [2025-09-16T18:46:36.805517] [DEBUG] [LOCAL ] HEQ: Saving primary data. [2025-09-16T18:46:36.806272] [DEBUG] [LOCAL ] HEQ: Trimmed 6532 records to 4338 (keeping 150 days of data) [2025-09-16T18:46:36.859836] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-09-16T18:46:36.876529] [DEBUG] [LOCAL ] HEQ: Primary data saved (4338 records). [2025-09-16T18:46:36.876690] [INFO ] [MAIN ] HEQ: Full history saved. [2025-09-16T18:46:36.876814] [INFO ] [MAIN ] Update Progress: 1006/2369 symbols completed. HEQ took 22.5s. Estimated remaining: 2h 24m 23s. Rate: 9.4 symbols/min [2025-09-16T18:46:37.054386] [INFO ] [MAIN ] HFRO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:46:37.070007] [DEBUG] [LOCAL ] HFRO: Loaded 11867 existing records from primary storage. [2025-09-16T18:46:37.070173] [INFO ] [MAIN ] HFRO: Loaded 11867 existing records. [2025-09-16T18:46:37.179977] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-05. [2025-09-16T18:46:37.256439] [WARN ] [MAIN ] HFRO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:46:37.256742] [INFO ] [MAIN ] HFRO: Wiping existing primary data before full refresh. [2025-09-16T18:46:37.256814] [INFO ] [LOCAL ] HFRO: Starting deletion of primary data. [2025-09-16T18:46:37.257281] [DEBUG] [LOCAL ] HFRO: Deleted primary data file. [2025-09-16T18:46:37.257345] [INFO ] [MAIN ] HFRO: Starting full primary refresh. [2025-09-16T18:46:37.599268] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T18:46:37.631296] [INFO ] [REMOTE ] HFRO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:46:37.974889] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-06. [2025-09-16T18:46:38.279224] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-06 (0.30 seconds). [2025-09-16T18:46:38.774891] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-08. [2025-09-16T18:46:39.574902] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-09. [2025-09-16T18:46:39.980080] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-09-16T18:46:39.980790] [INFO ] [REMOTE ] HF: Full fetch complete (293 records). [2025-09-16T18:46:39.980871] [DEBUG] [VALIDATOR ] HF: Validating primary data. [2025-09-16T18:46:40.161516] [WARN ] [VALIDATOR ] HF: Validation failed - insufficient historical data. Have 102 unique days, need 150 days [2025-09-16T18:46:40.161701] [FATAL] [MAIN ] HF: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:46:40.161771] [WARN ] [MAIN ] HF: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:46:40.161821] [INFO ] [LOCAL ] HF: Starting deletion of primary data. [2025-09-16T18:46:40.161880] [DEBUG] [LOCAL ] HF: No primary data file found to delete. [2025-09-16T18:46:40.161918] [INFO ] [LOCAL ] HF: Starting deletion of derivative data. [2025-09-16T18:46:40.161973] [DEBUG] [LOCAL ] HF: No derivative data file found to delete. [2025-09-16T18:46:40.162060] [INFO ] [MAIN ] Update Progress: 1007/2369 symbols completed. HF took 22.5s. Estimated remaining: 2h 24m 12s. Rate: 9.4 symbols/min [2025-09-16T18:46:40.333310] [INFO ] [MAIN ] HGER: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:46:40.345859] [DEBUG] [LOCAL ] HGER: Loaded 9870 existing records from primary storage. [2025-09-16T18:46:40.346001] [INFO ] [MAIN ] HGER: Loaded 9870 existing records. [2025-09-16T18:46:40.379972] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-02. [2025-09-16T18:46:40.528565] [WARN ] [MAIN ] HGER: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:46:40.528839] [INFO ] [MAIN ] HGER: Wiping existing primary data before full refresh. [2025-09-16T18:46:40.528923] [INFO ] [LOCAL ] HGER: Starting deletion of primary data. [2025-09-16T18:46:40.529373] [DEBUG] [LOCAL ] HGER: Deleted primary data file. [2025-09-16T18:46:40.529441] [INFO ] [MAIN ] HGER: Starting full primary refresh. [2025-09-16T18:46:40.924199] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-08 (2.15 seconds). [2025-09-16T18:46:40.928205] [INFO ] [REMOTE ] HGER: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:46:41.174864] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-09. [2025-09-16T18:46:41.220138] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T18:46:41.974916] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-03. [2025-09-16T18:46:42.774926] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-02. [2025-09-16T18:46:42.835862] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-09-16T18:46:42.842669] [INFO ] [REMOTE ] HESM: Full fetch complete (52499 records). [2025-09-16T18:46:42.842786] [DEBUG] [VALIDATOR ] HESM: Validating primary data. [2025-09-16T18:46:43.171487] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T18:46:43.246118] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-09-16T18:46:43.265572] [DEBUG] [VALIDATOR ] HESM: All expected trading days are present. [2025-09-16T18:46:43.265690] [DEBUG] [VALIDATOR ] HESM: Primary data validation passed. [2025-09-16T18:46:43.266456] [INFO ] [MAIN ] HESM: Fetched primary data is valid. [2025-09-16T18:46:43.266512] [DEBUG] [LOCAL ] HESM: Saving primary data. [2025-09-16T18:46:43.267771] [DEBUG] [LOCAL ] HESM: Trimmed 52499 records to 35958 (keeping 150 days of data) [2025-09-16T18:46:43.494744] [DEBUG] [LOCAL ] HESM: Primary data saved (35958 records). [2025-09-16T18:46:43.494904] [INFO ] [MAIN ] HESM: Full history saved. [2025-09-16T18:46:43.495025] [INFO ] [MAIN ] Update Progress: 1008/2369 symbols completed. HESM took 25.9s. Estimated remaining: 2h 24m 2s. Rate: 9.4 symbols/min [2025-09-16T18:46:43.579986] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-04. [2025-09-16T18:46:43.691221] [INFO ] [MAIN ] HGLB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:46:43.700942] [DEBUG] [LOCAL ] HGLB: Loaded 6515 existing records from primary storage. [2025-09-16T18:46:43.701070] [INFO ] [MAIN ] HGLB: Loaded 6515 existing records. [2025-09-16T18:46:43.887583] [WARN ] [MAIN ] HGLB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:46:43.887840] [INFO ] [MAIN ] HGLB: Wiping existing primary data before full refresh. [2025-09-16T18:46:43.887887] [INFO ] [LOCAL ] HGLB: Starting deletion of primary data. [2025-09-16T18:46:43.888149] [DEBUG] [LOCAL ] HGLB: Deleted primary data file. [2025-09-16T18:46:43.888205] [INFO ] [MAIN ] HGLB: Starting full primary refresh. [2025-09-16T18:46:44.261767] [INFO ] [REMOTE ] HGLB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:46:44.374926] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-02. [2025-09-16T18:46:44.472484] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-09-16T18:46:45.174949] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-07. [2025-09-16T18:46:45.246052] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T18:46:45.442943] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-07 (0.27 seconds). [2025-09-16T18:46:45.974941] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-03. [2025-09-16T18:46:46.764008] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T18:46:46.774841] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-04. [2025-09-16T18:46:47.574942] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-03. [2025-09-16T18:46:47.659322] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-09-16T18:46:48.365967] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T18:46:48.374844] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-04. [2025-09-16T18:46:49.174901] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-05. [2025-09-16T18:46:49.268013] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-09-16T18:46:49.887921] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-09-16T18:46:49.974930] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-08. [2025-09-16T18:46:50.445738] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-09-16T18:46:50.774949] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-09. [2025-09-16T18:46:51.253069] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-09-16T18:46:51.254328] [INFO ] [REMOTE ] HFND: Full fetch complete (1349 records). [2025-09-16T18:46:51.254431] [DEBUG] [VALIDATOR ] HFND: Validating primary data. [2025-09-16T18:46:51.454418] [WARN ] [VALIDATOR ] HFND: Validation failed - insufficient historical data. Have 146 unique days, need 150 days [2025-09-16T18:46:51.454604] [FATAL] [MAIN ] HFND: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:46:51.454673] [WARN ] [MAIN ] HFND: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:46:51.454732] [INFO ] [LOCAL ] HFND: Starting deletion of primary data. [2025-09-16T18:46:51.454811] [DEBUG] [LOCAL ] HFND: No primary data file found to delete. [2025-09-16T18:46:51.454856] [INFO ] [LOCAL ] HFND: Starting deletion of derivative data. [2025-09-16T18:46:51.454921] [DEBUG] [LOCAL ] HFND: No derivative data file found to delete. [2025-09-16T18:46:51.455015] [INFO ] [MAIN ] Update Progress: 1009/2369 symbols completed. HFND took 26.7s. Estimated remaining: 2h 23m 58s. Rate: 9.4 symbols/min [2025-09-16T18:46:51.580024] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-06. [2025-09-16T18:46:51.643198] [INFO ] [MAIN ] HGTY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:46:51.657613] [DEBUG] [LOCAL ] HGTY: Loaded 9760 existing records from primary storage. [2025-09-16T18:46:51.657771] [INFO ] [MAIN ] HGTY: Loaded 9760 existing records. [2025-09-16T18:46:51.843788] [WARN ] [MAIN ] HGTY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:46:51.844075] [INFO ] [MAIN ] HGTY: Wiping existing primary data before full refresh. [2025-09-16T18:46:51.844125] [INFO ] [LOCAL ] HGTY: Starting deletion of primary data. [2025-09-16T18:46:51.844541] [DEBUG] [LOCAL ] HGTY: Deleted primary data file. [2025-09-16T18:46:51.844589] [INFO ] [MAIN ] HGTY: Starting full primary refresh. [2025-09-16T18:46:52.202500] [INFO ] [REMOTE ] HGTY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:46:52.345950] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T18:46:52.374871] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-07. [2025-09-16T18:46:53.165074] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T18:46:53.174852] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-08. [2025-09-16T18:46:53.901332] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-09-16T18:46:53.974915] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-05. [2025-09-16T18:46:54.774929] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-05. [2025-09-16T18:46:54.838275] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T18:46:55.312262] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-09-16T18:46:55.574902] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-06. [2025-09-16T18:46:56.374901] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-09. [2025-09-16T18:46:56.432628] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-09-16T18:46:57.174892] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-07. [2025-09-16T18:46:57.234163] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T18:46:57.235991] [INFO ] [REMOTE ] HGLB: Full fetch complete (10784 records). [2025-09-16T18:46:57.236059] [DEBUG] [VALIDATOR ] HGLB: Validating primary data. [2025-09-16T18:46:57.592145] [DEBUG] [VALIDATOR ] HGLB: All expected trading days are present. [2025-09-16T18:46:57.592223] [DEBUG] [VALIDATOR ] HGLB: Primary data validation passed. [2025-09-16T18:46:57.592474] [INFO ] [MAIN ] HGLB: Fetched primary data is valid. [2025-09-16T18:46:57.592516] [DEBUG] [LOCAL ] HGLB: Saving primary data. [2025-09-16T18:46:57.592983] [DEBUG] [LOCAL ] HGLB: Trimmed 10784 records to 6562 (keeping 150 days of data) [2025-09-16T18:46:57.633325] [DEBUG] [LOCAL ] HGLB: Primary data saved (6562 records). [2025-09-16T18:46:57.633435] [INFO ] [MAIN ] HGLB: Full history saved. [2025-09-16T18:46:57.633521] [INFO ] [MAIN ] Update Progress: 1010/2369 symbols completed. HGLB took 14.1s. Estimated remaining: 2h 23m 51s. Rate: 9.4 symbols/min [2025-09-16T18:46:57.804989] [INFO ] [MAIN ] HGV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:46:57.839938] [DEBUG] [LOCAL ] HGV: Loaded 34755 existing records from primary storage. [2025-09-16T18:46:57.840079] [INFO ] [MAIN ] HGV: Loaded 34755 existing records. [2025-09-16T18:46:57.980902] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-02. [2025-09-16T18:46:58.017510] [WARN ] [MAIN ] HGV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:46:58.017777] [INFO ] [MAIN ] HGV: Wiping existing primary data before full refresh. [2025-09-16T18:46:58.017823] [INFO ] [LOCAL ] HGV: Starting deletion of primary data. [2025-09-16T18:46:58.018728] [DEBUG] [LOCAL ] HGV: Deleted primary data file. [2025-09-16T18:46:58.018779] [INFO ] [MAIN ] HGV: Starting full primary refresh. [2025-09-16T18:46:58.389914] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-09-16T18:46:58.421918] [INFO ] [REMOTE ] HGV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:46:58.545670] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-09-16T18:46:58.774912] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-08. [2025-09-16T18:46:59.574944] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-03. [2025-09-16T18:46:59.647458] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-09-16T18:47:00.106442] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-09-16T18:47:00.374918] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-09. [2025-09-16T18:47:01.174869] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-02. [2025-09-16T18:47:01.453066] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T18:47:01.455980] [INFO ] [REMOTE ] HGER: Full fetch complete (12968 records). [2025-09-16T18:47:01.456110] [DEBUG] [VALIDATOR ] HGER: Validating primary data. [2025-09-16T18:47:01.832822] [DEBUG] [VALIDATOR ] HGER: All expected trading days are present. [2025-09-16T18:47:01.832906] [DEBUG] [VALIDATOR ] HGER: Primary data validation passed. [2025-09-16T18:47:01.833195] [INFO ] [MAIN ] HGER: Fetched primary data is valid. [2025-09-16T18:47:01.833266] [DEBUG] [LOCAL ] HGER: Saving primary data. [2025-09-16T18:47:01.833869] [DEBUG] [LOCAL ] HGER: Trimmed 12968 records to 9995 (keeping 150 days of data) [2025-09-16T18:47:01.898936] [DEBUG] [LOCAL ] HGER: Primary data saved (9995 records). [2025-09-16T18:47:01.899077] [INFO ] [MAIN ] HGER: Full history saved. [2025-09-16T18:47:01.899163] [INFO ] [MAIN ] Update Progress: 1011/2369 symbols completed. HGER took 21.7s. Estimated remaining: 2h 23m 42s. Rate: 9.5 symbols/min [2025-09-16T18:47:01.980038] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-04. [2025-09-16T18:47:02.076983] [INFO ] [MAIN ] HHH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:47:02.102521] [DEBUG] [LOCAL ] HHH: Loaded 25501 existing records from primary storage. [2025-09-16T18:47:02.102676] [INFO ] [MAIN ] HHH: Loaded 25501 existing records. [2025-09-16T18:47:02.277970] [WARN ] [MAIN ] HHH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:47:02.278176] [INFO ] [MAIN ] HHH: Wiping existing primary data before full refresh. [2025-09-16T18:47:02.278223] [INFO ] [LOCAL ] HHH: Starting deletion of primary data. [2025-09-16T18:47:02.278910] [DEBUG] [LOCAL ] HHH: Deleted primary data file. [2025-09-16T18:47:02.278961] [INFO ] [MAIN ] HHH: Starting full primary refresh. [2025-09-16T18:47:02.637337] [INFO ] [REMOTE ] HHH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:47:02.661609] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T18:47:02.774867] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-02. [2025-09-16T18:47:02.802554] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-09-16T18:47:03.574904] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-05. [2025-09-16T18:47:04.153007] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T18:47:04.346784] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-09-16T18:47:04.374919] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-03. [2025-09-16T18:47:05.174920] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-03. [2025-09-16T18:47:05.971185] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T18:47:05.980171] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-06. [2025-09-16T18:47:06.104694] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-09-16T18:47:06.774909] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-04. [2025-09-16T18:47:06.917424] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-09-16T18:47:07.574860] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-07. [2025-09-16T18:47:07.766414] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-09-16T18:47:08.374905] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-05. [2025-09-16T18:47:08.504109] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-09-16T18:47:09.174916] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-06. [2025-09-16T18:47:09.933906] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T18:47:09.948371] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T18:47:09.974848] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-06. [2025-09-16T18:47:10.774889] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-08. [2025-09-16T18:47:11.398965] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T18:47:11.574897] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-07. [2025-09-16T18:47:11.762882] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T18:47:12.230482] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-09-16T18:47:12.374931] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-08. [2025-09-16T18:47:13.174921] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-07. [2025-09-16T18:47:13.193968] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-09-16T18:47:13.974931] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-04. [2025-09-16T18:47:14.532688] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-09-16T18:47:14.774942] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-09. [2025-09-16T18:47:15.393625] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T18:47:15.538969] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-09-16T18:47:15.541599] [INFO ] [REMOTE ] HFRO: Full fetch complete (19681 records). [2025-09-16T18:47:15.541720] [DEBUG] [VALIDATOR ] HFRO: Validating primary data. [2025-09-16T18:47:15.579998] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-09. [2025-09-16T18:47:15.917562] [DEBUG] [VALIDATOR ] HFRO: All expected trading days are present. [2025-09-16T18:47:15.917656] [DEBUG] [VALIDATOR ] HFRO: Primary data validation passed. [2025-09-16T18:47:15.918075] [INFO ] [MAIN ] HFRO: Fetched primary data is valid. [2025-09-16T18:47:15.918115] [DEBUG] [LOCAL ] HFRO: Saving primary data. [2025-09-16T18:47:15.918665] [DEBUG] [LOCAL ] HFRO: Trimmed 19681 records to 11949 (keeping 150 days of data) [2025-09-16T18:47:15.994600] [DEBUG] [LOCAL ] HFRO: Primary data saved (11949 records). [2025-09-16T18:47:15.994773] [INFO ] [MAIN ] HFRO: Full history saved. [2025-09-16T18:47:15.994878] [INFO ] [MAIN ] Update Progress: 1012/2369 symbols completed. HFRO took 39.1s. Estimated remaining: 2h 23m 46s. Rate: 9.4 symbols/min [2025-09-16T18:47:16.166585] [INFO ] [MAIN ] HI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:47:16.191421] [DEBUG] [LOCAL ] HI: Loaded 25684 existing records from primary storage. [2025-09-16T18:47:16.191572] [INFO ] [MAIN ] HI: Loaded 25684 existing records. [2025-09-16T18:47:16.367487] [WARN ] [MAIN ] HI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:47:16.367730] [INFO ] [MAIN ] HI: Wiping existing primary data before full refresh. [2025-09-16T18:47:16.367777] [INFO ] [LOCAL ] HI: Starting deletion of primary data. [2025-09-16T18:47:16.368516] [DEBUG] [LOCAL ] HI: Deleted primary data file. [2025-09-16T18:47:16.368565] [INFO ] [MAIN ] HI: Starting full primary refresh. [2025-09-16T18:47:16.381250] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-05. [2025-09-16T18:47:16.710154] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T18:47:16.734755] [INFO ] [REMOTE ] HI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:47:16.734916] [INFO ] [REMOTE ] HGTY: Full fetch complete (13217 records). [2025-09-16T18:47:16.735347] [DEBUG] [VALIDATOR ] HGTY: Validating primary data. [2025-09-16T18:47:17.099455] [DEBUG] [VALIDATOR ] HGTY: All expected trading days are present. [2025-09-16T18:47:17.099552] [DEBUG] [VALIDATOR ] HGTY: Primary data validation passed. [2025-09-16T18:47:17.099912] [INFO ] [MAIN ] HGTY: Fetched primary data is valid. [2025-09-16T18:47:17.099958] [DEBUG] [LOCAL ] HGTY: Saving primary data. [2025-09-16T18:47:17.100470] [DEBUG] [LOCAL ] HGTY: Trimmed 13217 records to 10001 (keeping 150 days of data) [2025-09-16T18:47:17.163116] [DEBUG] [LOCAL ] HGTY: Primary data saved (10001 records). [2025-09-16T18:47:17.163271] [INFO ] [MAIN ] HGTY: Full history saved. [2025-09-16T18:47:17.163386] [INFO ] [MAIN ] Update Progress: 1013/2369 symbols completed. HGTY took 25.7s. Estimated remaining: 2h 23m 33s. Rate: 9.4 symbols/min [2025-09-16T18:47:17.185209] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-08. [2025-09-16T18:47:17.337515] [INFO ] [MAIN ] HIG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:47:17.381315] [DEBUG] [LOCAL ] HIG: Loaded 37942 existing records from primary storage. [2025-09-16T18:47:17.381460] [INFO ] [MAIN ] HIG: Loaded 37942 existing records. [2025-09-16T18:47:17.557253] [WARN ] [MAIN ] HIG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:47:17.557505] [INFO ] [MAIN ] HIG: Wiping existing primary data before full refresh. [2025-09-16T18:47:17.557558] [INFO ] [LOCAL ] HIG: Starting deletion of primary data. [2025-09-16T18:47:17.558516] [DEBUG] [LOCAL ] HIG: Deleted primary data file. [2025-09-16T18:47:17.558570] [INFO ] [MAIN ] HIG: Starting full primary refresh. [2025-09-16T18:47:17.954582] [INFO ] [REMOTE ] HIG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:47:17.974858] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-02. [2025-09-16T18:47:18.126333] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-09-16T18:47:18.704499] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-09-16T18:47:18.774859] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-09. [2025-09-16T18:47:19.338887] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T18:47:19.574985] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-06. [2025-09-16T18:47:20.218573] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T18:47:20.223126] [INFO ] [REMOTE ] HHH: Full fetch complete (36038 records). [2025-09-16T18:47:20.223268] [DEBUG] [VALIDATOR ] HHH: Validating primary data. [2025-09-16T18:47:20.386129] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-02. [2025-09-16T18:47:20.644503] [DEBUG] [VALIDATOR ] HHH: All expected trading days are present. [2025-09-16T18:47:20.644601] [DEBUG] [VALIDATOR ] HHH: Primary data validation passed. [2025-09-16T18:47:20.645309] [INFO ] [MAIN ] HHH: Fetched primary data is valid. [2025-09-16T18:47:20.645359] [DEBUG] [LOCAL ] HHH: Saving primary data. [2025-09-16T18:47:20.646582] [DEBUG] [LOCAL ] HHH: Trimmed 36038 records to 25732 (keeping 150 days of data) [2025-09-16T18:47:20.807800] [DEBUG] [LOCAL ] HHH: Primary data saved (25732 records). [2025-09-16T18:47:20.807962] [INFO ] [MAIN ] HHH: Full history saved. [2025-09-16T18:47:20.808061] [INFO ] [MAIN ] Update Progress: 1014/2369 symbols completed. HHH took 18.9s. Estimated remaining: 2h 23m 23s. Rate: 9.5 symbols/min [2025-09-16T18:47:20.981092] [INFO ] [MAIN ] HII: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:47:21.004245] [DEBUG] [LOCAL ] HII: Loaded 24141 existing records from primary storage. [2025-09-16T18:47:21.004394] [INFO ] [MAIN ] HII: Loaded 24141 existing records. [2025-09-16T18:47:21.185065] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-03. [2025-09-16T18:47:21.185429] [WARN ] [MAIN ] HII: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:47:21.186821] [INFO ] [MAIN ] HII: Wiping existing primary data before full refresh. [2025-09-16T18:47:21.186885] [INFO ] [LOCAL ] HII: Starting deletion of primary data. [2025-09-16T18:47:21.211755] [DEBUG] [LOCAL ] HII: Deleted primary data file. [2025-09-16T18:47:21.212117] [INFO ] [MAIN ] HII: Starting full primary refresh. [2025-09-16T18:47:21.219285] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T18:47:21.590471] [INFO ] [REMOTE ] HII: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:47:21.909915] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T18:47:21.974891] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-03. [2025-09-16T18:47:22.774942] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-07. [2025-09-16T18:47:23.057941] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-09-16T18:47:23.370632] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T18:47:23.574888] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-04. [2025-09-16T18:47:24.374953] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-04. [2025-09-16T18:47:24.425553] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T18:47:25.140411] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-09-16T18:47:25.174853] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-05. [2025-09-16T18:47:25.974921] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-02. [2025-09-16T18:47:26.211383] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T18:47:26.245998] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-09-16T18:47:26.774957] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-05. [2025-09-16T18:47:27.340821] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T18:47:27.574980] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-08. [2025-09-16T18:47:28.374948] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-03. [2025-09-16T18:47:28.515334] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T18:47:28.890143] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-09-16T18:47:29.174910] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-09. [2025-09-16T18:47:29.974951] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-06. [2025-09-16T18:47:30.153378] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T18:47:30.584005] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T18:47:30.589509] [INFO ] [REMOTE ] HGV: Full fetch complete (51060 records). [2025-09-16T18:47:30.589594] [DEBUG] [VALIDATOR ] HGV: Validating primary data. [2025-09-16T18:47:30.781522] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-06. [2025-09-16T18:47:30.970717] [DEBUG] [VALIDATOR ] HGV: All expected trading days are present. [2025-09-16T18:47:30.970862] [DEBUG] [VALIDATOR ] HGV: Primary data validation passed. [2025-09-16T18:47:30.971661] [INFO ] [MAIN ] HGV: Fetched primary data is valid. [2025-09-16T18:47:30.971735] [DEBUG] [LOCAL ] HGV: Saving primary data. [2025-09-16T18:47:30.972996] [DEBUG] [LOCAL ] HGV: Trimmed 51060 records to 35072 (keeping 150 days of data) [2025-09-16T18:47:31.077238] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-09-16T18:47:31.201439] [DEBUG] [LOCAL ] HGV: Primary data saved (35072 records). [2025-09-16T18:47:31.201576] [INFO ] [MAIN ] HGV: Full history saved. [2025-09-16T18:47:31.201673] [INFO ] [MAIN ] Update Progress: 1015/2369 symbols completed. HGV took 33.6s. Estimated remaining: 2h 23m 22s. Rate: 9.4 symbols/min [2025-09-16T18:47:31.369945] [INFO ] [MAIN ] HIMS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:47:31.406480] [DEBUG] [LOCAL ] HIMS: Loaded 39777 existing records from primary storage. [2025-09-16T18:47:31.406620] [INFO ] [MAIN ] HIMS: Loaded 39777 existing records. [2025-09-16T18:47:31.579627] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-07. [2025-09-16T18:47:31.582953] [WARN ] [MAIN ] HIMS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:47:31.584103] [INFO ] [MAIN ] HIMS: Wiping existing primary data before full refresh. [2025-09-16T18:47:31.584150] [INFO ] [LOCAL ] HIMS: Starting deletion of primary data. [2025-09-16T18:47:31.585071] [DEBUG] [LOCAL ] HIMS: Deleted primary data file. [2025-09-16T18:47:31.585123] [INFO ] [MAIN ] HIMS: Starting full primary refresh. [2025-09-16T18:47:31.940061] [INFO ] [REMOTE ] HIMS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:47:32.374876] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-04. [2025-09-16T18:47:32.482262] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T18:47:32.820328] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-09-16T18:47:33.174945] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-08. [2025-09-16T18:47:33.974898] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-02. [2025-09-16T18:47:34.041017] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T18:47:34.308042] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T18:47:34.774902] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-07. [2025-09-16T18:47:35.574913] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-05. [2025-09-16T18:47:36.374907] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-09. [2025-09-16T18:47:36.431768] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-02 (2.46 seconds). [2025-09-16T18:47:36.616900] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-09-16T18:47:37.011320] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T18:47:37.174916] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-03. [2025-09-16T18:47:37.630248] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T18:47:37.634537] [INFO ] [REMOTE ] HI: Full fetch complete (37974 records). [2025-09-16T18:47:37.634680] [DEBUG] [VALIDATOR ] HI: Validating primary data. [2025-09-16T18:47:37.977908] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-06. [2025-09-16T18:47:38.008009] [DEBUG] [VALIDATOR ] HI: All expected trading days are present. [2025-09-16T18:47:38.008087] [DEBUG] [VALIDATOR ] HI: Primary data validation passed. [2025-09-16T18:47:38.008598] [INFO ] [MAIN ] HI: Fetched primary data is valid. [2025-09-16T18:47:38.008669] [DEBUG] [LOCAL ] HI: Saving primary data. [2025-09-16T18:47:38.009465] [DEBUG] [LOCAL ] HI: Trimmed 37974 records to 25850 (keeping 150 days of data) [2025-09-16T18:47:38.169246] [DEBUG] [LOCAL ] HI: Primary data saved (25850 records). [2025-09-16T18:47:38.169382] [INFO ] [MAIN ] HI: Full history saved. [2025-09-16T18:47:38.169481] [INFO ] [MAIN ] Update Progress: 1016/2369 symbols completed. HI took 22.2s. Estimated remaining: 2h 23m 16s. Rate: 9.4 symbols/min [2025-09-16T18:47:38.338950] [INFO ] [MAIN ] HIO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:47:38.353999] [DEBUG] [LOCAL ] HIO: Loaded 15857 existing records from primary storage. [2025-09-16T18:47:38.354120] [INFO ] [MAIN ] HIO: Loaded 15857 existing records. [2025-09-16T18:47:38.530535] [WARN ] [MAIN ] HIO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:47:38.530823] [INFO ] [MAIN ] HIO: Wiping existing primary data before full refresh. [2025-09-16T18:47:38.530871] [INFO ] [LOCAL ] HIO: Starting deletion of primary data. [2025-09-16T18:47:38.531343] [DEBUG] [LOCAL ] HIO: Deleted primary data file. [2025-09-16T18:47:38.531391] [INFO ] [MAIN ] HIO: Starting full primary refresh. [2025-09-16T18:47:38.780017] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-08. [2025-09-16T18:47:38.880472] [INFO ] [REMOTE ] HIO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:47:39.456329] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T18:47:39.581680] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-07. [2025-09-16T18:47:39.584450] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-03 (2.41 seconds). [2025-09-16T18:47:40.060983] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-09-16T18:47:40.374898] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-09. [2025-09-16T18:47:40.582294] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T18:47:41.174921] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-04. [2025-09-16T18:47:41.724682] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T18:47:41.730885] [INFO ] [REMOTE ] HIG: Full fetch complete (55602 records). [2025-09-16T18:47:41.731006] [DEBUG] [VALIDATOR ] HIG: Validating primary data. [2025-09-16T18:47:41.983740] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-08. [2025-09-16T18:47:42.120132] [DEBUG] [VALIDATOR ] HIG: All expected trading days are present. [2025-09-16T18:47:42.120223] [DEBUG] [VALIDATOR ] HIG: Primary data validation passed. [2025-09-16T18:47:42.121045] [INFO ] [MAIN ] HIG: Fetched primary data is valid. [2025-09-16T18:47:42.121092] [DEBUG] [LOCAL ] HIG: Saving primary data. [2025-09-16T18:47:42.122341] [DEBUG] [LOCAL ] HIG: Trimmed 55602 records to 38323 (keeping 150 days of data) [2025-09-16T18:47:42.367581] [DEBUG] [LOCAL ] HIG: Primary data saved (38323 records). [2025-09-16T18:47:42.367746] [INFO ] [MAIN ] HIG: Full history saved. [2025-09-16T18:47:42.369473] [INFO ] [MAIN ] Update Progress: 1017/2369 symbols completed. HIG took 25.2s. Estimated remaining: 2h 23m 7s. Rate: 9.4 symbols/min [2025-09-16T18:47:42.541191] [INFO ] [MAIN ] HIPO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:47:42.557725] [DEBUG] [LOCAL ] HIPO: Loaded 16581 existing records from primary storage. [2025-09-16T18:47:42.557858] [INFO ] [MAIN ] HIPO: Loaded 16581 existing records. [2025-09-16T18:47:42.740717] [WARN ] [MAIN ] HIPO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:47:42.740993] [INFO ] [MAIN ] HIPO: Wiping existing primary data before full refresh. [2025-09-16T18:47:42.741066] [INFO ] [LOCAL ] HIPO: Starting deletion of primary data. [2025-09-16T18:47:42.741548] [DEBUG] [LOCAL ] HIPO: Deleted primary data file. [2025-09-16T18:47:42.741597] [INFO ] [MAIN ] HIPO: Starting full primary refresh. [2025-09-16T18:47:42.784534] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-02. [2025-09-16T18:47:43.094123] [INFO ] [REMOTE ] HIPO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:47:43.419438] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T18:47:43.531861] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-04 (2.36 seconds). [2025-09-16T18:47:43.574862] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-05. [2025-09-16T18:47:44.345842] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T18:47:44.374847] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-03. [2025-09-16T18:47:45.174899] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-09. [2025-09-16T18:47:45.667173] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-05 (2.09 seconds). [2025-09-16T18:47:45.696089] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-09-16T18:47:45.974891] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-04. [2025-09-16T18:47:46.468175] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T18:47:46.472174] [INFO ] [REMOTE ] HII: Full fetch complete (39745 records). [2025-09-16T18:47:46.472253] [DEBUG] [VALIDATOR ] HII: Validating primary data. [2025-09-16T18:47:46.780117] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-06. [2025-09-16T18:47:46.856389] [DEBUG] [VALIDATOR ] HII: All expected trading days are present. [2025-09-16T18:47:46.856475] [DEBUG] [VALIDATOR ] HII: Primary data validation passed. [2025-09-16T18:47:46.857234] [INFO ] [MAIN ] HII: Fetched primary data is valid. [2025-09-16T18:47:46.857282] [DEBUG] [LOCAL ] HII: Saving primary data. [2025-09-16T18:47:46.858171] [DEBUG] [LOCAL ] HII: Trimmed 39745 records to 24405 (keeping 150 days of data) [2025-09-16T18:47:46.979892] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-09-16T18:47:47.045801] [DEBUG] [LOCAL ] HII: Primary data saved (24405 records). [2025-09-16T18:47:47.045969] [INFO ] [MAIN ] HII: Full history saved. [2025-09-16T18:47:47.046082] [INFO ] [MAIN ] Update Progress: 1018/2369 symbols completed. HII took 26.2s. Estimated remaining: 2h 22m 58s. Rate: 9.4 symbols/min [2025-09-16T18:47:47.215536] [INFO ] [MAIN ] HIW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:47:47.253224] [DEBUG] [LOCAL ] HIW: Loaded 32873 existing records from primary storage. [2025-09-16T18:47:47.253385] [INFO ] [MAIN ] HIW: Loaded 32873 existing records. [2025-09-16T18:47:47.429603] [WARN ] [MAIN ] HIW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:47:47.429886] [INFO ] [MAIN ] HIW: Wiping existing primary data before full refresh. [2025-09-16T18:47:47.429936] [INFO ] [LOCAL ] HIW: Starting deletion of primary data. [2025-09-16T18:47:47.430825] [DEBUG] [LOCAL ] HIW: Deleted primary data file. [2025-09-16T18:47:47.430875] [INFO ] [MAIN ] HIW: Starting full primary refresh. [2025-09-16T18:47:47.577110] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-05. [2025-09-16T18:47:47.791124] [INFO ] [REMOTE ] HIW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:47:48.374889] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-02. [2025-09-16T18:47:48.708341] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-09-16T18:47:49.163206] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-06 (2.38 seconds). [2025-09-16T18:47:49.174836] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-07. [2025-09-16T18:47:49.330877] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-09-16T18:47:49.974899] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-06. [2025-09-16T18:47:50.774947] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-03. [2025-09-16T18:47:50.918066] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-09-16T18:47:51.574896] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-07. [2025-09-16T18:47:51.745421] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-07 (2.57 seconds). [2025-09-16T18:47:52.240448] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-09-16T18:47:52.374921] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-04. [2025-09-16T18:47:52.608179] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-09-16T18:47:53.174974] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-02. [2025-09-16T18:47:53.874076] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T18:47:53.924370] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T18:47:53.974861] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-03. [2025-09-16T18:47:54.774933] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-08. [2025-09-16T18:47:55.094041] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-09-16T18:47:55.574909] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-08. [2025-09-16T18:47:56.374918] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-05. [2025-09-16T18:47:56.715677] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T18:47:57.056163] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-08 (2.28 seconds). [2025-09-16T18:47:57.174948] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-09. [2025-09-16T18:47:57.963863] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T18:47:57.974946] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-06. [2025-09-16T18:47:58.421764] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-09-16T18:47:58.425511] [INFO ] [REMOTE ] HIO: Full fetch complete (26825 records). [2025-09-16T18:47:58.425624] [DEBUG] [VALIDATOR ] HIO: Validating primary data. [2025-09-16T18:47:58.780500] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-04. [2025-09-16T18:47:58.810336] [DEBUG] [VALIDATOR ] HIO: All expected trading days are present. [2025-09-16T18:47:58.810434] [DEBUG] [VALIDATOR ] HIO: Primary data validation passed. [2025-09-16T18:47:58.811138] [INFO ] [MAIN ] HIO: Fetched primary data is valid. [2025-09-16T18:47:58.811185] [DEBUG] [LOCAL ] HIO: Saving primary data. [2025-09-16T18:47:58.811913] [DEBUG] [LOCAL ] HIO: Trimmed 26825 records to 16041 (keeping 150 days of data) [2025-09-16T18:47:58.904975] [DEBUG] [LOCAL ] HIO: Primary data saved (16041 records). [2025-09-16T18:47:58.905125] [INFO ] [MAIN ] HIO: Full history saved. [2025-09-16T18:47:58.905213] [INFO ] [MAIN ] Update Progress: 1019/2369 symbols completed. HIO took 20.7s. Estimated remaining: 2h 22m 59s. Rate: 9.4 symbols/min [2025-09-16T18:47:59.077427] [INFO ] [MAIN ] HIX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:47:59.092766] [DEBUG] [LOCAL ] HIX: Loaded 14901 existing records from primary storage. [2025-09-16T18:47:59.092881] [INFO ] [MAIN ] HIX: Loaded 14901 existing records. [2025-09-16T18:47:59.267932] [WARN ] [MAIN ] HIX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:47:59.268185] [INFO ] [MAIN ] HIX: Wiping existing primary data before full refresh. [2025-09-16T18:47:59.268238] [INFO ] [LOCAL ] HIX: Starting deletion of primary data. [2025-09-16T18:47:59.268885] [DEBUG] [LOCAL ] HIX: Deleted primary data file. [2025-09-16T18:47:59.268941] [INFO ] [MAIN ] HIX: Starting full primary refresh. [2025-09-16T18:47:59.579049] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-09. [2025-09-16T18:47:59.638391] [INFO ] [REMOTE ] HIX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:47:59.662339] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T18:47:59.908782] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-09-16T18:48:00.374928] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-02. [2025-09-16T18:48:01.174953] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-05. [2025-09-16T18:48:01.587611] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-09-16T18:48:01.975048] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-07. [2025-09-16T18:48:02.055280] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-09 (2.48 seconds). [2025-09-16T18:48:02.063448] [INFO ] [REMOTE ] HIMS: Full fetch complete (58497 records). [2025-09-16T18:48:02.063959] [DEBUG] [VALIDATOR ] HIMS: Validating primary data. [2025-09-16T18:48:02.147164] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T18:48:02.568923] [DEBUG] [VALIDATOR ] HIMS: All expected trading days are present. [2025-09-16T18:48:02.569077] [DEBUG] [VALIDATOR ] HIMS: Primary data validation passed. [2025-09-16T18:48:02.570473] [INFO ] [MAIN ] HIMS: Fetched primary data is valid. [2025-09-16T18:48:02.570555] [DEBUG] [LOCAL ] HIMS: Saving primary data. [2025-09-16T18:48:02.572266] [DEBUG] [LOCAL ] HIMS: Trimmed 58497 records to 40167 (keeping 150 days of data) [2025-09-16T18:48:02.782424] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-03. [2025-09-16T18:48:02.824913] [DEBUG] [LOCAL ] HIMS: Primary data saved (40167 records). [2025-09-16T18:48:02.825074] [INFO ] [MAIN ] HIMS: Full history saved. [2025-09-16T18:48:02.825172] [INFO ] [MAIN ] Update Progress: 1020/2369 symbols completed. HIMS took 31.6s. Estimated remaining: 2h 22m 50s. Rate: 9.4 symbols/min [2025-09-16T18:48:02.998469] [INFO ] [MAIN ] HKD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:48:03.037423] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-09-16T18:48:03.053165] [DEBUG] [LOCAL ] HKD: Loaded 13415 existing records from primary storage. [2025-09-16T18:48:03.053862] [INFO ] [MAIN ] HKD: Loaded 13415 existing records. [2025-09-16T18:48:03.237529] [WARN ] [MAIN ] HKD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:48:03.237835] [INFO ] [MAIN ] HKD: Wiping existing primary data before full refresh. [2025-09-16T18:48:03.237885] [INFO ] [LOCAL ] HKD: Starting deletion of primary data. [2025-09-16T18:48:03.238268] [DEBUG] [LOCAL ] HKD: Deleted primary data file. [2025-09-16T18:48:03.238313] [INFO ] [MAIN ] HKD: Starting full primary refresh. [2025-09-16T18:48:03.583015] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-06. [2025-09-16T18:48:03.598075] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-09-16T18:48:03.626608] [INFO ] [REMOTE ] HKD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:48:04.374953] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-04. [2025-09-16T18:48:04.602003] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-09-16T18:48:05.174890] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-08. [2025-09-16T18:48:05.488373] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-09-16T18:48:05.974916] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-07. [2025-09-16T18:48:06.706097] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T18:48:06.774966] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-09. [2025-09-16T18:48:06.920553] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-09-16T18:48:07.574934] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-05. [2025-09-16T18:48:08.280811] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-09-16T18:48:08.289168] [INFO ] [REMOTE ] HIW: Full fetch complete (48975 records). [2025-09-16T18:48:08.289288] [DEBUG] [VALIDATOR ] HIW: Validating primary data. [2025-09-16T18:48:08.380089] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-08. [2025-09-16T18:48:08.684108] [DEBUG] [VALIDATOR ] HIW: All expected trading days are present. [2025-09-16T18:48:08.684215] [DEBUG] [VALIDATOR ] HIW: Primary data validation passed. [2025-09-16T18:48:08.685103] [INFO ] [MAIN ] HIW: Fetched primary data is valid. [2025-09-16T18:48:08.685344] [DEBUG] [LOCAL ] HIW: Saving primary data. [2025-09-16T18:48:08.699353] [DEBUG] [LOCAL ] HIW: Trimmed 48975 records to 33209 (keeping 150 days of data) [2025-09-16T18:48:08.699917] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T18:48:08.911560] [DEBUG] [LOCAL ] HIW: Primary data saved (33209 records). [2025-09-16T18:48:08.911745] [INFO ] [MAIN ] HIW: Full history saved. [2025-09-16T18:48:08.913658] [INFO ] [MAIN ] Update Progress: 1021/2369 symbols completed. HIW took 21.9s. Estimated remaining: 2h 22m 43s. Rate: 9.4 symbols/min [2025-09-16T18:48:09.080821] [INFO ] [MAIN ] HL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:48:09.119853] [DEBUG] [LOCAL ] HL: Loaded 39637 existing records from primary storage. [2025-09-16T18:48:09.120023] [INFO ] [MAIN ] HL: Loaded 39637 existing records. [2025-09-16T18:48:09.181061] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-02. [2025-09-16T18:48:09.297467] [WARN ] [MAIN ] HL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:48:09.297719] [INFO ] [MAIN ] HL: Wiping existing primary data before full refresh. [2025-09-16T18:48:09.297765] [INFO ] [LOCAL ] HL: Starting deletion of primary data. [2025-09-16T18:48:09.298624] [DEBUG] [LOCAL ] HL: Deleted primary data file. [2025-09-16T18:48:09.298699] [INFO ] [MAIN ] HL: Starting full primary refresh. [2025-09-16T18:48:09.655482] [INFO ] [REMOTE ] HL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:48:09.668571] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T18:48:09.974988] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-06. [2025-09-16T18:48:10.401599] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T18:48:10.732084] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-09-16T18:48:10.774909] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-07. [2025-09-16T18:48:11.574959] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-03. [2025-09-16T18:48:11.799618] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T18:48:12.374995] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-08. [2025-09-16T18:48:12.552916] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-09-16T18:48:13.175008] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-04. [2025-09-16T18:48:13.423361] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T18:48:13.974968] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-09. [2025-09-16T18:48:14.205042] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T18:48:14.756441] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-09-16T18:48:14.759039] [INFO ] [REMOTE ] HIX: Full fetch complete (21811 records). [2025-09-16T18:48:14.759129] [DEBUG] [VALIDATOR ] HIX: Validating primary data. [2025-09-16T18:48:14.780913] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-09. [2025-09-16T18:48:15.157556] [DEBUG] [VALIDATOR ] HIX: All expected trading days are present. [2025-09-16T18:48:15.157669] [DEBUG] [VALIDATOR ] HIX: Primary data validation passed. [2025-09-16T18:48:15.158278] [INFO ] [MAIN ] HIX: Fetched primary data is valid. [2025-09-16T18:48:15.158347] [DEBUG] [LOCAL ] HIX: Saving primary data. [2025-09-16T18:48:15.159027] [DEBUG] [LOCAL ] HIX: Trimmed 21811 records to 15027 (keeping 150 days of data) [2025-09-16T18:48:15.247938] [DEBUG] [LOCAL ] HIX: Primary data saved (15027 records). [2025-09-16T18:48:15.248083] [INFO ] [MAIN ] HIX: Full history saved. [2025-09-16T18:48:15.248190] [INFO ] [MAIN ] Update Progress: 1022/2369 symbols completed. HIX took 16.3s. Estimated remaining: 2h 22m 37s. Rate: 9.4 symbols/min [2025-09-16T18:48:15.420080] [INFO ] [MAIN ] HLF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:48:15.456927] [DEBUG] [LOCAL ] HLF: Loaded 36076 existing records from primary storage. [2025-09-16T18:48:15.457093] [INFO ] [MAIN ] HLF: Loaded 36076 existing records. [2025-09-16T18:48:15.580127] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-05. [2025-09-16T18:48:15.641698] [WARN ] [MAIN ] HLF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:48:15.642007] [INFO ] [MAIN ] HLF: Wiping existing primary data before full refresh. [2025-09-16T18:48:15.642137] [INFO ] [LOCAL ] HLF: Starting deletion of primary data. [2025-09-16T18:48:15.648164] [DEBUG] [LOCAL ] HLF: Deleted primary data file. [2025-09-16T18:48:15.654163] [INFO ] [MAIN ] HLF: Starting full primary refresh. [2025-09-16T18:48:15.688164] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-09-16T18:48:15.762574] [INFO ] [REMOTE ] HIPO: Full fetch complete (24489 records). [2025-09-16T18:48:15.763117] [DEBUG] [VALIDATOR ] HIPO: Validating primary data. [2025-09-16T18:48:16.245898] [INFO ] [REMOTE ] HLF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:48:16.380034] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-02. [2025-09-16T18:48:16.441969] [DEBUG] [VALIDATOR ] HIPO: All expected trading days are present. [2025-09-16T18:48:16.442048] [DEBUG] [VALIDATOR ] HIPO: Primary data validation passed. [2025-09-16T18:48:16.442619] [INFO ] [MAIN ] HIPO: Fetched primary data is valid. [2025-09-16T18:48:16.442678] [DEBUG] [LOCAL ] HIPO: Saving primary data. [2025-09-16T18:48:16.443373] [DEBUG] [LOCAL ] HIPO: Trimmed 24489 records to 16781 (keeping 150 days of data) [2025-09-16T18:48:16.547362] [DEBUG] [LOCAL ] HIPO: Primary data saved (16781 records). [2025-09-16T18:48:16.547508] [INFO ] [MAIN ] HIPO: Full history saved. [2025-09-16T18:48:16.547596] [INFO ] [MAIN ] Update Progress: 1023/2369 symbols completed. HIPO took 34.2s. Estimated remaining: 2h 22m 24s. Rate: 9.5 symbols/min [2025-09-16T18:48:16.718233] [INFO ] [MAIN ] HLI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:48:16.727484] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T18:48:16.750273] [DEBUG] [LOCAL ] HLI: Loaded 22468 existing records from primary storage. [2025-09-16T18:48:16.750408] [INFO ] [MAIN ] HLI: Loaded 22468 existing records. [2025-09-16T18:48:16.924422] [WARN ] [MAIN ] HLI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:48:16.924708] [INFO ] [MAIN ] HLI: Wiping existing primary data before full refresh. [2025-09-16T18:48:16.924774] [INFO ] [LOCAL ] HLI: Starting deletion of primary data. [2025-09-16T18:48:16.925421] [DEBUG] [LOCAL ] HLI: Deleted primary data file. [2025-09-16T18:48:16.925471] [INFO ] [MAIN ] HLI: Starting full primary refresh. [2025-09-16T18:48:17.180127] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-06. [2025-09-16T18:48:17.296533] [INFO ] [REMOTE ] HLI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:48:17.974880] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-02. [2025-09-16T18:48:18.016193] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:48:18.421824] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T18:48:18.774906] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-02. [2025-09-16T18:48:19.113708] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-09-16T18:48:19.574925] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-03. [2025-09-16T18:48:20.343153] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T18:48:20.374859] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-03. [2025-09-16T18:48:20.997085] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T18:48:21.174902] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-04. [2025-09-16T18:48:21.441323] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-09-16T18:48:21.974901] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-04. [2025-09-16T18:48:22.598266] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T18:48:22.774911] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-07. [2025-09-16T18:48:23.216207] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-09-16T18:48:23.574891] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-03. [2025-09-16T18:48:23.860217] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T18:48:24.374903] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-05. [2025-09-16T18:48:25.174879] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-08. [2025-09-16T18:48:25.313661] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T18:48:25.527105] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T18:48:25.974894] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-05. [2025-09-16T18:48:26.257244] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T18:48:26.774896] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-04. [2025-09-16T18:48:27.324400] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T18:48:27.574906] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-06. [2025-09-16T18:48:28.374872] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-06. [2025-09-16T18:48:28.778188] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-09-16T18:48:28.805409] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T18:48:29.174910] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-07. [2025-09-16T18:48:29.974917] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-09. [2025-09-16T18:48:30.039891] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-09-16T18:48:30.555368] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-09-16T18:48:30.774913] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-07. [2025-09-16T18:48:30.943229] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T18:48:30.946514] [INFO ] [REMOTE ] HKD: Full fetch complete (20089 records). [2025-09-16T18:48:30.946619] [DEBUG] [VALIDATOR ] HKD: Validating primary data. [2025-09-16T18:48:31.322048] [DEBUG] [VALIDATOR ] HKD: All expected trading days are present. [2025-09-16T18:48:31.322145] [DEBUG] [VALIDATOR ] HKD: Primary data validation passed. [2025-09-16T18:48:31.322740] [INFO ] [MAIN ] HKD: Fetched primary data is valid. [2025-09-16T18:48:31.322801] [DEBUG] [LOCAL ] HKD: Saving primary data. [2025-09-16T18:48:31.323477] [DEBUG] [LOCAL ] HKD: Trimmed 20089 records to 13531 (keeping 150 days of data) [2025-09-16T18:48:31.405500] [DEBUG] [LOCAL ] HKD: Primary data saved (13531 records). [2025-09-16T18:48:31.405675] [INFO ] [MAIN ] HKD: Full history saved. [2025-09-16T18:48:31.405790] [INFO ] [MAIN ] Update Progress: 1024/2369 symbols completed. HKD took 28.6s. Estimated remaining: 2h 22m 29s. Rate: 9.4 symbols/min [2025-09-16T18:48:31.578985] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-05. [2025-09-16T18:48:31.580596] [INFO ] [MAIN ] HLIO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:48:31.603010] [DEBUG] [LOCAL ] HLIO: Loaded 21669 existing records from primary storage. [2025-09-16T18:48:31.603195] [INFO ] [MAIN ] HLIO: Loaded 21669 existing records. [2025-09-16T18:48:31.794213] [WARN ] [MAIN ] HLIO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:48:31.794543] [INFO ] [MAIN ] HLIO: Wiping existing primary data before full refresh. [2025-09-16T18:48:31.794596] [INFO ] [LOCAL ] HLIO: Starting deletion of primary data. [2025-09-16T18:48:31.795435] [DEBUG] [LOCAL ] HLIO: Deleted primary data file. [2025-09-16T18:48:31.795486] [INFO ] [MAIN ] HLIO: Starting full primary refresh. [2025-09-16T18:48:32.150977] [INFO ] [REMOTE ] HLIO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:48:32.374871] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-02. [2025-09-16T18:48:32.507414] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T18:48:33.174861] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-08. [2025-09-16T18:48:33.269001] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-09-16T18:48:33.290044] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-09-16T18:48:33.974886] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-06. [2025-09-16T18:48:34.144123] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-09-16T18:48:34.774893] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-09. [2025-09-16T18:48:35.574857] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-03. [2025-09-16T18:48:35.954520] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-09-16T18:48:36.006028] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-09-16T18:48:36.009696] [INFO ] [REMOTE ] HLI: Full fetch complete (34060 records). [2025-09-16T18:48:36.009773] [DEBUG] [VALIDATOR ] HLI: Validating primary data. [2025-09-16T18:48:36.384950] [DEBUG] [VALIDATOR ] HLI: All expected trading days are present. [2025-09-16T18:48:36.383007] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-08. [2025-09-16T18:48:36.385061] [DEBUG] [VALIDATOR ] HLI: Primary data validation passed. [2025-09-16T18:48:36.386397] [INFO ] [MAIN ] HLI: Fetched primary data is valid. [2025-09-16T18:48:36.386450] [DEBUG] [LOCAL ] HLI: Saving primary data. [2025-09-16T18:48:36.387301] [DEBUG] [LOCAL ] HLI: Trimmed 34060 records to 22663 (keeping 150 days of data) [2025-09-16T18:48:36.533845] [DEBUG] [LOCAL ] HLI: Primary data saved (22663 records). [2025-09-16T18:48:36.534001] [INFO ] [MAIN ] HLI: Full history saved. [2025-09-16T18:48:36.534101] [INFO ] [MAIN ] Update Progress: 1025/2369 symbols completed. HLI took 20.0s. Estimated remaining: 2h 22m 21s. Rate: 9.4 symbols/min [2025-09-16T18:48:36.709709] [INFO ] [MAIN ] HLLY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:48:36.727460] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T18:48:36.753378] [DEBUG] [LOCAL ] HLLY: Loaded 24451 existing records from primary storage. [2025-09-16T18:48:36.753526] [INFO ] [MAIN ] HLLY: Loaded 24451 existing records. [2025-09-16T18:48:36.936565] [WARN ] [MAIN ] HLLY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:48:36.936868] [INFO ] [MAIN ] HLLY: Wiping existing primary data before full refresh. [2025-09-16T18:48:36.936922] [INFO ] [LOCAL ] HLLY: Starting deletion of primary data. [2025-09-16T18:48:36.937445] [DEBUG] [LOCAL ] HLLY: Deleted primary data file. [2025-09-16T18:48:36.937490] [INFO ] [MAIN ] HLLY: Starting full primary refresh. [2025-09-16T18:48:37.185176] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-04. [2025-09-16T18:48:37.301076] [INFO ] [REMOTE ] HLLY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:48:37.974899] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-07. [2025-09-16T18:48:38.200386] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T18:48:38.293854] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-09-16T18:48:38.774909] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-02. [2025-09-16T18:48:39.419914] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T18:48:39.574908] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-09. [2025-09-16T18:48:39.816082] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T18:48:40.374891] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-05. [2025-09-16T18:48:41.074826] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:48:41.080709] [INFO ] [REMOTE ] HLF: Full fetch complete (53449 records). [2025-09-16T18:48:41.080810] [DEBUG] [VALIDATOR ] HLF: Validating primary data. [2025-09-16T18:48:41.180084] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-03. [2025-09-16T18:48:41.463880] [DEBUG] [VALIDATOR ] HLF: All expected trading days are present. [2025-09-16T18:48:41.463989] [DEBUG] [VALIDATOR ] HLF: Primary data validation passed. [2025-09-16T18:48:41.464725] [INFO ] [MAIN ] HLF: Fetched primary data is valid. [2025-09-16T18:48:41.464813] [DEBUG] [LOCAL ] HLF: Saving primary data. [2025-09-16T18:48:41.467726] [DEBUG] [LOCAL ] HLF: Trimmed 53449 records to 36426 (keeping 150 days of data) [2025-09-16T18:48:41.685215] [DEBUG] [LOCAL ] HLF: Primary data saved (36426 records). [2025-09-16T18:48:41.685346] [INFO ] [MAIN ] HLF: Full history saved. [2025-09-16T18:48:41.685445] [INFO ] [MAIN ] Update Progress: 1026/2369 symbols completed. HLF took 26.4s. Estimated remaining: 2h 22m 13s. Rate: 9.4 symbols/min [2025-09-16T18:48:41.851043] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T18:48:41.880073] [INFO ] [MAIN ] HLN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:48:41.918415] [DEBUG] [LOCAL ] HLN: Loaded 39179 existing records from primary storage. [2025-09-16T18:48:41.918563] [INFO ] [MAIN ] HLN: Loaded 39179 existing records. [2025-09-16T18:48:41.984417] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-08. [2025-09-16T18:48:42.123705] [WARN ] [MAIN ] HLN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:48:42.124017] [INFO ] [MAIN ] HLN: Wiping existing primary data before full refresh. [2025-09-16T18:48:42.124093] [INFO ] [LOCAL ] HLN: Starting deletion of primary data. [2025-09-16T18:48:42.125011] [DEBUG] [LOCAL ] HLN: Deleted primary data file. [2025-09-16T18:48:42.125080] [INFO ] [MAIN ] HLN: Starting full primary refresh. [2025-09-16T18:48:42.481263] [INFO ] [REMOTE ] HLN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:48:42.542081] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T18:48:42.774974] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-04. [2025-09-16T18:48:43.574926] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-02. [2025-09-16T18:48:43.842839] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-09-16T18:48:43.885531] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-09-16T18:48:44.374941] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-05. [2025-09-16T18:48:45.174858] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-09. [2025-09-16T18:48:45.209768] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T18:48:45.686501] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T18:48:45.974901] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-03. [2025-09-16T18:48:46.774901] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-06. [2025-09-16T18:48:47.295935] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-09 (2.12 seconds). [2025-09-16T18:48:47.302910] [INFO ] [REMOTE ] HL: Full fetch complete (58349 records). [2025-09-16T18:48:47.303016] [DEBUG] [VALIDATOR ] HL: Validating primary data. [2025-09-16T18:48:47.580037] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-06. [2025-09-16T18:48:47.694020] [DEBUG] [VALIDATOR ] HL: All expected trading days are present. [2025-09-16T18:48:47.694177] [DEBUG] [VALIDATOR ] HL: Primary data validation passed. [2025-09-16T18:48:47.694983] [INFO ] [MAIN ] HL: Fetched primary data is valid. [2025-09-16T18:48:47.695051] [DEBUG] [LOCAL ] HL: Saving primary data. [2025-09-16T18:48:47.696459] [DEBUG] [LOCAL ] HL: Trimmed 58349 records to 40027 (keeping 150 days of data) [2025-09-16T18:48:47.830173] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-09-16T18:48:47.988194] [DEBUG] [LOCAL ] HL: Primary data saved (40027 records). [2025-09-16T18:48:47.988323] [INFO ] [MAIN ] HL: Full history saved. [2025-09-16T18:48:47.989112] [INFO ] [MAIN ] Update Progress: 1027/2369 symbols completed. HL took 39.1s. Estimated remaining: 2h 22m 6s. Rate: 9.4 symbols/min [2025-09-16T18:48:48.165714] [INFO ] [MAIN ] HLT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:48:48.187284] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T18:48:48.227390] [DEBUG] [LOCAL ] HLT: Loaded 37122 existing records from primary storage. [2025-09-16T18:48:48.227559] [INFO ] [MAIN ] HLT: Loaded 37122 existing records. [2025-09-16T18:48:48.380152] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-07. [2025-09-16T18:48:48.406878] [WARN ] [MAIN ] HLT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:48:48.407736] [INFO ] [MAIN ] HLT: Wiping existing primary data before full refresh. [2025-09-16T18:48:48.407787] [INFO ] [LOCAL ] HLT: Starting deletion of primary data. [2025-09-16T18:48:48.408675] [DEBUG] [LOCAL ] HLT: Deleted primary data file. [2025-09-16T18:48:48.408730] [INFO ] [MAIN ] HLT: Starting full primary refresh. [2025-09-16T18:48:48.787344] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T18:48:48.786405] [INFO ] [REMOTE ] HLT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:48:49.174903] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-02. [2025-09-16T18:48:49.815332] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T18:48:49.974941] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-04. [2025-09-16T18:48:50.774946] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-08. [2025-09-16T18:48:50.943095] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-09-16T18:48:51.574926] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-07. [2025-09-16T18:48:51.670558] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-09-16T18:48:52.046626] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-09-16T18:48:52.374960] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-05. [2025-09-16T18:48:52.774300] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T18:48:53.174948] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-09. [2025-09-16T18:48:53.974859] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-03. [2025-09-16T18:48:54.169923] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T18:48:54.279962] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T18:48:54.282962] [INFO ] [REMOTE ] HLIO: Full fetch complete (29729 records). [2025-09-16T18:48:54.283047] [DEBUG] [VALIDATOR ] HLIO: Validating primary data. [2025-09-16T18:48:54.658754] [DEBUG] [VALIDATOR ] HLIO: All expected trading days are present. [2025-09-16T18:48:54.658834] [DEBUG] [VALIDATOR ] HLIO: Primary data validation passed. [2025-09-16T18:48:54.659463] [INFO ] [MAIN ] HLIO: Fetched primary data is valid. [2025-09-16T18:48:54.659504] [DEBUG] [LOCAL ] HLIO: Saving primary data. [2025-09-16T18:48:54.660755] [DEBUG] [LOCAL ] HLIO: Trimmed 29729 records to 21798 (keeping 150 days of data) [2025-09-16T18:48:54.785416] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-08. [2025-09-16T18:48:54.797067] [DEBUG] [LOCAL ] HLIO: Primary data saved (21798 records). [2025-09-16T18:48:54.797152] [INFO ] [MAIN ] HLIO: Full history saved. [2025-09-16T18:48:54.797247] [INFO ] [MAIN ] Update Progress: 1028/2369 symbols completed. HLIO took 23.4s. Estimated remaining: 2h 22m 0s. Rate: 9.4 symbols/min [2025-09-16T18:48:54.975425] [INFO ] [MAIN ] HLX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:48:55.011484] [DEBUG] [LOCAL ] HLX: Loaded 35148 existing records from primary storage. [2025-09-16T18:48:55.011631] [INFO ] [MAIN ] HLX: Loaded 35148 existing records. [2025-09-16T18:48:55.191188] [WARN ] [MAIN ] HLX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:48:55.191515] [INFO ] [MAIN ] HLX: Wiping existing primary data before full refresh. [2025-09-16T18:48:55.191578] [INFO ] [LOCAL ] HLX: Starting deletion of primary data. [2025-09-16T18:48:55.192402] [DEBUG] [LOCAL ] HLX: Deleted primary data file. [2025-09-16T18:48:55.192461] [INFO ] [MAIN ] HLX: Starting full primary refresh. [2025-09-16T18:48:55.578566] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-06. [2025-09-16T18:48:55.581913] [INFO ] [REMOTE ] HLX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:48:55.845743] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-09-16T18:48:56.000204] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-09-16T18:48:56.374930] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-04. [2025-09-16T18:48:57.174939] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-02. [2025-09-16T18:48:57.382561] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-09-16T18:48:57.974943] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-07. [2025-09-16T18:48:58.325250] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-09-16T18:48:58.738454] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T18:48:58.774854] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-03. [2025-09-16T18:48:59.574894] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-09. [2025-09-16T18:48:59.852062] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T18:48:59.972859] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T18:49:00.374927] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-05. [2025-09-16T18:49:01.146806] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T18:49:01.151562] [INFO ] [REMOTE ] HLLY: Full fetch complete (35039 records). [2025-09-16T18:49:01.151688] [DEBUG] [VALIDATOR ] HLLY: Validating primary data. [2025-09-16T18:49:01.182673] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-08. [2025-09-16T18:49:01.521703] [DEBUG] [VALIDATOR ] HLLY: All expected trading days are present. [2025-09-16T18:49:01.521789] [DEBUG] [VALIDATOR ] HLLY: Primary data validation passed. [2025-09-16T18:49:01.522394] [INFO ] [MAIN ] HLLY: Fetched primary data is valid. [2025-09-16T18:49:01.522437] [DEBUG] [LOCAL ] HLLY: Saving primary data. [2025-09-16T18:49:01.523363] [DEBUG] [LOCAL ] HLLY: Trimmed 35039 records to 24752 (keeping 150 days of data) [2025-09-16T18:49:01.683229] [DEBUG] [LOCAL ] HLLY: Primary data saved (24752 records). [2025-09-16T18:49:01.683380] [INFO ] [MAIN ] HLLY: Full history saved. [2025-09-16T18:49:01.683479] [INFO ] [MAIN ] Update Progress: 1029/2369 symbols completed. HLLY took 25.1s. Estimated remaining: 2h 21m 55s. Rate: 9.4 symbols/min [2025-09-16T18:49:01.856831] [INFO ] [MAIN ] HMC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:49:01.906266] [DEBUG] [LOCAL ] HMC: Loaded 33997 existing records from primary storage. [2025-09-16T18:49:01.906475] [INFO ] [MAIN ] HMC: Loaded 33997 existing records. [2025-09-16T18:49:01.980758] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-04. [2025-09-16T18:49:02.101389] [WARN ] [MAIN ] HMC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:49:02.101659] [INFO ] [MAIN ] HMC: Wiping existing primary data before full refresh. [2025-09-16T18:49:02.101710] [INFO ] [LOCAL ] HMC: Starting deletion of primary data. [2025-09-16T18:49:02.102447] [DEBUG] [LOCAL ] HMC: Deleted primary data file. [2025-09-16T18:49:02.102494] [INFO ] [MAIN ] HMC: Starting full primary refresh. [2025-09-16T18:49:02.460555] [INFO ] [REMOTE ] HMC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:49:02.513826] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-09-16T18:49:02.774921] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-02. [2025-09-16T18:49:03.044378] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-09-16T18:49:03.574976] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-06. [2025-09-16T18:49:03.752246] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T18:49:04.374941] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-05. [2025-09-16T18:49:04.528932] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T18:49:05.174907] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-03. [2025-09-16T18:49:05.368242] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-09-16T18:49:05.945874] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T18:49:05.974855] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-06. [2025-09-16T18:49:06.777239] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-09. [2025-09-16T18:49:06.956745] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T18:49:07.497292] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T18:49:07.574922] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-04. [2025-09-16T18:49:08.374920] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-07. [2025-09-16T18:49:08.569550] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-09-16T18:49:08.576256] [INFO ] [REMOTE ] HLN: Full fetch complete (57684 records). [2025-09-16T18:49:08.576374] [DEBUG] [VALIDATOR ] HLN: Validating primary data. [2025-09-16T18:49:08.960233] [DEBUG] [VALIDATOR ] HLN: All expected trading days are present. [2025-09-16T18:49:08.960325] [DEBUG] [VALIDATOR ] HLN: Primary data validation passed. [2025-09-16T18:49:08.961122] [INFO ] [MAIN ] HLN: Fetched primary data is valid. [2025-09-16T18:49:08.961215] [DEBUG] [LOCAL ] HLN: Saving primary data. [2025-09-16T18:49:08.962507] [DEBUG] [LOCAL ] HLN: Trimmed 57684 records to 39568 (keeping 150 days of data) [2025-09-16T18:49:09.186604] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-07. [2025-09-16T18:49:09.220131] [DEBUG] [LOCAL ] HLN: Primary data saved (39568 records). [2025-09-16T18:49:09.220286] [INFO ] [MAIN ] HLN: Full history saved. [2025-09-16T18:49:09.220376] [INFO ] [MAIN ] Update Progress: 1030/2369 symbols completed. HLN took 27.5s. Estimated remaining: 2h 21m 50s. Rate: 9.4 symbols/min [2025-09-16T18:49:09.395157] [INFO ] [MAIN ] HMN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:49:09.427458] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-09-16T18:49:09.450433] [DEBUG] [LOCAL ] HMN: Loaded 17769 existing records from primary storage. [2025-09-16T18:49:09.450579] [INFO ] [MAIN ] HMN: Loaded 17769 existing records. [2025-09-16T18:49:09.632191] [WARN ] [MAIN ] HMN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:49:09.632426] [INFO ] [MAIN ] HMN: Wiping existing primary data before full refresh. [2025-09-16T18:49:09.632471] [INFO ] [LOCAL ] HMN: Starting deletion of primary data. [2025-09-16T18:49:09.632962] [DEBUG] [LOCAL ] HMN: Deleted primary data file. [2025-09-16T18:49:09.633011] [INFO ] [MAIN ] HMN: Starting full primary refresh. [2025-09-16T18:49:09.988700] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-05. [2025-09-16T18:49:10.024035] [INFO ] [REMOTE ] HMN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:49:10.108128] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T18:49:10.497072] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T18:49:10.774913] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-08. [2025-09-16T18:49:11.562535] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T18:49:11.574835] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-06. [2025-09-16T18:49:12.253892] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T18:49:12.374957] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-08. [2025-09-16T18:49:13.158971] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T18:49:13.174838] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-07. [2025-09-16T18:49:13.974909] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-09. [2025-09-16T18:49:14.082925] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-09-16T18:49:14.714622] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-09-16T18:49:14.774864] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-08. [2025-09-16T18:49:15.574907] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-09. [2025-09-16T18:49:15.628287] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-09-16T18:49:15.637093] [INFO ] [REMOTE ] HLX: Full fetch complete (52284 records). [2025-09-16T18:49:15.637262] [DEBUG] [VALIDATOR ] HLX: Validating primary data. [2025-09-16T18:49:16.023965] [DEBUG] [VALIDATOR ] HLX: All expected trading days are present. [2025-09-16T18:49:16.024051] [DEBUG] [VALIDATOR ] HLX: Primary data validation passed. [2025-09-16T18:49:16.024846] [INFO ] [MAIN ] HLX: Fetched primary data is valid. [2025-09-16T18:49:16.024896] [DEBUG] [LOCAL ] HLX: Saving primary data. [2025-09-16T18:49:16.025967] [DEBUG] [LOCAL ] HLX: Trimmed 52284 records to 35463 (keeping 150 days of data) [2025-09-16T18:49:16.225585] [DEBUG] [LOCAL ] HLX: Primary data saved (35463 records). [2025-09-16T18:49:16.225757] [INFO ] [MAIN ] HLX: Full history saved. [2025-09-16T18:49:16.225842] [INFO ] [MAIN ] Update Progress: 1031/2369 symbols completed. HLX took 21.4s. Estimated remaining: 2h 21m 44s. Rate: 9.4 symbols/min [2025-09-16T18:49:16.380016] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-02. [2025-09-16T18:49:16.402052] [INFO ] [MAIN ] HMY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:49:16.434567] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T18:49:16.479699] [DEBUG] [LOCAL ] HMY: Loaded 39146 existing records from primary storage. [2025-09-16T18:49:16.479865] [INFO ] [MAIN ] HMY: Loaded 39146 existing records. [2025-09-16T18:49:16.664291] [WARN ] [MAIN ] HMY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:49:16.664531] [INFO ] [MAIN ] HMY: Wiping existing primary data before full refresh. [2025-09-16T18:49:16.664578] [INFO ] [LOCAL ] HMY: Starting deletion of primary data. [2025-09-16T18:49:16.665377] [DEBUG] [LOCAL ] HMY: Deleted primary data file. [2025-09-16T18:49:16.665426] [INFO ] [MAIN ] HMY: Starting full primary refresh. [2025-09-16T18:49:17.032164] [INFO ] [REMOTE ] HMY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:49:17.050631] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T18:49:17.057192] [INFO ] [REMOTE ] HLT: Full fetch complete (55221 records). [2025-09-16T18:49:17.057348] [DEBUG] [VALIDATOR ] HLT: Validating primary data. [2025-09-16T18:49:17.176039] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-02. [2025-09-16T18:49:17.366705] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-09-16T18:49:17.457561] [DEBUG] [VALIDATOR ] HLT: All expected trading days are present. [2025-09-16T18:49:17.457661] [DEBUG] [VALIDATOR ] HLT: Primary data validation passed. [2025-09-16T18:49:17.458425] [INFO ] [MAIN ] HLT: Fetched primary data is valid. [2025-09-16T18:49:17.458469] [DEBUG] [LOCAL ] HLT: Saving primary data. [2025-09-16T18:49:17.459747] [DEBUG] [LOCAL ] HLT: Trimmed 55221 records to 37473 (keeping 150 days of data) [2025-09-16T18:49:17.696408] [DEBUG] [LOCAL ] HLT: Primary data saved (37473 records). [2025-09-16T18:49:17.696581] [INFO ] [MAIN ] HLT: Full history saved. [2025-09-16T18:49:17.697668] [INFO ] [MAIN ] Update Progress: 1032/2369 symbols completed. HLT took 29.7s. Estimated remaining: 2h 21m 32s. Rate: 9.4 symbols/min [2025-09-16T18:49:17.864548] [INFO ] [MAIN ] HNGE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:49:17.864789] [DEBUG] [LOCAL ] HNGE: No existing data file found in primary storage. [2025-09-16T18:49:17.864829] [WARN ] [MAIN ] HNGE: No existing data found, performing full refresh instead. [2025-09-16T18:49:17.864950] [INFO ] [MAIN ] HNGE: Wiping existing primary data before full refresh. [2025-09-16T18:49:17.864990] [INFO ] [LOCAL ] HNGE: Starting deletion of primary data. [2025-09-16T18:49:17.865032] [DEBUG] [LOCAL ] HNGE: No primary data file found to delete. [2025-09-16T18:49:17.865065] [INFO ] [MAIN ] HNGE: Starting full primary refresh. [2025-09-16T18:49:17.980031] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-09. [2025-09-16T18:49:18.224741] [INFO ] [REMOTE ] HNGE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:49:18.774931] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-02. [2025-09-16T18:49:18.912177] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T18:49:18.970217] [DEBUG] [REMOTE ] HNGE: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:49:18.970577] [WARN ] [REMOTE ] HNGE: No data returned for month 2025-02 [2025-09-16T18:49:19.451800] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T18:49:19.457318] [INFO ] [REMOTE ] HMC: Full fetch complete (51289 records). [2025-09-16T18:49:19.457440] [DEBUG] [VALIDATOR ] HMC: Validating primary data. [2025-09-16T18:49:19.580011] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-03. [2025-09-16T18:49:19.840972] [DEBUG] [VALIDATOR ] HMC: All expected trading days are present. [2025-09-16T18:49:19.841057] [DEBUG] [VALIDATOR ] HMC: Primary data validation passed. [2025-09-16T18:49:19.841815] [INFO ] [MAIN ] HMC: Fetched primary data is valid. [2025-09-16T18:49:19.841866] [DEBUG] [LOCAL ] HMC: Saving primary data. [2025-09-16T18:49:19.843045] [DEBUG] [LOCAL ] HMC: Trimmed 51289 records to 34329 (keeping 150 days of data) [2025-09-16T18:49:19.894715] [DEBUG] [REMOTE ] HNGE: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:49:19.903782] [WARN ] [REMOTE ] HNGE: No data returned for month 2025-03 [2025-09-16T18:49:20.045548] [DEBUG] [LOCAL ] HMC: Primary data saved (34329 records). [2025-09-16T18:49:20.045715] [INFO ] [MAIN ] HMC: Full history saved. [2025-09-16T18:49:20.045815] [INFO ] [MAIN ] Update Progress: 1033/2369 symbols completed. HMC took 18.4s. Estimated remaining: 2h 21m 20s. Rate: 9.5 symbols/min [2025-09-16T18:49:20.213567] [INFO ] [MAIN ] HNI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:49:20.233523] [DEBUG] [LOCAL ] HNI: Loaded 21203 existing records from primary storage. [2025-09-16T18:49:20.233667] [INFO ] [MAIN ] HNI: Loaded 21203 existing records. [2025-09-16T18:49:20.385003] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-03. [2025-09-16T18:49:20.413479] [WARN ] [MAIN ] HNI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:49:20.413730] [INFO ] [MAIN ] HNI: Wiping existing primary data before full refresh. [2025-09-16T18:49:20.413786] [INFO ] [LOCAL ] HNI: Starting deletion of primary data. [2025-09-16T18:49:20.414389] [DEBUG] [LOCAL ] HNI: Deleted primary data file. [2025-09-16T18:49:20.414438] [INFO ] [MAIN ] HNI: Starting full primary refresh. [2025-09-16T18:49:20.766947] [INFO ] [REMOTE ] HNI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:49:21.174891] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-02. [2025-09-16T18:49:21.359872] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-09-16T18:49:21.974902] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-04. [2025-09-16T18:49:22.160811] [DEBUG] [REMOTE ] HNGE: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T18:49:22.161122] [WARN ] [REMOTE ] HNGE: No data returned for month 2025-04 [2025-09-16T18:49:22.354424] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-09-16T18:49:22.774972] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-03. [2025-09-16T18:49:23.574923] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-03. [2025-09-16T18:49:24.066948] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-09-16T18:49:24.374908] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-05. [2025-09-16T18:49:25.174901] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-04. [2025-09-16T18:49:25.250133] [INFO ] [REMOTE ] HNGE: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-09-16T18:49:25.579366] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-09-16T18:49:25.981448] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-06. [2025-09-16T18:49:25.985663] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-09-16T18:49:26.774907] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-05. [2025-09-16T18:49:27.573911] [INFO ] [REMOTE ] HNGE: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T18:49:27.582442] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-04. [2025-09-16T18:49:28.374939] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-07. [2025-09-16T18:49:28.411123] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T18:49:28.564636] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-09-16T18:49:29.174942] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-06. [2025-09-16T18:49:29.877539] [INFO ] [REMOTE ] HNGE: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T18:49:29.974952] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-08. [2025-09-16T18:49:30.450196] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-09-16T18:49:30.774915] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-04. [2025-09-16T18:49:31.516127] [INFO ] [REMOTE ] HNGE: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T18:49:31.574883] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-09. [2025-09-16T18:49:32.374942] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-07. [2025-09-16T18:49:32.418682] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T18:49:33.174899] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-05. [2025-09-16T18:49:33.283284] [INFO ] [REMOTE ] HNGE: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T18:49:33.288633] [INFO ] [REMOTE ] HNGE: Full fetch complete (24280 records). [2025-09-16T18:49:33.288733] [DEBUG] [VALIDATOR ] HNGE: Validating primary data. [2025-09-16T18:49:33.469336] [WARN ] [VALIDATOR ] HNGE: Validation failed - insufficient historical data. Have 80 unique days, need 150 days [2025-09-16T18:49:33.469750] [FATAL] [MAIN ] HNGE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:49:33.469803] [WARN ] [MAIN ] HNGE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:49:33.469847] [INFO ] [LOCAL ] HNGE: Starting deletion of primary data. [2025-09-16T18:49:33.469903] [DEBUG] [LOCAL ] HNGE: No primary data file found to delete. [2025-09-16T18:49:33.469938] [INFO ] [LOCAL ] HNGE: Starting deletion of derivative data. [2025-09-16T18:49:33.469986] [DEBUG] [LOCAL ] HNGE: No derivative data file found to delete. [2025-09-16T18:49:33.470047] [INFO ] [MAIN ] Update Progress: 1034/2369 symbols completed. HNGE took 15.8s. Estimated remaining: 2h 21m 23s. Rate: 9.4 symbols/min [2025-09-16T18:49:33.643182] [INFO ] [MAIN ] HOG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:49:33.657960] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-09-16T18:49:33.699034] [DEBUG] [LOCAL ] HOG: Loaded 37469 existing records from primary storage. [2025-09-16T18:49:33.699183] [INFO ] [MAIN ] HOG: Loaded 37469 existing records. [2025-09-16T18:49:33.883544] [WARN ] [MAIN ] HOG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:49:33.883818] [INFO ] [MAIN ] HOG: Wiping existing primary data before full refresh. [2025-09-16T18:49:33.883868] [INFO ] [LOCAL ] HOG: Starting deletion of primary data. [2025-09-16T18:49:33.885370] [DEBUG] [LOCAL ] HOG: Deleted primary data file. [2025-09-16T18:49:33.885598] [INFO ] [MAIN ] HOG: Starting full primary refresh. [2025-09-16T18:49:33.983188] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-05. [2025-09-16T18:49:34.252488] [INFO ] [REMOTE ] HOG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:49:34.422591] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T18:49:34.774900] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-06. [2025-09-16T18:49:35.574868] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-08. [2025-09-16T18:49:35.926975] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-09-16T18:49:36.117528] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-09-16T18:49:36.374912] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-07. [2025-09-16T18:49:36.865823] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T18:49:37.174888] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-02. [2025-09-16T18:49:37.523144] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T18:49:37.974899] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-08. [2025-09-16T18:49:38.774914] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-09. [2025-09-16T18:49:38.941494] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-09-16T18:49:38.987855] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-09-16T18:49:39.574896] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-03. [2025-09-16T18:49:40.309757] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-09-16T18:49:40.314512] [INFO ] [REMOTE ] HNI: Full fetch complete (31310 records). [2025-09-16T18:49:40.314614] [DEBUG] [VALIDATOR ] HNI: Validating primary data. [2025-09-16T18:49:40.382408] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-06. [2025-09-16T18:49:40.691547] [DEBUG] [VALIDATOR ] HNI: All expected trading days are present. [2025-09-16T18:49:40.691626] [DEBUG] [VALIDATOR ] HNI: Primary data validation passed. [2025-09-16T18:49:40.692217] [INFO ] [MAIN ] HNI: Fetched primary data is valid. [2025-09-16T18:49:40.692259] [DEBUG] [LOCAL ] HNI: Saving primary data. [2025-09-16T18:49:40.693040] [DEBUG] [LOCAL ] HNI: Trimmed 31310 records to 21452 (keeping 150 days of data) [2025-09-16T18:49:40.823579] [DEBUG] [LOCAL ] HNI: Primary data saved (21452 records). [2025-09-16T18:49:40.823723] [INFO ] [MAIN ] HNI: Full history saved. [2025-09-16T18:49:40.823804] [INFO ] [MAIN ] Update Progress: 1035/2369 symbols completed. HNI took 20.8s. Estimated remaining: 2h 21m 18s. Rate: 9.4 symbols/min [2025-09-16T18:49:40.994693] [INFO ] [MAIN ] HOMB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:49:41.024562] [DEBUG] [LOCAL ] HOMB: Loaded 31393 existing records from primary storage. [2025-09-16T18:49:41.024721] [INFO ] [MAIN ] HOMB: Loaded 31393 existing records. [2025-09-16T18:49:41.177589] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-09. [2025-09-16T18:49:41.210386] [WARN ] [MAIN ] HOMB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:49:41.211720] [INFO ] [MAIN ] HOMB: Wiping existing primary data before full refresh. [2025-09-16T18:49:41.211773] [INFO ] [LOCAL ] HOMB: Starting deletion of primary data. [2025-09-16T18:49:41.212775] [DEBUG] [LOCAL ] HOMB: Deleted primary data file. [2025-09-16T18:49:41.212829] [INFO ] [MAIN ] HOMB: Starting full primary refresh. [2025-09-16T18:49:41.539389] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-09-16T18:49:41.616997] [INFO ] [REMOTE ] HOMB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:49:41.974941] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-02. [2025-09-16T18:49:42.431991] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-06 (2.05 seconds). [2025-09-16T18:49:42.451125] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T18:49:42.454107] [INFO ] [REMOTE ] HMN: Full fetch complete (24885 records). [2025-09-16T18:49:42.454225] [DEBUG] [VALIDATOR ] HMN: Validating primary data. [2025-09-16T18:49:42.780068] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-07. [2025-09-16T18:49:42.821286] [DEBUG] [VALIDATOR ] HMN: All expected trading days are present. [2025-09-16T18:49:42.821367] [DEBUG] [VALIDATOR ] HMN: Primary data validation passed. [2025-09-16T18:49:42.821890] [INFO ] [MAIN ] HMN: Fetched primary data is valid. [2025-09-16T18:49:42.821933] [DEBUG] [LOCAL ] HMN: Saving primary data. [2025-09-16T18:49:42.822550] [DEBUG] [LOCAL ] HMN: Trimmed 24885 records to 17941 (keeping 150 days of data) [2025-09-16T18:49:42.932083] [DEBUG] [LOCAL ] HMN: Primary data saved (17941 records). [2025-09-16T18:49:42.932229] [INFO ] [MAIN ] HMN: Full history saved. [2025-09-16T18:49:42.932328] [INFO ] [MAIN ] Update Progress: 1036/2369 symbols completed. HMN took 33.7s. Estimated remaining: 2h 21m 6s. Rate: 9.4 symbols/min [2025-09-16T18:49:43.102521] [INFO ] [MAIN ] HOUS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:49:43.132319] [DEBUG] [LOCAL ] HOUS: Loaded 32545 existing records from primary storage. [2025-09-16T18:49:43.132486] [INFO ] [MAIN ] HOUS: Loaded 32545 existing records. [2025-09-16T18:49:43.313772] [WARN ] [MAIN ] HOUS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:49:43.314067] [INFO ] [MAIN ] HOUS: Wiping existing primary data before full refresh. [2025-09-16T18:49:43.314140] [INFO ] [LOCAL ] HOUS: Starting deletion of primary data. [2025-09-16T18:49:43.314832] [DEBUG] [LOCAL ] HOUS: Deleted primary data file. [2025-09-16T18:49:43.314894] [INFO ] [MAIN ] HOUS: Starting full primary refresh. [2025-09-16T18:49:43.583697] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-04. [2025-09-16T18:49:43.678272] [INFO ] [REMOTE ] HOUS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:49:43.702115] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T18:49:44.374906] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-02. [2025-09-16T18:49:45.009093] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-09-16T18:49:45.174925] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-03. [2025-09-16T18:49:45.337890] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-09-16T18:49:45.753075] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T18:49:45.974914] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-03. [2025-09-16T18:49:46.774918] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-05. [2025-09-16T18:49:46.842603] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T18:49:47.217855] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-09-16T18:49:47.574966] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-04. [2025-09-16T18:49:48.374918] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-08. [2025-09-16T18:49:48.543261] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-09-16T18:49:49.032869] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-09-16T18:49:49.174923] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-06. [2025-09-16T18:49:49.974873] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-04. [2025-09-16T18:49:50.069700] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T18:49:50.777073] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-05. [2025-09-16T18:49:50.780563] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-09-16T18:49:51.212230] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-09-16T18:49:51.574911] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-09. [2025-09-16T18:49:52.299040] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T18:49:52.374916] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-05. [2025-09-16T18:49:53.174934] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-06. [2025-09-16T18:49:53.649955] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-09-16T18:49:53.656515] [INFO ] [REMOTE ] HMY: Full fetch complete (57772 records). [2025-09-16T18:49:53.656712] [DEBUG] [VALIDATOR ] HMY: Validating primary data. [2025-09-16T18:49:53.995369] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-07. [2025-09-16T18:49:54.046450] [DEBUG] [VALIDATOR ] HMY: All expected trading days are present. [2025-09-16T18:49:54.046687] [DEBUG] [VALIDATOR ] HMY: Primary data validation passed. [2025-09-16T18:49:54.047459] [INFO ] [MAIN ] HMY: Fetched primary data is valid. [2025-09-16T18:49:54.055554] [DEBUG] [LOCAL ] HMY: Saving primary data. [2025-09-16T18:49:54.071781] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T18:49:54.081244] [DEBUG] [LOCAL ] HMY: Trimmed 57772 records to 39534 (keeping 150 days of data) [2025-09-16T18:49:54.326277] [DEBUG] [LOCAL ] HMY: Primary data saved (39534 records). [2025-09-16T18:49:54.326435] [INFO ] [MAIN ] HMY: Full history saved. [2025-09-16T18:49:54.326526] [INFO ] [MAIN ] Update Progress: 1037/2369 symbols completed. HMY took 38.1s. Estimated remaining: 2h 21m 6s. Rate: 9.4 symbols/min [2025-09-16T18:49:54.435905] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T18:49:54.774906] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-06. [2025-09-16T18:49:55.574924] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-07. [2025-09-16T18:49:55.935237] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-09-16T18:49:56.200435] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T18:49:56.374908] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-08. [2025-09-16T18:49:57.075404] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T18:49:57.174952] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-08. [2025-09-16T18:49:57.865420] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T18:49:57.974924] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-09. [2025-09-16T18:49:58.737736] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T18:49:58.774850] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-09. [2025-09-16T18:49:59.574912] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-07. [2025-09-16T18:49:59.612023] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T18:49:59.618146] [INFO ] [REMOTE ] HOG: Full fetch complete (55905 records). [2025-09-16T18:49:59.618279] [DEBUG] [VALIDATOR ] HOG: Validating primary data. [2025-09-16T18:50:00.007290] [DEBUG] [VALIDATOR ] HOG: All expected trading days are present. [2025-09-16T18:50:00.007389] [DEBUG] [VALIDATOR ] HOG: Primary data validation passed. [2025-09-16T18:50:00.008191] [INFO ] [MAIN ] HOG: Fetched primary data is valid. [2025-09-16T18:50:00.008255] [DEBUG] [LOCAL ] HOG: Saving primary data. [2025-09-16T18:50:00.009689] [DEBUG] [LOCAL ] HOG: Trimmed 55905 records to 37825 (keeping 150 days of data) [2025-09-16T18:50:00.255924] [DEBUG] [LOCAL ] HOG: Primary data saved (37825 records). [2025-09-16T18:50:00.256195] [INFO ] [MAIN ] HOG: Full history saved. [2025-09-16T18:50:00.257030] [INFO ] [MAIN ] Update Progress: 1038/2369 symbols completed. HOG took 26.8s. Estimated remaining: 2h 20m 59s. Rate: 9.4 symbols/min [2025-09-16T18:50:00.277439] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T18:50:00.283862] [INFO ] [REMOTE ] HOUS: Full fetch complete (47615 records). [2025-09-16T18:50:00.284131] [DEBUG] [VALIDATOR ] HOUS: Validating primary data. [2025-09-16T18:50:00.661041] [DEBUG] [VALIDATOR ] HOUS: All expected trading days are present. [2025-09-16T18:50:00.661127] [DEBUG] [VALIDATOR ] HOUS: Primary data validation passed. [2025-09-16T18:50:00.661886] [INFO ] [MAIN ] HOUS: Fetched primary data is valid. [2025-09-16T18:50:00.661931] [DEBUG] [LOCAL ] HOUS: Saving primary data. [2025-09-16T18:50:00.662956] [DEBUG] [LOCAL ] HOUS: Trimmed 47615 records to 32881 (keeping 150 days of data) [2025-09-16T18:50:00.849069] [DEBUG] [LOCAL ] HOUS: Primary data saved (32881 records). [2025-09-16T18:50:00.849216] [INFO ] [MAIN ] HOUS: Full history saved. [2025-09-16T18:50:00.849304] [INFO ] [MAIN ] Update Progress: 1039/2369 symbols completed. HOUS took 17.9s. Estimated remaining: 2h 20m 46s. Rate: 9.4 symbols/min [2025-09-16T18:50:01.009539] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-09-16T18:50:01.021547] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-08. [2025-09-16T18:50:02.582967] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T18:50:02.592744] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-09. [2025-09-16T18:50:04.136976] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T18:50:04.142545] [INFO ] [REMOTE ] HOMB: Full fetch complete (47566 records). [2025-09-16T18:50:04.142672] [DEBUG] [VALIDATOR ] HOMB: Validating primary data. [2025-09-16T18:50:04.534432] [DEBUG] [VALIDATOR ] HOMB: All expected trading days are present. [2025-09-16T18:50:04.534509] [DEBUG] [VALIDATOR ] HOMB: Primary data validation passed. [2025-09-16T18:50:04.535223] [INFO ] [MAIN ] HOMB: Fetched primary data is valid. [2025-09-16T18:50:04.535266] [DEBUG] [LOCAL ] HOMB: Saving primary data. [2025-09-16T18:50:04.536213] [DEBUG] [LOCAL ] HOMB: Trimmed 47566 records to 31713 (keeping 150 days of data) [2025-09-16T18:50:04.730675] [DEBUG] [LOCAL ] HOMB: Primary data saved (31713 records). [2025-09-16T18:50:04.730820] [INFO ] [MAIN ] HOMB: Full history saved. [2025-09-16T18:50:04.730914] [INFO ] [MAIN ] Update Progress: 1040/2369 symbols completed. HOMB took 23.9s. Estimated remaining: 2h 20m 36s. Rate: 9.5 symbols/min [2025-09-16T18:50:05.311069] [INFO ] [MAIN ] HPE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:05.340262] [INFO ] [MAIN ] HPF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:05.419557] [INFO ] [MAIN ] HP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:05.465383] [INFO ] [MAIN ] HOV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:05.472103] [DEBUG] [LOCAL ] HPF: Loaded 5553 existing records from primary storage. [2025-09-16T18:50:05.472369] [INFO ] [MAIN ] HPF: Loaded 5553 existing records. [2025-09-16T18:50:05.587686] [DEBUG] [LOCAL ] HOV: Loaded 10490 existing records from primary storage. [2025-09-16T18:50:05.598672] [INFO ] [MAIN ] HOV: Loaded 10490 existing records. [2025-09-16T18:50:05.737385] [WARN ] [MAIN ] HPF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:05.787749] [INFO ] [MAIN ] HPF: Wiping existing primary data before full refresh. [2025-09-16T18:50:05.798489] [INFO ] [LOCAL ] HPF: Starting deletion of primary data. [2025-09-16T18:50:05.859216] [DEBUG] [LOCAL ] HPF: Deleted primary data file. [2025-09-16T18:50:05.878764] [WARN ] [MAIN ] HOV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:05.879006] [INFO ] [MAIN ] HPF: Starting full primary refresh. [2025-09-16T18:50:05.880290] [INFO ] [MAIN ] HOV: Wiping existing primary data before full refresh. [2025-09-16T18:50:05.880053] [DEBUG] [LOCAL ] HP: Loaded 38311 existing records from primary storage. [2025-09-16T18:50:05.880523] [INFO ] [MAIN ] HP: Loaded 38311 existing records. [2025-09-16T18:50:05.891598] [INFO ] [LOCAL ] HOV: Starting deletion of primary data. [2025-09-16T18:50:05.971625] [DEBUG] [LOCAL ] HOV: Deleted primary data file. [2025-09-16T18:50:05.977467] [DEBUG] [LOCAL ] HPE: Loaded 39657 existing records from primary storage. [2025-09-16T18:50:05.991152] [INFO ] [MAIN ] HOV: Starting full primary refresh. [2025-09-16T18:50:06.021853] [INFO ] [MAIN ] HPE: Loaded 39657 existing records. [2025-09-16T18:50:06.369401] [WARN ] [MAIN ] HP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:06.549173] [INFO ] [MAIN ] HP: Wiping existing primary data before full refresh. [2025-09-16T18:50:06.633632] [INFO ] [LOCAL ] HP: Starting deletion of primary data. [2025-09-16T18:50:06.741891] [DEBUG] [LOCAL ] HP: Deleted primary data file. [2025-09-16T18:50:06.802114] [INFO ] [MAIN ] HP: Starting full primary refresh. [2025-09-16T18:50:06.791786] [WARN ] [MAIN ] HPE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:06.852992] [INFO ] [MAIN ] HPE: Wiping existing primary data before full refresh. [2025-09-16T18:50:06.911615] [INFO ] [LOCAL ] HPE: Starting deletion of primary data. [2025-09-16T18:50:06.953714] [INFO ] [REMOTE ] HPF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:06.982096] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-02. [2025-09-16T18:50:06.981803] [DEBUG] [LOCAL ] HPE: Deleted primary data file. [2025-09-16T18:50:07.030177] [INFO ] [MAIN ] HPE: Starting full primary refresh. [2025-09-16T18:50:07.115376] [INFO ] [REMOTE ] HOV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:07.137194] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-02. [2025-09-16T18:50:07.633346] [INFO ] [REMOTE ] HP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:07.644127] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-02. [2025-09-16T18:50:07.723041] [INFO ] [REMOTE ] HPE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:07.723403] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-02. [2025-09-16T18:50:07.728895] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T18:50:07.730714] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-03. [2025-09-16T18:50:07.994394] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T18:50:07.997192] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-03. [2025-09-16T18:50:08.350211] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-09-16T18:50:08.353563] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-04. [2025-09-16T18:50:08.713373] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-09-16T18:50:08.716382] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-04. [2025-09-16T18:50:09.138244] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T18:50:09.152789] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-03. [2025-09-16T18:50:09.153499] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-09-16T18:50:09.174867] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-05. [2025-09-16T18:50:09.309473] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T18:50:09.654921] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-09-16T18:50:09.693636] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-09-16T18:50:09.974950] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-06. [2025-09-16T18:50:10.774915] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-03. [2025-09-16T18:50:10.868838] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T18:50:10.900000] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-09-16T18:50:11.574986] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-05. [2025-09-16T18:50:12.374899] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-04. [2025-09-16T18:50:12.450134] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-09-16T18:50:12.895113] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-03 (2.12 seconds). [2025-09-16T18:50:13.174950] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-07. [2025-09-16T18:50:13.974915] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-04. [2025-09-16T18:50:14.020176] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-09-16T18:50:14.104714] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T18:50:14.774929] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-06. [2025-09-16T18:50:15.574887] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-08. [2025-09-16T18:50:15.777194] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T18:50:15.993182] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-09-16T18:50:15.992974] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-09-16T18:50:16.374944] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-05. [2025-09-16T18:50:17.174894] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-09. [2025-09-16T18:50:17.974937] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-05. [2025-09-16T18:50:18.268111] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T18:50:18.272962] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-09-16T18:50:18.288456] [INFO ] [REMOTE ] HPF: Full fetch complete (7740 records). [2025-09-16T18:50:18.289267] [DEBUG] [VALIDATOR ] HPF: Validating primary data. [2025-09-16T18:50:18.780341] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-06. [2025-09-16T18:50:18.881950] [DEBUG] [VALIDATOR ] HPF: All expected trading days are present. [2025-09-16T18:50:18.882044] [DEBUG] [VALIDATOR ] HPF: Primary data validation passed. [2025-09-16T18:50:18.882314] [INFO ] [MAIN ] HPF: Fetched primary data is valid. [2025-09-16T18:50:18.882358] [DEBUG] [LOCAL ] HPF: Saving primary data. [2025-09-16T18:50:18.883029] [DEBUG] [LOCAL ] HPF: Trimmed 7740 records to 5625 (keeping 150 days of data) [2025-09-16T18:50:18.929441] [DEBUG] [LOCAL ] HPF: Primary data saved (5625 records). [2025-09-16T18:50:18.929623] [INFO ] [MAIN ] HPF: Full history saved. [2025-09-16T18:50:18.929768] [INFO ] [MAIN ] Update Progress: 1041/2369 symbols completed. HPF took 14.2s. Estimated remaining: 2h 20m 40s. Rate: 9.4 symbols/min [2025-09-16T18:50:19.177327] [INFO ] [MAIN ] HPI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:19.187570] [DEBUG] [LOCAL ] HPI: Loaded 7603 existing records from primary storage. [2025-09-16T18:50:19.187762] [INFO ] [MAIN ] HPI: Loaded 7603 existing records. [2025-09-16T18:50:19.417467] [WARN ] [MAIN ] HPI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:19.417957] [INFO ] [MAIN ] HPI: Wiping existing primary data before full refresh. [2025-09-16T18:50:19.418030] [INFO ] [LOCAL ] HPI: Starting deletion of primary data. [2025-09-16T18:50:19.418687] [DEBUG] [LOCAL ] HPI: Deleted primary data file. [2025-09-16T18:50:19.418809] [INFO ] [MAIN ] HPI: Starting full primary refresh. [2025-09-16T18:50:19.589360] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-07. [2025-09-16T18:50:19.776506] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-09-16T18:50:19.865861] [INFO ] [REMOTE ] HPI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:20.374905] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-06. [2025-09-16T18:50:20.465461] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T18:50:20.679967] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T18:50:21.174917] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-08. [2025-09-16T18:50:21.974895] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-07. [2025-09-16T18:50:22.148506] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-09-16T18:50:22.237607] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-09-16T18:50:22.774920] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-02. [2025-09-16T18:50:23.374735] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-09-16T18:50:23.574911] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-09. [2025-09-16T18:50:23.855448] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-09-16T18:50:24.374942] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-08. [2025-09-16T18:50:24.541380] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T18:50:24.543406] [INFO ] [REMOTE ] HOV: Full fetch complete (14859 records). [2025-09-16T18:50:24.543483] [DEBUG] [VALIDATOR ] HOV: Validating primary data. [2025-09-16T18:50:24.909312] [DEBUG] [VALIDATOR ] HOV: All expected trading days are present. [2025-09-16T18:50:24.909418] [DEBUG] [VALIDATOR ] HOV: Primary data validation passed. [2025-09-16T18:50:24.909969] [INFO ] [MAIN ] HOV: Fetched primary data is valid. [2025-09-16T18:50:24.910039] [DEBUG] [LOCAL ] HOV: Saving primary data. [2025-09-16T18:50:24.910985] [DEBUG] [LOCAL ] HOV: Trimmed 14859 records to 10592 (keeping 150 days of data) [2025-09-16T18:50:24.986357] [DEBUG] [LOCAL ] HOV: Primary data saved (10592 records). [2025-09-16T18:50:24.986513] [INFO ] [MAIN ] HOV: Full history saved. [2025-09-16T18:50:24.986630] [INFO ] [MAIN ] Update Progress: 1042/2369 symbols completed. HOV took 20.3s. Estimated remaining: 2h 20m 33s. Rate: 9.4 symbols/min [2025-09-16T18:50:25.188591] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-03. [2025-09-16T18:50:25.199933] [INFO ] [MAIN ] HPP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:25.200904] [DEBUG] [LOCAL ] HPP: No existing data file found in primary storage. [2025-09-16T18:50:25.200946] [WARN ] [MAIN ] HPP: No existing data found, performing full refresh instead. [2025-09-16T18:50:25.201086] [INFO ] [MAIN ] HPP: Wiping existing primary data before full refresh. [2025-09-16T18:50:25.201127] [INFO ] [LOCAL ] HPP: Starting deletion of primary data. [2025-09-16T18:50:25.201168] [DEBUG] [LOCAL ] HPP: No primary data file found to delete. [2025-09-16T18:50:25.201217] [INFO ] [MAIN ] HPP: Starting full primary refresh. [2025-09-16T18:50:25.555231] [INFO ] [REMOTE ] HPP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:25.838524] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-09-16T18:50:25.974906] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-04. [2025-09-16T18:50:26.307553] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T18:50:26.774936] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-07. [2025-09-16T18:50:26.792928] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-09-16T18:50:27.574908] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-02. [2025-09-16T18:50:28.374920] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-09. [2025-09-16T18:50:28.501212] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T18:50:29.075688] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-09-16T18:50:29.174909] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-08. [2025-09-16T18:50:29.974873] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-05. [2025-09-16T18:50:30.379734] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-09-16T18:50:30.387080] [INFO ] [REMOTE ] HPE: Full fetch complete (58377 records). [2025-09-16T18:50:30.387235] [DEBUG] [VALIDATOR ] HPE: Validating primary data. [2025-09-16T18:50:30.782393] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-03. [2025-09-16T18:50:30.785775] [DEBUG] [VALIDATOR ] HPE: All expected trading days are present. [2025-09-16T18:50:30.790128] [DEBUG] [VALIDATOR ] HPE: Primary data validation passed. [2025-09-16T18:50:30.793477] [INFO ] [MAIN ] HPE: Fetched primary data is valid. [2025-09-16T18:50:30.794333] [DEBUG] [LOCAL ] HPE: Saving primary data. [2025-09-16T18:50:30.794720] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T18:50:30.798302] [DEBUG] [LOCAL ] HPE: Trimmed 58377 records to 40047 (keeping 150 days of data) [2025-09-16T18:50:31.051935] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-09-16T18:50:31.086184] [DEBUG] [LOCAL ] HPE: Primary data saved (40047 records). [2025-09-16T18:50:31.086309] [INFO ] [MAIN ] HPE: Full history saved. [2025-09-16T18:50:31.086423] [INFO ] [MAIN ] Update Progress: 1043/2369 symbols completed. HPE took 26.4s. Estimated remaining: 2h 20m 26s. Rate: 9.4 symbols/min [2025-09-16T18:50:31.297212] [INFO ] [MAIN ] HPQ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:31.334911] [DEBUG] [LOCAL ] HPQ: Loaded 39632 existing records from primary storage. [2025-09-16T18:50:31.335059] [INFO ] [MAIN ] HPQ: Loaded 39632 existing records. [2025-09-16T18:50:31.514778] [WARN ] [MAIN ] HPQ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:31.515065] [INFO ] [MAIN ] HPQ: Wiping existing primary data before full refresh. [2025-09-16T18:50:31.515121] [INFO ] [LOCAL ] HPQ: Starting deletion of primary data. [2025-09-16T18:50:31.516096] [DEBUG] [LOCAL ] HPQ: Deleted primary data file. [2025-09-16T18:50:31.516150] [INFO ] [MAIN ] HPQ: Starting full primary refresh. [2025-09-16T18:50:31.580064] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-06. [2025-09-16T18:50:31.880563] [INFO ] [REMOTE ] HPQ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:32.380001] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-09. [2025-09-16T18:50:32.396834] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T18:50:32.413152] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-09-16T18:50:33.174896] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-07. [2025-09-16T18:50:33.890726] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-09-16T18:50:33.896931] [INFO ] [REMOTE ] HP: Full fetch complete (56250 records). [2025-09-16T18:50:33.897074] [DEBUG] [VALIDATOR ] HP: Validating primary data. [2025-09-16T18:50:33.979315] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-02. [2025-09-16T18:50:34.106911] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-09-16T18:50:34.306240] [DEBUG] [VALIDATOR ] HP: All expected trading days are present. [2025-09-16T18:50:34.306326] [DEBUG] [VALIDATOR ] HP: Primary data validation passed. [2025-09-16T18:50:34.307100] [INFO ] [MAIN ] HP: Fetched primary data is valid. [2025-09-16T18:50:34.307147] [DEBUG] [LOCAL ] HP: Saving primary data. [2025-09-16T18:50:34.308549] [DEBUG] [LOCAL ] HP: Trimmed 56250 records to 38678 (keeping 150 days of data) [2025-09-16T18:50:34.551297] [DEBUG] [LOCAL ] HP: Primary data saved (38678 records). [2025-09-16T18:50:34.551449] [INFO ] [MAIN ] HP: Full history saved. [2025-09-16T18:50:34.551563] [INFO ] [MAIN ] Update Progress: 1044/2369 symbols completed. HP took 29.8s. Estimated remaining: 2h 20m 16s. Rate: 9.4 symbols/min [2025-09-16T18:50:34.725178] [INFO ] [MAIN ] HPS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:34.734367] [DEBUG] [LOCAL ] HPS: Loaded 7654 existing records from primary storage. [2025-09-16T18:50:34.734467] [INFO ] [MAIN ] HPS: Loaded 7654 existing records. [2025-09-16T18:50:34.780116] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-08. [2025-09-16T18:50:34.914370] [WARN ] [MAIN ] HPS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:34.914614] [INFO ] [MAIN ] HPS: Wiping existing primary data before full refresh. [2025-09-16T18:50:34.914688] [INFO ] [LOCAL ] HPS: Starting deletion of primary data. [2025-09-16T18:50:34.914998] [DEBUG] [LOCAL ] HPS: Deleted primary data file. [2025-09-16T18:50:34.915042] [INFO ] [MAIN ] HPS: Starting full primary refresh. [2025-09-16T18:50:35.267399] [INFO ] [REMOTE ] HPS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:35.574874] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-02. [2025-09-16T18:50:35.640206] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T18:50:35.687888] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-09-16T18:50:36.141881] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-09-16T18:50:36.374906] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-03. [2025-09-16T18:50:37.174900] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-04. [2025-09-16T18:50:37.974903] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-03. [2025-09-16T18:50:38.324710] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-09-16T18:50:38.756356] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-09-16T18:50:38.774871] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-04. [2025-09-16T18:50:38.833174] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T18:50:39.500983] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-09-16T18:50:39.574898] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-05. [2025-09-16T18:50:40.374896] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-05. [2025-09-16T18:50:40.869589] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-09-16T18:50:41.174916] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-04. [2025-09-16T18:50:41.234106] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T18:50:41.974886] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-06. [2025-09-16T18:50:42.774865] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-06. [2025-09-16T18:50:43.317545] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-09-16T18:50:43.577114] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-05. [2025-09-16T18:50:43.579687] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-09-16T18:50:43.597626] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-09-16T18:50:44.374900] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-07. [2025-09-16T18:50:45.174960] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-07. [2025-09-16T18:50:45.527138] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-09-16T18:50:45.978003] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-09. [2025-09-16T18:50:45.979913] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T18:50:46.205251] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-09-16T18:50:46.774928] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-08. [2025-09-16T18:50:46.992961] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T18:50:46.995611] [INFO ] [REMOTE ] HPI: Full fetch complete (10279 records). [2025-09-16T18:50:46.995706] [DEBUG] [VALIDATOR ] HPI: Validating primary data. [2025-09-16T18:50:47.352520] [DEBUG] [VALIDATOR ] HPI: All expected trading days are present. [2025-09-16T18:50:47.352598] [DEBUG] [VALIDATOR ] HPI: Primary data validation passed. [2025-09-16T18:50:47.352848] [INFO ] [MAIN ] HPI: Fetched primary data is valid. [2025-09-16T18:50:47.352893] [DEBUG] [LOCAL ] HPI: Saving primary data. [2025-09-16T18:50:47.353353] [DEBUG] [LOCAL ] HPI: Trimmed 10279 records to 7756 (keeping 150 days of data) [2025-09-16T18:50:47.406790] [DEBUG] [LOCAL ] HPI: Primary data saved (7756 records). [2025-09-16T18:50:47.406937] [INFO ] [MAIN ] HPI: Full history saved. [2025-09-16T18:50:47.407028] [INFO ] [MAIN ] Update Progress: 1045/2369 symbols completed. HPI took 28.5s. Estimated remaining: 2h 20m 18s. Rate: 9.4 symbols/min [2025-09-16T18:50:47.580022] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-06. [2025-09-16T18:50:47.581676] [INFO ] [MAIN ] HQH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:47.603315] [DEBUG] [LOCAL ] HQH: Loaded 18689 existing records from primary storage. [2025-09-16T18:50:47.603464] [INFO ] [MAIN ] HQH: Loaded 18689 existing records. [2025-09-16T18:50:47.777941] [WARN ] [MAIN ] HQH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:47.778156] [INFO ] [MAIN ] HQH: Wiping existing primary data before full refresh. [2025-09-16T18:50:47.778203] [INFO ] [LOCAL ] HQH: Starting deletion of primary data. [2025-09-16T18:50:47.778725] [DEBUG] [LOCAL ] HQH: Deleted primary data file. [2025-09-16T18:50:47.778781] [INFO ] [MAIN ] HQH: Starting full primary refresh. [2025-09-16T18:50:48.126984] [INFO ] [REMOTE ] HQH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:48.374858] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-02. [2025-09-16T18:50:48.419280] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T18:50:49.174947] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-08. [2025-09-16T18:50:49.475741] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-09-16T18:50:49.500593] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-09-16T18:50:49.944982] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-09-16T18:50:49.974850] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-09. [2025-09-16T18:50:50.774892] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-07. [2025-09-16T18:50:50.912274] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-09-16T18:50:50.913916] [INFO ] [REMOTE ] HPS: Full fetch complete (11369 records). [2025-09-16T18:50:50.914003] [DEBUG] [VALIDATOR ] HPS: Validating primary data. [2025-09-16T18:50:51.269696] [DEBUG] [VALIDATOR ] HPS: All expected trading days are present. [2025-09-16T18:50:51.269774] [DEBUG] [VALIDATOR ] HPS: Primary data validation passed. [2025-09-16T18:50:51.270043] [INFO ] [MAIN ] HPS: Fetched primary data is valid. [2025-09-16T18:50:51.270083] [DEBUG] [LOCAL ] HPS: Saving primary data. [2025-09-16T18:50:51.270522] [DEBUG] [LOCAL ] HPS: Trimmed 11369 records to 7704 (keeping 150 days of data) [2025-09-16T18:50:51.319057] [DEBUG] [LOCAL ] HPS: Primary data saved (7704 records). [2025-09-16T18:50:51.319157] [INFO ] [MAIN ] HPS: Full history saved. [2025-09-16T18:50:51.319236] [INFO ] [MAIN ] Update Progress: 1046/2369 symbols completed. HPS took 16.8s. Estimated remaining: 2h 20m 9s. Rate: 9.4 symbols/min [2025-09-16T18:50:51.487747] [INFO ] [MAIN ] HQL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:51.499879] [DEBUG] [LOCAL ] HQL: Loaded 11683 existing records from primary storage. [2025-09-16T18:50:51.499999] [INFO ] [MAIN ] HQL: Loaded 11683 existing records. [2025-09-16T18:50:51.580062] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-09. [2025-09-16T18:50:51.677406] [WARN ] [MAIN ] HQL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:51.677687] [INFO ] [MAIN ] HQL: Wiping existing primary data before full refresh. [2025-09-16T18:50:51.677737] [INFO ] [LOCAL ] HQL: Starting deletion of primary data. [2025-09-16T18:50:51.678173] [DEBUG] [LOCAL ] HQL: Deleted primary data file. [2025-09-16T18:50:51.678219] [INFO ] [MAIN ] HQL: Starting full primary refresh. [2025-09-16T18:50:52.029628] [INFO ] [REMOTE ] HQL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:52.374909] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-02. [2025-09-16T18:50:52.812880] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-09-16T18:50:53.174982] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-03. [2025-09-16T18:50:53.281882] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T18:50:53.288879] [INFO ] [REMOTE ] HPP: Full fetch complete (54970 records). [2025-09-16T18:50:53.289014] [DEBUG] [VALIDATOR ] HPP: Validating primary data. [2025-09-16T18:50:53.530560] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-09-16T18:50:53.694188] [DEBUG] [VALIDATOR ] HPP: All expected trading days are present. [2025-09-16T18:50:53.694271] [DEBUG] [VALIDATOR ] HPP: Primary data validation passed. [2025-09-16T18:50:53.695091] [INFO ] [MAIN ] HPP: Fetched primary data is valid. [2025-09-16T18:50:53.695138] [DEBUG] [LOCAL ] HPP: Saving primary data. [2025-09-16T18:50:53.696307] [DEBUG] [LOCAL ] HPP: Trimmed 54970 records to 38143 (keeping 150 days of data) [2025-09-16T18:50:53.920470] [DEBUG] [LOCAL ] HPP: Primary data saved (38143 records). [2025-09-16T18:50:53.920627] [INFO ] [MAIN ] HPP: Full history saved. [2025-09-16T18:50:53.920729] [INFO ] [MAIN ] Update Progress: 1047/2369 symbols completed. HPP took 28.9s. Estimated remaining: 2h 19m 58s. Rate: 9.4 symbols/min [2025-09-16T18:50:53.985250] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-03. [2025-09-16T18:50:54.097040] [INFO ] [MAIN ] HR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:50:54.112163] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T18:50:54.153154] [DEBUG] [LOCAL ] HR: Loaded 38391 existing records from primary storage. [2025-09-16T18:50:54.153351] [INFO ] [MAIN ] HR: Loaded 38391 existing records. [2025-09-16T18:50:54.340026] [WARN ] [MAIN ] HR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:50:54.340320] [INFO ] [MAIN ] HR: Wiping existing primary data before full refresh. [2025-09-16T18:50:54.340370] [INFO ] [LOCAL ] HR: Starting deletion of primary data. [2025-09-16T18:50:54.341291] [DEBUG] [LOCAL ] HR: Deleted primary data file. [2025-09-16T18:50:54.341341] [INFO ] [MAIN ] HR: Starting full primary refresh. [2025-09-16T18:50:54.694935] [INFO ] [REMOTE ] HR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:50:54.774849] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-02. [2025-09-16T18:50:54.842239] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-09-16T18:50:55.574939] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-04. [2025-09-16T18:50:56.373402] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T18:50:56.376906] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-08. [2025-09-16T18:50:56.692937] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T18:50:57.174898] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-03. [2025-09-16T18:50:57.974940] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-04. [2025-09-16T18:50:58.434084] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-09-16T18:50:58.645753] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-09-16T18:50:58.774904] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-04. [2025-09-16T18:50:58.906535] [DEBUG] [REMOTE ] HR: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-09-16T18:50:58.906889] [WARN ] [REMOTE ] HR: No data returned for month 2025-04 [2025-09-16T18:50:58.987217] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-09-16T18:50:59.574927] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-05. [2025-09-16T18:51:00.374951] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-05. [2025-09-16T18:51:00.528974] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-09-16T18:51:01.174920] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-06. [2025-09-16T18:51:01.974970] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-05. [2025-09-16T18:51:02.130402] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T18:51:02.154314] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-09-16T18:51:02.774911] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-07. [2025-09-16T18:51:02.971413] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T18:51:03.574925] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-06. [2025-09-16T18:51:03.859898] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-09-16T18:51:04.374890] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-09. [2025-09-16T18:51:04.685312] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-09-16T18:51:05.174916] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-06. [2025-09-16T18:51:05.974943] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-08. [2025-09-16T18:51:06.432846] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-09-16T18:51:06.441989] [INFO ] [REMOTE ] HPQ: Full fetch complete (58347 records). [2025-09-16T18:51:06.442172] [DEBUG] [VALIDATOR ] HPQ: Validating primary data. [2025-09-16T18:51:06.727782] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T18:51:06.785236] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-09. [2025-09-16T18:51:06.883743] [DEBUG] [VALIDATOR ] HPQ: All expected trading days are present. [2025-09-16T18:51:06.883916] [DEBUG] [VALIDATOR ] HPQ: Primary data validation passed. [2025-09-16T18:51:06.884914] [INFO ] [MAIN ] HPQ: Fetched primary data is valid. [2025-09-16T18:51:06.884982] [DEBUG] [LOCAL ] HPQ: Saving primary data. [2025-09-16T18:51:06.886604] [DEBUG] [LOCAL ] HPQ: Trimmed 58347 records to 40021 (keeping 150 days of data) [2025-09-16T18:51:07.021582] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-09-16T18:51:07.214183] [DEBUG] [LOCAL ] HPQ: Primary data saved (40021 records). [2025-09-16T18:51:07.214320] [INFO ] [MAIN ] HPQ: Full history saved. [2025-09-16T18:51:07.214406] [INFO ] [MAIN ] Update Progress: 1048/2369 symbols completed. HPQ took 36.1s. Estimated remaining: 2h 20m 0s. Rate: 9.4 symbols/min [2025-09-16T18:51:07.395187] [INFO ] [MAIN ] HRB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:51:07.428809] [DEBUG] [LOCAL ] HRB: Loaded 35716 existing records from primary storage. [2025-09-16T18:51:07.428957] [INFO ] [MAIN ] HRB: Loaded 35716 existing records. [2025-09-16T18:51:07.587599] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-07. [2025-09-16T18:51:07.629161] [WARN ] [MAIN ] HRB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:51:07.629399] [INFO ] [MAIN ] HRB: Wiping existing primary data before full refresh. [2025-09-16T18:51:07.629446] [INFO ] [LOCAL ] HRB: Starting deletion of primary data. [2025-09-16T18:51:07.630341] [DEBUG] [LOCAL ] HRB: Deleted primary data file. [2025-09-16T18:51:07.630391] [INFO ] [MAIN ] HRB: Starting full primary refresh. [2025-09-16T18:51:07.956473] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-09-16T18:51:07.961671] [INFO ] [REMOTE ] HQL: Full fetch complete (17804 records). [2025-09-16T18:51:07.972217] [DEBUG] [VALIDATOR ] HQL: Validating primary data. [2025-09-16T18:51:08.112926] [INFO ] [REMOTE ] HRB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:51:08.381668] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-07. [2025-09-16T18:51:08.407764] [DEBUG] [VALIDATOR ] HQL: All expected trading days are present. [2025-09-16T18:51:08.407849] [DEBUG] [VALIDATOR ] HQL: Primary data validation passed. [2025-09-16T18:51:08.408233] [INFO ] [MAIN ] HQL: Fetched primary data is valid. [2025-09-16T18:51:08.408273] [DEBUG] [LOCAL ] HQL: Saving primary data. [2025-09-16T18:51:08.408855] [DEBUG] [LOCAL ] HQL: Trimmed 17804 records to 11773 (keeping 150 days of data) [2025-09-16T18:51:08.483624] [DEBUG] [LOCAL ] HQL: Primary data saved (11773 records). [2025-09-16T18:51:08.483768] [INFO ] [MAIN ] HQL: Full history saved. [2025-09-16T18:51:08.483852] [INFO ] [MAIN ] Update Progress: 1049/2369 symbols completed. HQL took 17.2s. Estimated remaining: 2h 19m 47s. Rate: 9.4 symbols/min [2025-09-16T18:51:08.653694] [INFO ] [MAIN ] HRI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:51:08.677256] [DEBUG] [LOCAL ] HRI: Loaded 23844 existing records from primary storage. [2025-09-16T18:51:08.677394] [INFO ] [MAIN ] HRI: Loaded 23844 existing records. [2025-09-16T18:51:08.854873] [WARN ] [MAIN ] HRI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:51:08.855144] [INFO ] [MAIN ] HRI: Wiping existing primary data before full refresh. [2025-09-16T18:51:08.855205] [INFO ] [LOCAL ] HRI: Starting deletion of primary data. [2025-09-16T18:51:08.855940] [DEBUG] [LOCAL ] HRI: Deleted primary data file. [2025-09-16T18:51:08.856003] [INFO ] [MAIN ] HRI: Starting full primary refresh. [2025-09-16T18:51:09.204214] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-02. [2025-09-16T18:51:09.210598] [INFO ] [REMOTE ] HRI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:51:09.227508] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T18:51:09.974926] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-08. [2025-09-16T18:51:10.178797] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-09-16T18:51:10.774877] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-08. [2025-09-16T18:51:10.816298] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T18:51:11.225037] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-09-16T18:51:11.574972] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-03. [2025-09-16T18:51:12.374981] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-09. [2025-09-16T18:51:12.560755] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T18:51:13.174908] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-09. [2025-09-16T18:51:13.258713] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-09-16T18:51:13.442213] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-09-16T18:51:13.447036] [INFO ] [REMOTE ] HQH: Full fetch complete (27271 records). [2025-09-16T18:51:13.447236] [DEBUG] [VALIDATOR ] HQH: Validating primary data. [2025-09-16T18:51:13.823607] [DEBUG] [VALIDATOR ] HQH: All expected trading days are present. [2025-09-16T18:51:13.823702] [DEBUG] [VALIDATOR ] HQH: Primary data validation passed. [2025-09-16T18:51:13.824252] [INFO ] [MAIN ] HQH: Fetched primary data is valid. [2025-09-16T18:51:13.824294] [DEBUG] [LOCAL ] HQH: Saving primary data. [2025-09-16T18:51:13.824998] [DEBUG] [LOCAL ] HQH: Trimmed 27271 records to 18896 (keeping 150 days of data) [2025-09-16T18:51:13.944463] [DEBUG] [LOCAL ] HQH: Primary data saved (18896 records). [2025-09-16T18:51:13.944614] [INFO ] [MAIN ] HQH: Full history saved. [2025-09-16T18:51:13.944843] [INFO ] [MAIN ] Update Progress: 1050/2369 symbols completed. HQH took 26.5s. Estimated remaining: 2h 19m 40s. Rate: 9.4 symbols/min [2025-09-16T18:51:13.980031] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-04. [2025-09-16T18:51:14.119708] [INFO ] [MAIN ] HRL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:51:14.119910] [DEBUG] [LOCAL ] HRL: No existing data file found in primary storage. [2025-09-16T18:51:14.119950] [WARN ] [MAIN ] HRL: No existing data found, performing full refresh instead. [2025-09-16T18:51:14.120072] [INFO ] [MAIN ] HRL: Wiping existing primary data before full refresh. [2025-09-16T18:51:14.120115] [INFO ] [LOCAL ] HRL: Starting deletion of primary data. [2025-09-16T18:51:14.120157] [DEBUG] [LOCAL ] HRL: No primary data file found to delete. [2025-09-16T18:51:14.120190] [INFO ] [MAIN ] HRL: Starting full primary refresh. [2025-09-16T18:51:14.474944] [INFO ] [REMOTE ] HRL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:51:14.774964] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-02. [2025-09-16T18:51:14.951594] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T18:51:14.958032] [INFO ] [REMOTE ] HR: Full fetch complete (48443 records). [2025-09-16T18:51:14.958135] [DEBUG] [VALIDATOR ] HR: Validating primary data. [2025-09-16T18:51:15.185472] [WARN ] [VALIDATOR ] HR: Validation failed - insufficient historical data. Have 129 unique days, need 150 days [2025-09-16T18:51:15.186097] [FATAL] [MAIN ] HR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:51:15.186154] [WARN ] [MAIN ] HR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:51:15.186201] [INFO ] [LOCAL ] HR: Starting deletion of primary data. [2025-09-16T18:51:15.186267] [DEBUG] [LOCAL ] HR: No primary data file found to delete. [2025-09-16T18:51:15.186305] [INFO ] [LOCAL ] HR: Starting deletion of derivative data. [2025-09-16T18:51:15.186687] [DEBUG] [LOCAL ] HR: Deleted derivative data file. [2025-09-16T18:51:15.186783] [INFO ] [MAIN ] Update Progress: 1051/2369 symbols completed. HR took 21.3s. Estimated remaining: 2h 19m 27s. Rate: 9.5 symbols/min [2025-09-16T18:51:15.358767] [INFO ] [MAIN ] HRTG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:51:15.383397] [DEBUG] [LOCAL ] HRTG: Loaded 25072 existing records from primary storage. [2025-09-16T18:51:15.383598] [INFO ] [MAIN ] HRTG: Loaded 25072 existing records. [2025-09-16T18:51:15.567987] [WARN ] [MAIN ] HRTG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:51:15.568309] [INFO ] [MAIN ] HRTG: Wiping existing primary data before full refresh. [2025-09-16T18:51:15.568432] [INFO ] [LOCAL ] HRTG: Starting deletion of primary data. [2025-09-16T18:51:15.569210] [DEBUG] [LOCAL ] HRTG: Deleted primary data file. [2025-09-16T18:51:15.569263] [INFO ] [MAIN ] HRTG: Starting full primary refresh. [2025-09-16T18:51:15.577006] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-02. [2025-09-16T18:51:15.941045] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-09-16T18:51:16.006973] [INFO ] [REMOTE ] HRTG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:51:16.374880] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-05. [2025-09-16T18:51:16.483318] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-09-16T18:51:16.681727] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T18:51:17.174935] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-03. [2025-09-16T18:51:17.762445] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T18:51:17.974911] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-03. [2025-09-16T18:51:18.774949] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-02. [2025-09-16T18:51:19.018220] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T18:51:19.413704] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T18:51:19.461828] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T18:51:19.574901] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-03. [2025-09-16T18:51:20.374861] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-04. [2025-09-16T18:51:20.735302] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T18:51:21.174903] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-04. [2025-09-16T18:51:21.548916] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-09-16T18:51:21.974899] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-06. [2025-09-16T18:51:22.516355] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-09-16T18:51:22.774920] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-05. [2025-09-16T18:51:23.532516] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T18:51:23.574859] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-07. [2025-09-16T18:51:24.260182] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T18:51:24.374911] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-06. [2025-09-16T18:51:24.938377] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-09-16T18:51:25.174908] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-05. [2025-09-16T18:51:25.788817] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T18:51:25.974868] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-07. [2025-09-16T18:51:26.610829] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T18:51:26.774889] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-06. [2025-09-16T18:51:27.088362] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-09-16T18:51:27.574967] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-08. [2025-09-16T18:51:28.112709] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-09-16T18:51:28.374923] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-08. [2025-09-16T18:51:29.174882] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-07. [2025-09-16T18:51:29.243600] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T18:51:29.683342] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T18:51:29.974909] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-04. [2025-09-16T18:51:30.180554] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-09-16T18:51:30.774889] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-08. [2025-09-16T18:51:31.574873] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-09. [2025-09-16T18:51:31.752279] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-09-16T18:51:31.809147] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T18:51:32.374948] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-09. [2025-09-16T18:51:32.725532] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T18:51:32.729436] [INFO ] [REMOTE ] HRTG: Full fetch complete (33787 records). [2025-09-16T18:51:32.729531] [DEBUG] [VALIDATOR ] HRTG: Validating primary data. [2025-09-16T18:51:33.101151] [DEBUG] [VALIDATOR ] HRTG: All expected trading days are present. [2025-09-16T18:51:33.101246] [DEBUG] [VALIDATOR ] HRTG: Primary data validation passed. [2025-09-16T18:51:33.101866] [INFO ] [MAIN ] HRTG: Fetched primary data is valid. [2025-09-16T18:51:33.101915] [DEBUG] [LOCAL ] HRTG: Saving primary data. [2025-09-16T18:51:33.102944] [DEBUG] [LOCAL ] HRTG: Trimmed 33787 records to 25347 (keeping 150 days of data) [2025-09-16T18:51:33.188156] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-05. [2025-09-16T18:51:33.255944] [DEBUG] [LOCAL ] HRTG: Primary data saved (25347 records). [2025-09-16T18:51:33.256086] [INFO ] [MAIN ] HRTG: Full history saved. [2025-09-16T18:51:33.256177] [INFO ] [MAIN ] Update Progress: 1052/2369 symbols completed. HRTG took 18.1s. Estimated remaining: 2h 19m 35s. Rate: 9.4 symbols/min [2025-09-16T18:51:33.446760] [INFO ] [MAIN ] HSBC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:51:33.459308] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T18:51:33.463863] [INFO ] [REMOTE ] HRI: Full fetch complete (36723 records). [2025-09-16T18:51:33.464093] [DEBUG] [VALIDATOR ] HRI: Validating primary data. [2025-09-16T18:51:33.711609] [DEBUG] [LOCAL ] HSBC: Loaded 37442 existing records from primary storage. [2025-09-16T18:51:33.776468] [INFO ] [MAIN ] HSBC: Loaded 37442 existing records. [2025-09-16T18:51:33.976296] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-09. [2025-09-16T18:51:34.101237] [DEBUG] [VALIDATOR ] HRI: All expected trading days are present. [2025-09-16T18:51:34.101526] [DEBUG] [VALIDATOR ] HRI: Primary data validation passed. [2025-09-16T18:51:34.101445] [WARN ] [MAIN ] HSBC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:51:34.102380] [INFO ] [MAIN ] HRI: Fetched primary data is valid. [2025-09-16T18:51:34.102451] [DEBUG] [LOCAL ] HRI: Saving primary data. [2025-09-16T18:51:34.102868] [INFO ] [MAIN ] HSBC: Wiping existing primary data before full refresh. [2025-09-16T18:51:34.103246] [INFO ] [LOCAL ] HSBC: Starting deletion of primary data. [2025-09-16T18:51:34.103855] [DEBUG] [LOCAL ] HRI: Trimmed 36723 records to 24045 (keeping 150 days of data) [2025-09-16T18:51:34.104568] [DEBUG] [LOCAL ] HSBC: Deleted primary data file. [2025-09-16T18:51:34.104702] [INFO ] [MAIN ] HSBC: Starting full primary refresh. [2025-09-16T18:51:34.591732] [DEBUG] [LOCAL ] HRI: Primary data saved (24045 records). [2025-09-16T18:51:34.591880] [INFO ] [MAIN ] HRI: Full history saved. [2025-09-16T18:51:34.592226] [INFO ] [MAIN ] Update Progress: 1053/2369 symbols completed. HRI took 26.1s. Estimated remaining: 2h 19m 23s. Rate: 9.4 symbols/min [2025-09-16T18:51:34.768126] [INFO ] [REMOTE ] HSBC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:51:34.780335] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-02. [2025-09-16T18:51:34.860391] [INFO ] [MAIN ] HSHP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:51:34.872546] [DEBUG] [LOCAL ] HSHP: Loaded 10106 existing records from primary storage. [2025-09-16T18:51:34.872719] [INFO ] [MAIN ] HSHP: Loaded 10106 existing records. [2025-09-16T18:51:35.053832] [WARN ] [MAIN ] HSHP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:51:35.054276] [INFO ] [MAIN ] HSHP: Wiping existing primary data before full refresh. [2025-09-16T18:51:35.054394] [INFO ] [LOCAL ] HSHP: Starting deletion of primary data. [2025-09-16T18:51:35.054831] [DEBUG] [LOCAL ] HSHP: Deleted primary data file. [2025-09-16T18:51:35.054882] [INFO ] [MAIN ] HSHP: Starting full primary refresh. [2025-09-16T18:51:35.444221] [INFO ] [REMOTE ] HSHP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:51:35.446773] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-05 (2.26 seconds). [2025-09-16T18:51:35.574887] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-06. [2025-09-16T18:51:35.738865] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T18:51:35.744924] [INFO ] [REMOTE ] HRB: Full fetch complete (52889 records). [2025-09-16T18:51:35.745032] [DEBUG] [VALIDATOR ] HRB: Validating primary data. [2025-09-16T18:51:36.126877] [DEBUG] [VALIDATOR ] HRB: All expected trading days are present. [2025-09-16T18:51:36.126952] [DEBUG] [VALIDATOR ] HRB: Primary data validation passed. [2025-09-16T18:51:36.127687] [INFO ] [MAIN ] HRB: Fetched primary data is valid. [2025-09-16T18:51:36.127733] [DEBUG] [LOCAL ] HRB: Saving primary data. [2025-09-16T18:51:36.128791] [DEBUG] [LOCAL ] HRB: Trimmed 52889 records to 36056 (keeping 150 days of data) [2025-09-16T18:51:36.354594] [DEBUG] [LOCAL ] HRB: Primary data saved (36056 records). [2025-09-16T18:51:36.354898] [INFO ] [MAIN ] HRB: Full history saved. [2025-09-16T18:51:36.355003] [INFO ] [MAIN ] Update Progress: 1054/2369 symbols completed. HRB took 29.1s. Estimated remaining: 2h 19m 11s. Rate: 9.4 symbols/min [2025-09-16T18:51:36.387635] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-02. [2025-09-16T18:51:36.532405] [INFO ] [MAIN ] HSY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:51:36.574864] [DEBUG] [LOCAL ] HSY: Loaded 37273 existing records from primary storage. [2025-09-16T18:51:36.575043] [INFO ] [MAIN ] HSY: Loaded 37273 existing records. [2025-09-16T18:51:36.755317] [WARN ] [MAIN ] HSY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:51:36.755720] [INFO ] [MAIN ] HSY: Wiping existing primary data before full refresh. [2025-09-16T18:51:36.755858] [INFO ] [LOCAL ] HSY: Starting deletion of primary data. [2025-09-16T18:51:36.765177] [DEBUG] [LOCAL ] HSY: Deleted primary data file. [2025-09-16T18:51:36.778540] [INFO ] [MAIN ] HSY: Starting full primary refresh. [2025-09-16T18:51:36.853475] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-09-16T18:51:37.165580] [INFO ] [REMOTE ] HSY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:51:37.185845] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-03. [2025-09-16T18:51:37.215315] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T18:51:37.576604] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T18:51:37.974937] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-07. [2025-09-16T18:51:38.774936] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-02. [2025-09-16T18:51:39.238389] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-09-16T18:51:39.574904] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-03. [2025-09-16T18:51:40.001853] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-09-16T18:51:40.374913] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-08. [2025-09-16T18:51:40.512566] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T18:51:40.643584] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-09-16T18:51:41.174910] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-04. [2025-09-16T18:51:41.951003] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-09-16T18:51:41.982149] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-04. [2025-09-16T18:51:41.987690] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-09-16T18:51:42.774934] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-05. [2025-09-16T18:51:43.578239] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-09. [2025-09-16T18:51:43.578094] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-09-16T18:51:43.708339] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T18:51:44.374912] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-06. [2025-09-16T18:51:45.174872] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-03. [2025-09-16T18:51:45.417951] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-09-16T18:51:45.425001] [INFO ] [REMOTE ] HRL: Full fetch complete (57725 records). [2025-09-16T18:51:45.425200] [DEBUG] [VALIDATOR ] HRL: Validating primary data. [2025-09-16T18:51:45.765903] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T18:51:45.841289] [DEBUG] [VALIDATOR ] HRL: All expected trading days are present. [2025-09-16T18:51:45.841374] [DEBUG] [VALIDATOR ] HRL: Primary data validation passed. [2025-09-16T18:51:45.842217] [INFO ] [MAIN ] HRL: Fetched primary data is valid. [2025-09-16T18:51:45.842287] [DEBUG] [LOCAL ] HRL: Saving primary data. [2025-09-16T18:51:45.843524] [DEBUG] [LOCAL ] HRL: Trimmed 57725 records to 39585 (keeping 150 days of data) [2025-09-16T18:51:46.022067] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-05. [2025-09-16T18:51:46.093840] [DEBUG] [LOCAL ] HRL: Primary data saved (39585 records). [2025-09-16T18:51:46.094032] [INFO ] [MAIN ] HRL: Full history saved. [2025-09-16T18:51:46.094118] [INFO ] [MAIN ] Update Progress: 1055/2369 symbols completed. HRL took 32.1s. Estimated remaining: 2h 19m 9s. Rate: 9.4 symbols/min [2025-09-16T18:51:46.265254] [INFO ] [MAIN ] HTB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:51:46.265469] [DEBUG] [LOCAL ] HTB: No existing data file found in primary storage. [2025-09-16T18:51:46.265507] [WARN ] [MAIN ] HTB: No existing data found, performing full refresh instead. [2025-09-16T18:51:46.265631] [INFO ] [MAIN ] HTB: Wiping existing primary data before full refresh. [2025-09-16T18:51:46.265693] [INFO ] [LOCAL ] HTB: Starting deletion of primary data. [2025-09-16T18:51:46.265736] [DEBUG] [LOCAL ] HTB: No primary data file found to delete. [2025-09-16T18:51:46.265771] [INFO ] [MAIN ] HTB: Starting full primary refresh. [2025-09-16T18:51:46.625599] [INFO ] [REMOTE ] HTB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:51:46.774969] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-02. [2025-09-16T18:51:47.031928] [INFO ] [REMOTE ] HTB: Incremental fetch complete for 2025-02 (0.26 seconds). [2025-09-16T18:51:47.081332] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-09-16T18:51:47.574930] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-03. [2025-09-16T18:51:48.204134] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-05 (2.18 seconds). [2025-09-16T18:51:48.220571] [INFO ] [REMOTE ] HTB: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-09-16T18:51:48.374895] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-06. [2025-09-16T18:51:49.174898] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-07. [2025-09-16T18:51:49.974921] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-04. [2025-09-16T18:51:50.343739] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-09-16T18:51:50.364246] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-09-16T18:51:50.774897] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-04. [2025-09-16T18:51:51.519621] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-09-16T18:51:51.535865] [INFO ] [REMOTE ] HTB: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-09-16T18:51:51.574850] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-05. [2025-09-16T18:51:52.374906] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-08. [2025-09-16T18:51:53.173880] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-09-16T18:51:53.176803] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-07. [2025-09-16T18:51:53.465185] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-09-16T18:51:53.974902] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-09. [2025-09-16T18:51:54.774870] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-06. [2025-09-16T18:51:54.906661] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T18:51:54.908992] [INFO ] [REMOTE ] HSHP: Full fetch complete (17524 records). [2025-09-16T18:51:54.909082] [DEBUG] [VALIDATOR ] HSHP: Validating primary data. [2025-09-16T18:51:55.272404] [DEBUG] [VALIDATOR ] HSHP: All expected trading days are present. [2025-09-16T18:51:55.272500] [DEBUG] [VALIDATOR ] HSHP: Primary data validation passed. [2025-09-16T18:51:55.273021] [INFO ] [MAIN ] HSHP: Fetched primary data is valid. [2025-09-16T18:51:55.273100] [DEBUG] [LOCAL ] HSHP: Saving primary data. [2025-09-16T18:51:55.273848] [DEBUG] [LOCAL ] HSHP: Trimmed 17524 records to 10229 (keeping 150 days of data) [2025-09-16T18:51:55.355142] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-07 (2.18 seconds). [2025-09-16T18:51:55.375615] [DEBUG] [LOCAL ] HSHP: Primary data saved (10229 records). [2025-09-16T18:51:55.375773] [INFO ] [MAIN ] HSHP: Full history saved. [2025-09-16T18:51:55.375864] [INFO ] [MAIN ] Update Progress: 1056/2369 symbols completed. HSHP took 20.8s. Estimated remaining: 2h 19m 6s. Rate: 9.4 symbols/min [2025-09-16T18:51:55.547990] [INFO ] [MAIN ] HTD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:51:55.558424] [DEBUG] [LOCAL ] HTD: Loaded 9858 existing records from primary storage. [2025-09-16T18:51:55.558531] [INFO ] [MAIN ] HTD: Loaded 9858 existing records. [2025-09-16T18:51:55.580051] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-05. [2025-09-16T18:51:55.741196] [WARN ] [MAIN ] HTD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:51:55.741460] [INFO ] [MAIN ] HTD: Wiping existing primary data before full refresh. [2025-09-16T18:51:55.741507] [INFO ] [LOCAL ] HTD: Starting deletion of primary data. [2025-09-16T18:51:55.741969] [DEBUG] [LOCAL ] HTD: Deleted primary data file. [2025-09-16T18:51:55.742020] [INFO ] [MAIN ] HTD: Starting full primary refresh. [2025-09-16T18:51:56.114884] [INFO ] [REMOTE ] HTD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:51:56.367105] [INFO ] [REMOTE ] HTB: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-09-16T18:51:56.374834] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-06. [2025-09-16T18:51:56.453612] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T18:51:57.085463] [INFO ] [REMOTE ] HTB: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-09-16T18:51:57.174872] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-08. [2025-09-16T18:51:57.974949] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-02. [2025-09-16T18:51:58.777036] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-07. [2025-09-16T18:51:58.779725] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-09-16T18:51:59.102932] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T18:51:59.514822] [INFO ] [REMOTE ] HTB: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-09-16T18:51:59.574846] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-08. [2025-09-16T18:52:00.293590] [INFO ] [REMOTE ] HTB: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-09-16T18:52:00.374897] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-09. [2025-09-16T18:52:01.174914] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-09. [2025-09-16T18:52:01.940779] [INFO ] [REMOTE ] HTB: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-09-16T18:52:01.943113] [INFO ] [REMOTE ] HTB: Full fetch complete (7011 records). [2025-09-16T18:52:01.943239] [DEBUG] [VALIDATOR ] HTB: Validating primary data. [2025-09-16T18:52:01.980001] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-03. [2025-09-16T18:52:02.132543] [WARN ] [VALIDATOR ] HTB: Validation failed - insufficient historical data. Have 138 unique days, need 150 days [2025-09-16T18:52:02.132787] [FATAL] [MAIN ] HTB: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:52:02.132838] [WARN ] [MAIN ] HTB: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:52:02.132879] [INFO ] [LOCAL ] HTB: Starting deletion of primary data. [2025-09-16T18:52:02.132957] [DEBUG] [LOCAL ] HTB: No primary data file found to delete. [2025-09-16T18:52:02.132994] [INFO ] [LOCAL ] HTB: Starting deletion of derivative data. [2025-09-16T18:52:02.133034] [DEBUG] [LOCAL ] HTB: No derivative data file found to delete. [2025-09-16T18:52:02.133110] [INFO ] [MAIN ] Update Progress: 1057/2369 symbols completed. HTB took 16.0s. Estimated remaining: 2h 19m 0s. Rate: 9.4 symbols/min [2025-09-16T18:52:02.300292] [INFO ] [MAIN ] HTFC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:02.300519] [DEBUG] [LOCAL ] HTFC: No existing data file found in primary storage. [2025-09-16T18:52:02.300662] [WARN ] [MAIN ] HTFC: No existing data found, performing full refresh instead. [2025-09-16T18:52:02.300908] [INFO ] [MAIN ] HTFC: Wiping existing primary data before full refresh. [2025-09-16T18:52:02.300977] [INFO ] [LOCAL ] HTFC: Starting deletion of primary data. [2025-09-16T18:52:02.301028] [DEBUG] [LOCAL ] HTFC: No primary data file found to delete. [2025-09-16T18:52:02.301200] [INFO ] [MAIN ] HTFC: Starting full primary refresh. [2025-09-16T18:52:02.449635] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-09-16T18:52:02.513480] [INFO ] [REMOTE ] HSBC: Full fetch complete (55800 records). [2025-09-16T18:52:02.583620] [DEBUG] [VALIDATOR ] HSBC: Validating primary data. [2025-09-16T18:52:02.793238] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-07. [2025-09-16T18:52:02.796428] [INFO ] [REMOTE ] HTFC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:03.087013] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-09-16T18:52:03.089944] [DEBUG] [VALIDATOR ] HSBC: All expected trading days are present. [2025-09-16T18:52:03.094694] [DEBUG] [VALIDATOR ] HSBC: Primary data validation passed. [2025-09-16T18:52:03.096008] [INFO ] [MAIN ] HSBC: Fetched primary data is valid. [2025-09-16T18:52:03.096564] [DEBUG] [LOCAL ] HSBC: Saving primary data. [2025-09-16T18:52:03.097781] [DEBUG] [LOCAL ] HSBC: Trimmed 55800 records to 37817 (keeping 150 days of data) [2025-09-16T18:52:03.331216] [DEBUG] [LOCAL ] HSBC: Primary data saved (37817 records). [2025-09-16T18:52:03.331382] [INFO ] [MAIN ] HSBC: Full history saved. [2025-09-16T18:52:03.331480] [INFO ] [MAIN ] Update Progress: 1058/2369 symbols completed. HSBC took 30.1s. Estimated remaining: 2h 18m 47s. Rate: 9.4 symbols/min [2025-09-16T18:52:03.500415] [INFO ] [MAIN ] HTGC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:03.534541] [DEBUG] [LOCAL ] HTGC: Loaded 34840 existing records from primary storage. [2025-09-16T18:52:03.534755] [INFO ] [MAIN ] HTGC: Loaded 34840 existing records. [2025-09-16T18:52:03.580121] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-04. [2025-09-16T18:52:03.771512] [WARN ] [MAIN ] HTGC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:52:03.771916] [INFO ] [MAIN ] HTGC: Wiping existing primary data before full refresh. [2025-09-16T18:52:03.772001] [INFO ] [LOCAL ] HTGC: Starting deletion of primary data. [2025-09-16T18:52:03.773075] [DEBUG] [LOCAL ] HTGC: Deleted primary data file. [2025-09-16T18:52:03.773152] [INFO ] [MAIN ] HTGC: Starting full primary refresh. [2025-09-16T18:52:04.287254] [INFO ] [REMOTE ] HTGC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:04.374994] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-02. [2025-09-16T18:52:04.710215] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T18:52:04.745673] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-09-16T18:52:04.749718] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-09-16T18:52:05.174896] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-03. [2025-09-16T18:52:05.491742] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-09-16T18:52:05.974920] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-02. [2025-09-16T18:52:06.774922] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-05. [2025-09-16T18:52:07.574878] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-04. [2025-09-16T18:52:07.602906] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T18:52:07.707508] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-09-16T18:52:07.874962] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-09-16T18:52:08.374956] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-08. [2025-09-16T18:52:09.174921] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-06. [2025-09-16T18:52:09.974936] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-03. [2025-09-16T18:52:10.034595] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-09-16T18:52:10.198921] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T18:52:10.774901] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-07. [2025-09-16T18:52:11.574885] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-05. [2025-09-16T18:52:11.730535] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T18:52:11.783752] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-09-16T18:52:11.824760] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-05 (0.25 seconds). [2025-09-16T18:52:12.374913] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-06. [2025-09-16T18:52:12.685383] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-09-16T18:52:13.174915] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-08. [2025-09-16T18:52:13.839688] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-09-16T18:52:13.974925] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-09. [2025-09-16T18:52:14.774919] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-04. [2025-09-16T18:52:14.968861] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T18:52:14.970998] [INFO ] [REMOTE ] HTD: Full fetch complete (14266 records). [2025-09-16T18:52:14.971070] [DEBUG] [VALIDATOR ] HTD: Validating primary data. [2025-09-16T18:52:15.335288] [DEBUG] [VALIDATOR ] HTD: All expected trading days are present. [2025-09-16T18:52:15.335360] [DEBUG] [VALIDATOR ] HTD: Primary data validation passed. [2025-09-16T18:52:15.335680] [INFO ] [MAIN ] HTD: Fetched primary data is valid. [2025-09-16T18:52:15.335724] [DEBUG] [LOCAL ] HTD: Saving primary data. [2025-09-16T18:52:15.336307] [DEBUG] [LOCAL ] HTD: Trimmed 14266 records to 10000 (keeping 150 days of data) [2025-09-16T18:52:15.398101] [DEBUG] [LOCAL ] HTD: Primary data saved (10000 records). [2025-09-16T18:52:15.398229] [INFO ] [MAIN ] HTD: Full history saved. [2025-09-16T18:52:15.398323] [INFO ] [MAIN ] Update Progress: 1059/2369 symbols completed. HTD took 20.0s. Estimated remaining: 2h 18m 48s. Rate: 9.4 symbols/min [2025-09-16T18:52:15.572502] [INFO ] [MAIN ] HTH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:15.574879] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-09. [2025-09-16T18:52:15.597580] [DEBUG] [LOCAL ] HTH: Loaded 24977 existing records from primary storage. [2025-09-16T18:52:15.597742] [INFO ] [MAIN ] HTH: Loaded 24977 existing records. [2025-09-16T18:52:15.783687] [WARN ] [MAIN ] HTH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:52:15.783967] [INFO ] [MAIN ] HTH: Wiping existing primary data before full refresh. [2025-09-16T18:52:15.784345] [INFO ] [LOCAL ] HTH: Starting deletion of primary data. [2025-09-16T18:52:15.785288] [DEBUG] [LOCAL ] HTH: Deleted primary data file. [2025-09-16T18:52:15.785343] [INFO ] [MAIN ] HTH: Starting full primary refresh. [2025-09-16T18:52:16.135509] [INFO ] [REMOTE ] HTH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:16.374910] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-07. [2025-09-16T18:52:16.504116] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T18:52:16.607422] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-07 (0.23 seconds). [2025-09-16T18:52:17.174905] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-02. [2025-09-16T18:52:17.449187] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-09-16T18:52:17.455258] [INFO ] [REMOTE ] HSY: Full fetch complete (55099 records). [2025-09-16T18:52:17.455396] [DEBUG] [VALIDATOR ] HSY: Validating primary data. [2025-09-16T18:52:17.833162] [DEBUG] [VALIDATOR ] HSY: All expected trading days are present. [2025-09-16T18:52:17.833239] [DEBUG] [VALIDATOR ] HSY: Primary data validation passed. [2025-09-16T18:52:17.834018] [INFO ] [MAIN ] HSY: Fetched primary data is valid. [2025-09-16T18:52:17.834067] [DEBUG] [LOCAL ] HSY: Saving primary data. [2025-09-16T18:52:17.835132] [DEBUG] [LOCAL ] HSY: Trimmed 55099 records to 37663 (keeping 150 days of data) [2025-09-16T18:52:18.013360] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-08. [2025-09-16T18:52:18.083386] [DEBUG] [LOCAL ] HSY: Primary data saved (37663 records). [2025-09-16T18:52:18.083529] [INFO ] [MAIN ] HSY: Full history saved. [2025-09-16T18:52:18.083663] [INFO ] [MAIN ] Update Progress: 1060/2369 symbols completed. HSY took 41.7s. Estimated remaining: 2h 18m 37s. Rate: 9.4 symbols/min [2025-09-16T18:52:18.253679] [INFO ] [MAIN ] HUBB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:18.279890] [DEBUG] [LOCAL ] HUBB: Loaded 26654 existing records from primary storage. [2025-09-16T18:52:18.280016] [INFO ] [MAIN ] HUBB: Loaded 26654 existing records. [2025-09-16T18:52:18.361728] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-09-16T18:52:18.459092] [WARN ] [MAIN ] HUBB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:52:18.459330] [INFO ] [MAIN ] HUBB: Wiping existing primary data before full refresh. [2025-09-16T18:52:18.459376] [INFO ] [LOCAL ] HUBB: Starting deletion of primary data. [2025-09-16T18:52:18.460173] [DEBUG] [LOCAL ] HUBB: Deleted primary data file. [2025-09-16T18:52:18.460224] [INFO ] [MAIN ] HUBB: Starting full primary refresh. [2025-09-16T18:52:18.778486] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-05. [2025-09-16T18:52:18.843102] [INFO ] [REMOTE ] HUBB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:18.866934] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T18:52:19.574938] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-09. [2025-09-16T18:52:19.919158] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-09 (0.34 seconds). [2025-09-16T18:52:19.919903] [INFO ] [REMOTE ] HTFC: Full fetch complete (629 records). [2025-09-16T18:52:19.919972] [DEBUG] [VALIDATOR ] HTFC: Validating primary data. [2025-09-16T18:52:20.103678] [WARN ] [VALIDATOR ] HTFC: Validation failed - insufficient historical data. Have 139 unique days, need 150 days [2025-09-16T18:52:20.103847] [FATAL] [MAIN ] HTFC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:52:20.103895] [WARN ] [MAIN ] HTFC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:52:20.103953] [INFO ] [LOCAL ] HTFC: Starting deletion of primary data. [2025-09-16T18:52:20.104016] [DEBUG] [LOCAL ] HTFC: No primary data file found to delete. [2025-09-16T18:52:20.104050] [INFO ] [LOCAL ] HTFC: Starting deletion of derivative data. [2025-09-16T18:52:20.104127] [DEBUG] [LOCAL ] HTFC: No derivative data file found to delete. [2025-09-16T18:52:20.104189] [INFO ] [MAIN ] Update Progress: 1061/2369 symbols completed. HTFC took 18.0s. Estimated remaining: 2h 18m 25s. Rate: 9.4 symbols/min [2025-09-16T18:52:20.284293] [INFO ] [MAIN ] HUBS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:20.317339] [DEBUG] [LOCAL ] HUBS: Loaded 28996 existing records from primary storage. [2025-09-16T18:52:20.317477] [INFO ] [MAIN ] HUBS: Loaded 28996 existing records. [2025-09-16T18:52:20.383189] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-02. [2025-09-16T18:52:20.503783] [WARN ] [MAIN ] HUBS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:52:20.504045] [INFO ] [MAIN ] HUBS: Wiping existing primary data before full refresh. [2025-09-16T18:52:20.504100] [INFO ] [LOCAL ] HUBS: Starting deletion of primary data. [2025-09-16T18:52:20.504847] [DEBUG] [LOCAL ] HUBS: Deleted primary data file. [2025-09-16T18:52:20.504900] [INFO ] [MAIN ] HUBS: Starting full primary refresh. [2025-09-16T18:52:20.868954] [INFO ] [REMOTE ] HUBS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:20.898029] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-09-16T18:52:21.174869] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-02. [2025-09-16T18:52:21.974917] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-06. [2025-09-16T18:52:22.091506] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-09-16T18:52:22.579229] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-09-16T18:52:22.774885] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-03. [2025-09-16T18:52:23.520618] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-09-16T18:52:23.574871] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-07. [2025-09-16T18:52:24.374922] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-03. [2025-09-16T18:52:24.530306] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T18:52:24.840255] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-09-16T18:52:25.174902] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-08. [2025-09-16T18:52:25.360980] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-09-16T18:52:25.974902] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-04. [2025-09-16T18:52:26.774912] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-04. [2025-09-16T18:52:26.968094] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T18:52:27.574898] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-03. [2025-09-16T18:52:27.712156] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-09-16T18:52:28.374888] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-05. [2025-09-16T18:52:28.616179] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T18:52:29.174899] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-05. [2025-09-16T18:52:29.282791] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T18:52:29.981565] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-09. [2025-09-16T18:52:29.986967] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T18:52:30.361775] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-09-16T18:52:30.774927] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-04. [2025-09-16T18:52:31.574905] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-06. [2025-09-16T18:52:31.607527] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-09-16T18:52:31.613190] [INFO ] [REMOTE ] HTGC: Full fetch complete (51815 records). [2025-09-16T18:52:31.613286] [DEBUG] [VALIDATOR ] HTGC: Validating primary data. [2025-09-16T18:52:32.000339] [DEBUG] [VALIDATOR ] HTGC: All expected trading days are present. [2025-09-16T18:52:32.000424] [DEBUG] [VALIDATOR ] HTGC: Primary data validation passed. [2025-09-16T18:52:32.001244] [INFO ] [MAIN ] HTGC: Fetched primary data is valid. [2025-09-16T18:52:32.001323] [DEBUG] [LOCAL ] HTGC: Saving primary data. [2025-09-16T18:52:32.002915] [DEBUG] [LOCAL ] HTGC: Trimmed 51815 records to 35199 (keeping 150 days of data) [2025-09-16T18:52:32.240194] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T18:52:32.257526] [DEBUG] [LOCAL ] HTGC: Primary data saved (35199 records). [2025-09-16T18:52:32.257669] [INFO ] [MAIN ] HTGC: Full history saved. [2025-09-16T18:52:32.257812] [INFO ] [MAIN ] Update Progress: 1062/2369 symbols completed. HTGC took 28.9s. Estimated remaining: 2h 18m 26s. Rate: 9.4 symbols/min [2025-09-16T18:52:32.380060] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-06. [2025-09-16T18:52:32.431703] [INFO ] [MAIN ] HUM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:32.479491] [DEBUG] [LOCAL ] HUM: Loaded 36176 existing records from primary storage. [2025-09-16T18:52:32.479671] [INFO ] [MAIN ] HUM: Loaded 36176 existing records. [2025-09-16T18:52:32.663292] [WARN ] [MAIN ] HUM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:52:32.663599] [INFO ] [MAIN ] HUM: Wiping existing primary data before full refresh. [2025-09-16T18:52:32.663674] [INFO ] [LOCAL ] HUM: Starting deletion of primary data. [2025-09-16T18:52:32.664602] [DEBUG] [LOCAL ] HUM: Deleted primary data file. [2025-09-16T18:52:32.664688] [INFO ] [MAIN ] HUM: Starting full primary refresh. [2025-09-16T18:52:33.019294] [INFO ] [REMOTE ] HUM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:33.035795] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T18:52:33.174866] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-02. [2025-09-16T18:52:33.500402] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T18:52:33.974903] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-07. [2025-09-16T18:52:34.459610] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-09-16T18:52:34.774896] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-07. [2025-09-16T18:52:35.574899] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-05. [2025-09-16T18:52:35.647752] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-09-16T18:52:35.830323] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-09-16T18:52:36.374896] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-08. [2025-09-16T18:52:37.174890] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-08. [2025-09-16T18:52:37.220067] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T18:52:37.890760] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-09-16T18:52:37.974890] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-09. [2025-09-16T18:52:38.774892] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-03. [2025-09-16T18:52:38.814372] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T18:52:38.973247] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T18:52:38.978557] [INFO ] [REMOTE ] HTH: Full fetch complete (36445 records). [2025-09-16T18:52:38.979131] [DEBUG] [VALIDATOR ] HTH: Validating primary data. [2025-09-16T18:52:39.354884] [DEBUG] [VALIDATOR ] HTH: All expected trading days are present. [2025-09-16T18:52:39.354979] [DEBUG] [VALIDATOR ] HTH: Primary data validation passed. [2025-09-16T18:52:39.355622] [INFO ] [MAIN ] HTH: Fetched primary data is valid. [2025-09-16T18:52:39.355683] [DEBUG] [LOCAL ] HTH: Saving primary data. [2025-09-16T18:52:39.356544] [DEBUG] [LOCAL ] HTH: Trimmed 36445 records to 25177 (keeping 150 days of data) [2025-09-16T18:52:39.514369] [DEBUG] [LOCAL ] HTH: Primary data saved (25177 records). [2025-09-16T18:52:39.514512] [INFO ] [MAIN ] HTH: Full history saved. [2025-09-16T18:52:39.514615] [INFO ] [MAIN ] Update Progress: 1063/2369 symbols completed. HTH took 24.1s. Estimated remaining: 2h 18m 21s. Rate: 9.4 symbols/min [2025-09-16T18:52:39.584608] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-06. [2025-09-16T18:52:39.699307] [INFO ] [MAIN ] HUN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:39.737752] [DEBUG] [LOCAL ] HUN: Loaded 39242 existing records from primary storage. [2025-09-16T18:52:39.737924] [INFO ] [MAIN ] HUN: Loaded 39242 existing records. [2025-09-16T18:52:39.914617] [WARN ] [MAIN ] HUN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:52:39.914861] [INFO ] [MAIN ] HUN: Wiping existing primary data before full refresh. [2025-09-16T18:52:39.914908] [INFO ] [LOCAL ] HUN: Starting deletion of primary data. [2025-09-16T18:52:39.915748] [DEBUG] [LOCAL ] HUN: Deleted primary data file. [2025-09-16T18:52:39.915802] [INFO ] [MAIN ] HUN: Starting full primary refresh. [2025-09-16T18:52:40.267727] [INFO ] [REMOTE ] HUN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:40.374849] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-02. [2025-09-16T18:52:40.447054] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T18:52:41.015296] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T18:52:41.174858] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-04. [2025-09-16T18:52:41.974874] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-09. [2025-09-16T18:52:42.023186] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-09-16T18:52:42.774892] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-07. [2025-09-16T18:52:43.034601] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-09-16T18:52:43.574886] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-05. [2025-09-16T18:52:43.590069] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T18:52:43.596902] [INFO ] [REMOTE ] HUBS: Full fetch complete (43321 records). [2025-09-16T18:52:43.597032] [DEBUG] [VALIDATOR ] HUBS: Validating primary data. [2025-09-16T18:52:43.974128] [DEBUG] [VALIDATOR ] HUBS: All expected trading days are present. [2025-09-16T18:52:43.974235] [DEBUG] [VALIDATOR ] HUBS: Primary data validation passed. [2025-09-16T18:52:43.975046] [INFO ] [MAIN ] HUBS: Fetched primary data is valid. [2025-09-16T18:52:43.975166] [DEBUG] [LOCAL ] HUBS: Saving primary data. [2025-09-16T18:52:43.976365] [DEBUG] [LOCAL ] HUBS: Trimmed 43321 records to 29255 (keeping 150 days of data) [2025-09-16T18:52:44.088998] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-09-16T18:52:44.189781] [DEBUG] [LOCAL ] HUBS: Primary data saved (29255 records). [2025-09-16T18:52:44.189963] [INFO ] [MAIN ] HUBS: Full history saved. [2025-09-16T18:52:44.190085] [INFO ] [MAIN ] Update Progress: 1064/2369 symbols completed. HUBS took 24.1s. Estimated remaining: 2h 18m 13s. Rate: 9.4 symbols/min [2025-09-16T18:52:44.359823] [INFO ] [MAIN ] HUYA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:44.379171] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-08. [2025-09-16T18:52:44.392953] [DEBUG] [LOCAL ] HUYA: Loaded 31968 existing records from primary storage. [2025-09-16T18:52:44.393098] [INFO ] [MAIN ] HUYA: Loaded 31968 existing records. [2025-09-16T18:52:44.568128] [WARN ] [MAIN ] HUYA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:52:44.568339] [INFO ] [MAIN ] HUYA: Wiping existing primary data before full refresh. [2025-09-16T18:52:44.568384] [INFO ] [LOCAL ] HUYA: Starting deletion of primary data. [2025-09-16T18:52:44.568989] [DEBUG] [LOCAL ] HUYA: Deleted primary data file. [2025-09-16T18:52:44.569036] [INFO ] [MAIN ] HUYA: Starting full primary refresh. [2025-09-16T18:52:44.920121] [INFO ] [REMOTE ] HUYA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:45.175849] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-03. [2025-09-16T18:52:45.181523] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T18:52:45.813680] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-09-16T18:52:45.974906] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-06. [2025-09-16T18:52:46.774906] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-09. [2025-09-16T18:52:46.950433] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T18:52:47.574931] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-02. [2025-09-16T18:52:47.654287] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T18:52:47.969681] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-09-16T18:52:47.974336] [INFO ] [REMOTE ] HUBB: Full fetch complete (41273 records). [2025-09-16T18:52:47.974418] [DEBUG] [VALIDATOR ] HUBB: Validating primary data. [2025-09-16T18:52:48.351841] [DEBUG] [VALIDATOR ] HUBB: All expected trading days are present. [2025-09-16T18:52:48.351960] [DEBUG] [VALIDATOR ] HUBB: Primary data validation passed. [2025-09-16T18:52:48.352878] [INFO ] [MAIN ] HUBB: Fetched primary data is valid. [2025-09-16T18:52:48.352937] [DEBUG] [LOCAL ] HUBB: Saving primary data. [2025-09-16T18:52:48.354006] [DEBUG] [LOCAL ] HUBB: Trimmed 41273 records to 26937 (keeping 150 days of data) [2025-09-16T18:52:48.428070] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-04. [2025-09-16T18:52:48.547760] [DEBUG] [LOCAL ] HUBB: Primary data saved (26937 records). [2025-09-16T18:52:48.547924] [INFO ] [MAIN ] HUBB: Full history saved. [2025-09-16T18:52:48.548012] [INFO ] [MAIN ] Update Progress: 1065/2369 symbols completed. HUBB took 30.5s. Estimated remaining: 2h 18m 4s. Rate: 9.4 symbols/min [2025-09-16T18:52:48.720178] [INFO ] [MAIN ] HVT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:48.731826] [DEBUG] [LOCAL ] HVT: Loaded 10975 existing records from primary storage. [2025-09-16T18:52:48.731937] [INFO ] [MAIN ] HVT: Loaded 10975 existing records. [2025-09-16T18:52:48.906018] [WARN ] [MAIN ] HVT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:52:48.906474] [INFO ] [MAIN ] HVT: Wiping existing primary data before full refresh. [2025-09-16T18:52:48.906601] [INFO ] [LOCAL ] HVT: Starting deletion of primary data. [2025-09-16T18:52:48.907071] [DEBUG] [LOCAL ] HVT: Deleted primary data file. [2025-09-16T18:52:48.907132] [INFO ] [MAIN ] HVT: Starting full primary refresh. [2025-09-16T18:52:49.183830] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-07. [2025-09-16T18:52:49.213811] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:52:49.304130] [INFO ] [REMOTE ] HVT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:49.974944] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-03. [2025-09-16T18:52:50.306598] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T18:52:50.774948] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-05. [2025-09-16T18:52:51.078793] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T18:52:51.475664] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-09-16T18:52:51.574855] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-04. [2025-09-16T18:52:52.379998] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-08. [2025-09-16T18:52:52.397126] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T18:52:52.633097] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-09-16T18:52:53.174914] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-05. [2025-09-16T18:52:53.974886] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-06. [2025-09-16T18:52:54.029393] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T18:52:54.670264] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-09-16T18:52:54.774951] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-09. [2025-09-16T18:52:55.574908] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-06. [2025-09-16T18:52:55.709553] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-09-16T18:52:56.374977] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-02. [2025-09-16T18:52:56.650169] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-09-16T18:52:56.657847] [INFO ] [REMOTE ] HUM: Full fetch complete (53285 records). [2025-09-16T18:52:56.658077] [DEBUG] [VALIDATOR ] HUM: Validating primary data. [2025-09-16T18:52:56.898579] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T18:52:57.176629] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-07. [2025-09-16T18:52:57.292154] [DEBUG] [VALIDATOR ] HUM: All expected trading days are present. [2025-09-16T18:52:57.292261] [DEBUG] [VALIDATOR ] HUM: Primary data validation passed. [2025-09-16T18:52:57.293199] [INFO ] [MAIN ] HUM: Fetched primary data is valid. [2025-09-16T18:52:57.293251] [DEBUG] [LOCAL ] HUM: Saving primary data. [2025-09-16T18:52:57.294799] [DEBUG] [LOCAL ] HUM: Trimmed 53285 records to 36539 (keeping 150 days of data) [2025-09-16T18:52:57.425746] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-09-16T18:52:57.572927] [DEBUG] [LOCAL ] HUM: Primary data saved (36539 records). [2025-09-16T18:52:57.573329] [INFO ] [MAIN ] HUM: Full history saved. [2025-09-16T18:52:57.573425] [INFO ] [MAIN ] Update Progress: 1066/2369 symbols completed. HUM took 25.3s. Estimated remaining: 2h 18m 1s. Rate: 9.4 symbols/min [2025-09-16T18:52:57.765980] [INFO ] [MAIN ] HWM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:52:57.811537] [DEBUG] [LOCAL ] HWM: Loaded 39128 existing records from primary storage. [2025-09-16T18:52:57.811749] [INFO ] [MAIN ] HWM: Loaded 39128 existing records. [2025-09-16T18:52:57.980069] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-07. [2025-09-16T18:52:58.017513] [WARN ] [MAIN ] HWM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:52:58.017825] [INFO ] [MAIN ] HWM: Wiping existing primary data before full refresh. [2025-09-16T18:52:58.017890] [INFO ] [LOCAL ] HWM: Starting deletion of primary data. [2025-09-16T18:52:58.018916] [DEBUG] [LOCAL ] HWM: Deleted primary data file. [2025-09-16T18:52:58.018983] [INFO ] [MAIN ] HWM: Starting full primary refresh. [2025-09-16T18:52:58.387924] [INFO ] [REMOTE ] HWM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:52:58.774881] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-02. [2025-09-16T18:52:59.043467] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T18:52:59.574896] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-03. [2025-09-16T18:52:59.723762] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T18:53:00.220119] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-09-16T18:53:00.374909] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-08. [2025-09-16T18:53:00.871612] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T18:53:01.174920] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-04. [2025-09-16T18:53:01.921945] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T18:53:01.974891] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-09. [2025-09-16T18:53:02.307236] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-09-16T18:53:02.774959] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-08. [2025-09-16T18:53:03.429437] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-09-16T18:53:03.435029] [INFO ] [REMOTE ] HUYA: Full fetch complete (48156 records). [2025-09-16T18:53:03.435130] [DEBUG] [VALIDATOR ] HUYA: Validating primary data. [2025-09-16T18:53:03.580065] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-05. [2025-09-16T18:53:03.824662] [DEBUG] [VALIDATOR ] HUYA: All expected trading days are present. [2025-09-16T18:53:03.824765] [DEBUG] [VALIDATOR ] HUYA: Primary data validation passed. [2025-09-16T18:53:03.825566] [INFO ] [MAIN ] HUYA: Fetched primary data is valid. [2025-09-16T18:53:03.825607] [DEBUG] [LOCAL ] HUYA: Saving primary data. [2025-09-16T18:53:03.826611] [DEBUG] [LOCAL ] HUYA: Trimmed 48156 records to 32344 (keeping 150 days of data) [2025-09-16T18:53:04.013786] [DEBUG] [LOCAL ] HUYA: Primary data saved (32344 records). [2025-09-16T18:53:04.013920] [INFO ] [MAIN ] HUYA: Full history saved. [2025-09-16T18:53:04.014019] [INFO ] [MAIN ] Update Progress: 1067/2369 symbols completed. HUYA took 19.8s. Estimated remaining: 2h 17m 54s. Rate: 9.4 symbols/min [2025-09-16T18:53:04.184421] [INFO ] [MAIN ] HXL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:04.216057] [DEBUG] [LOCAL ] HXL: Loaded 33913 existing records from primary storage. [2025-09-16T18:53:04.216188] [INFO ] [MAIN ] HXL: Loaded 33913 existing records. [2025-09-16T18:53:04.378195] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-03. [2025-09-16T18:53:04.400282] [WARN ] [MAIN ] HXL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:53:04.400438] [INFO ] [MAIN ] HXL: Wiping existing primary data before full refresh. [2025-09-16T18:53:04.400481] [INFO ] [LOCAL ] HXL: Starting deletion of primary data. [2025-09-16T18:53:04.401149] [DEBUG] [LOCAL ] HXL: Deleted primary data file. [2025-09-16T18:53:04.401196] [INFO ] [MAIN ] HXL: Starting full primary refresh. [2025-09-16T18:53:04.780153] [INFO ] [REMOTE ] HXL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:04.780833] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T18:53:04.851526] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-09-16T18:53:05.174900] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-09. [2025-09-16T18:53:05.974993] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-02. [2025-09-16T18:53:06.303748] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-09-16T18:53:06.690704] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T18:53:06.698202] [INFO ] [REMOTE ] HUN: Full fetch complete (57851 records). [2025-09-16T18:53:06.698334] [DEBUG] [VALIDATOR ] HUN: Validating primary data. [2025-09-16T18:53:06.780036] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-04. [2025-09-16T18:53:07.084133] [DEBUG] [VALIDATOR ] HUN: All expected trading days are present. [2025-09-16T18:53:07.084217] [DEBUG] [VALIDATOR ] HUN: Primary data validation passed. [2025-09-16T18:53:07.085003] [INFO ] [MAIN ] HUN: Fetched primary data is valid. [2025-09-16T18:53:07.085050] [DEBUG] [LOCAL ] HUN: Saving primary data. [2025-09-16T18:53:07.086283] [DEBUG] [LOCAL ] HUN: Trimmed 57851 records to 39628 (keeping 150 days of data) [2025-09-16T18:53:07.326696] [DEBUG] [LOCAL ] HUN: Primary data saved (39628 records). [2025-09-16T18:53:07.326841] [INFO ] [MAIN ] HUN: Full history saved. [2025-09-16T18:53:07.328547] [INFO ] [MAIN ] Update Progress: 1068/2369 symbols completed. HUN took 27.8s. Estimated remaining: 2h 17m 44s. Rate: 9.4 symbols/min [2025-09-16T18:53:07.498226] [INFO ] [MAIN ] HY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:07.525210] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T18:53:07.536609] [DEBUG] [LOCAL ] HY: Loaded 7126 existing records from primary storage. [2025-09-16T18:53:07.536717] [INFO ] [MAIN ] HY: Loaded 7126 existing records. [2025-09-16T18:53:07.580063] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-03. [2025-09-16T18:53:07.713718] [WARN ] [MAIN ] HY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:53:07.714014] [INFO ] [MAIN ] HY: Wiping existing primary data before full refresh. [2025-09-16T18:53:07.714068] [INFO ] [LOCAL ] HY: Starting deletion of primary data. [2025-09-16T18:53:07.714391] [DEBUG] [LOCAL ] HY: Deleted primary data file. [2025-09-16T18:53:07.714436] [INFO ] [MAIN ] HY: Starting full primary refresh. [2025-09-16T18:53:08.068315] [INFO ] [REMOTE ] HY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:08.374879] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-02. [2025-09-16T18:53:08.463605] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-09-16T18:53:09.174890] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-05. [2025-09-16T18:53:09.233264] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T18:53:09.291284] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T18:53:09.974919] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-03. [2025-09-16T18:53:10.774879] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-06. [2025-09-16T18:53:10.810590] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T18:53:10.887243] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-09-16T18:53:11.403065] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-09-16T18:53:11.574940] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-04. [2025-09-16T18:53:12.374910] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-06. [2025-09-16T18:53:12.479221] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-09-16T18:53:13.174918] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-05. [2025-09-16T18:53:13.974933] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-07. [2025-09-16T18:53:14.177892] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T18:53:14.501584] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-09-16T18:53:14.774902] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-06. [2025-09-16T18:53:15.075083] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-09-16T18:53:15.554088] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T18:53:15.574876] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-07. [2025-09-16T18:53:16.362913] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T18:53:16.374844] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-08. [2025-09-16T18:53:17.180534] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-07. [2025-09-16T18:53:17.182334] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T18:53:17.974906] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-09. [2025-09-16T18:53:18.774918] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-08. [2025-09-16T18:53:18.836680] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T18:53:18.838953] [INFO ] [REMOTE ] HY: Full fetch complete (11439 records). [2025-09-16T18:53:18.839029] [DEBUG] [VALIDATOR ] HY: Validating primary data. [2025-09-16T18:53:19.228925] [DEBUG] [VALIDATOR ] HY: All expected trading days are present. [2025-09-16T18:53:19.229023] [DEBUG] [VALIDATOR ] HY: Primary data validation passed. [2025-09-16T18:53:19.229408] [INFO ] [MAIN ] HY: Fetched primary data is valid. [2025-09-16T18:53:19.229495] [DEBUG] [LOCAL ] HY: Saving primary data. [2025-09-16T18:53:19.230162] [DEBUG] [LOCAL ] HY: Trimmed 11439 records to 7194 (keeping 150 days of data) [2025-09-16T18:53:19.275165] [DEBUG] [LOCAL ] HY: Primary data saved (7194 records). [2025-09-16T18:53:19.275388] [INFO ] [MAIN ] HY: Full history saved. [2025-09-16T18:53:19.275527] [INFO ] [MAIN ] Update Progress: 1069/2369 symbols completed. HY took 11.9s. Estimated remaining: 2h 17m 45s. Rate: 9.4 symbols/min [2025-09-16T18:53:19.470161] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-07 (2.29 seconds). [2025-09-16T18:53:19.498314] [INFO ] [MAIN ] HYAC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:19.498739] [DEBUG] [LOCAL ] HYAC: No existing data file found in primary storage. [2025-09-16T18:53:19.498870] [WARN ] [MAIN ] HYAC: No existing data found, performing full refresh instead. [2025-09-16T18:53:19.498995] [INFO ] [MAIN ] HYAC: Wiping existing primary data before full refresh. [2025-09-16T18:53:19.499036] [INFO ] [LOCAL ] HYAC: Starting deletion of primary data. [2025-09-16T18:53:19.499411] [DEBUG] [LOCAL ] HYAC: No primary data file found to delete. [2025-09-16T18:53:19.499464] [INFO ] [MAIN ] HYAC: Starting full primary refresh. [2025-09-16T18:53:19.580036] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-08. [2025-09-16T18:53:19.807460] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T18:53:19.877159] [INFO ] [REMOTE ] HYAC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:20.374932] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-02. [2025-09-16T18:53:20.711067] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-09-16T18:53:21.174953] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-04. [2025-09-16T18:53:21.460489] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-09-16T18:53:21.974916] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-03. [2025-09-16T18:53:22.190392] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-03 (0.22 seconds). [2025-09-16T18:53:22.774908] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-09. [2025-09-16T18:53:22.855228] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-09-16T18:53:23.574902] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-09. [2025-09-16T18:53:23.754596] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T18:53:23.756694] [INFO ] [REMOTE ] HVT: Full fetch complete (17904 records). [2025-09-16T18:53:23.756764] [DEBUG] [VALIDATOR ] HVT: Validating primary data. [2025-09-16T18:53:24.138407] [DEBUG] [VALIDATOR ] HVT: All expected trading days are present. [2025-09-16T18:53:24.138541] [DEBUG] [VALIDATOR ] HVT: Primary data validation passed. [2025-09-16T18:53:24.139023] [INFO ] [MAIN ] HVT: Fetched primary data is valid. [2025-09-16T18:53:24.139083] [DEBUG] [LOCAL ] HVT: Saving primary data. [2025-09-16T18:53:24.139769] [DEBUG] [LOCAL ] HVT: Trimmed 17904 records to 11049 (keeping 150 days of data) [2025-09-16T18:53:24.209188] [DEBUG] [LOCAL ] HVT: Primary data saved (11049 records). [2025-09-16T18:53:24.209338] [INFO ] [MAIN ] HVT: Full history saved. [2025-09-16T18:53:24.209425] [INFO ] [MAIN ] Update Progress: 1070/2369 symbols completed. HVT took 35.7s. Estimated remaining: 2h 17m 37s. Rate: 9.4 symbols/min [2025-09-16T18:53:24.376001] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-05. [2025-09-16T18:53:24.377253] [INFO ] [MAIN ] HYBX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:24.378119] [DEBUG] [LOCAL ] HYBX: No existing data file found in primary storage. [2025-09-16T18:53:24.378165] [WARN ] [MAIN ] HYBX: No existing data found, performing full refresh instead. [2025-09-16T18:53:24.378318] [INFO ] [MAIN ] HYBX: Wiping existing primary data before full refresh. [2025-09-16T18:53:24.378357] [INFO ] [LOCAL ] HYBX: Starting deletion of primary data. [2025-09-16T18:53:24.378397] [DEBUG] [LOCAL ] HYBX: No primary data file found to delete. [2025-09-16T18:53:24.378429] [INFO ] [MAIN ] HYBX: Starting full primary refresh. [2025-09-16T18:53:24.740717] [INFO ] [REMOTE ] HYBX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:25.174882] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-04. [2025-09-16T18:53:25.533633] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-09-16T18:53:25.550209] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-09-16T18:53:25.556675] [INFO ] [REMOTE ] HWM: Full fetch complete (57692 records). [2025-09-16T18:53:25.556785] [DEBUG] [VALIDATOR ] HWM: Validating primary data. [2025-09-16T18:53:25.935517] [DEBUG] [VALIDATOR ] HWM: All expected trading days are present. [2025-09-16T18:53:25.935598] [DEBUG] [VALIDATOR ] HWM: Primary data validation passed. [2025-09-16T18:53:25.936400] [INFO ] [MAIN ] HWM: Fetched primary data is valid. [2025-09-16T18:53:25.936448] [DEBUG] [LOCAL ] HWM: Saving primary data. [2025-09-16T18:53:25.937696] [DEBUG] [LOCAL ] HWM: Trimmed 57692 records to 39509 (keeping 150 days of data) [2025-09-16T18:53:26.001455] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-05. [2025-09-16T18:53:26.200713] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-09-16T18:53:26.238699] [DEBUG] [LOCAL ] HWM: Primary data saved (39509 records). [2025-09-16T18:53:26.238855] [INFO ] [MAIN ] HWM: Full history saved. [2025-09-16T18:53:26.238944] [INFO ] [MAIN ] Update Progress: 1071/2369 symbols completed. HWM took 28.7s. Estimated remaining: 2h 17m 25s. Rate: 9.4 symbols/min [2025-09-16T18:53:26.391898] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-09-16T18:53:26.421583] [INFO ] [MAIN ] HYI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:26.430025] [DEBUG] [LOCAL ] HYI: Loaded 6436 existing records from primary storage. [2025-09-16T18:53:26.430183] [INFO ] [MAIN ] HYI: Loaded 6436 existing records. [2025-09-16T18:53:26.604615] [WARN ] [MAIN ] HYI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:53:26.604903] [INFO ] [MAIN ] HYI: Wiping existing primary data before full refresh. [2025-09-16T18:53:26.604949] [INFO ] [LOCAL ] HYI: Starting deletion of primary data. [2025-09-16T18:53:26.605348] [DEBUG] [LOCAL ] HYI: Deleted primary data file. [2025-09-16T18:53:26.605396] [INFO ] [MAIN ] HYI: Starting full primary refresh. [2025-09-16T18:53:26.784422] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-02. [2025-09-16T18:53:26.973478] [INFO ] [REMOTE ] HYI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:27.117032] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-09-16T18:53:27.574905] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-06. [2025-09-16T18:53:28.374903] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-06. [2025-09-16T18:53:28.692307] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-06 (0.32 seconds). [2025-09-16T18:53:29.174887] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-03. [2025-09-16T18:53:29.341409] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-09-16T18:53:29.483960] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-09-16T18:53:29.974905] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-04. [2025-09-16T18:53:30.300822] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-09-16T18:53:30.774973] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-05. [2025-09-16T18:53:31.101256] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-09-16T18:53:31.574895] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-02. [2025-09-16T18:53:32.283012] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-09-16T18:53:32.374889] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-07. [2025-09-16T18:53:32.787974] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-09-16T18:53:33.174887] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-08. [2025-09-16T18:53:33.402936] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-08 (0.23 seconds). [2025-09-16T18:53:33.974915] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-07. [2025-09-16T18:53:34.774896] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-03. [2025-09-16T18:53:35.574909] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-09. [2025-09-16T18:53:35.632327] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-09-16T18:53:35.792873] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T18:53:35.895708] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-09-16T18:53:35.896578] [INFO ] [REMOTE ] HYAC: Full fetch complete (649 records). [2025-09-16T18:53:35.896691] [DEBUG] [VALIDATOR ] HYAC: Validating primary data. [2025-09-16T18:53:35.897166] [FATAL] [VALIDATOR ] HYAC: Data is stale - most recent date (2025-09-11) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:53:35.897266] [FATAL] [MAIN ] HYAC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:53:35.897321] [WARN ] [MAIN ] HYAC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:53:35.897383] [INFO ] [LOCAL ] HYAC: Starting deletion of primary data. [2025-09-16T18:53:35.897470] [DEBUG] [LOCAL ] HYAC: No primary data file found to delete. [2025-09-16T18:53:35.897529] [INFO ] [LOCAL ] HYAC: Starting deletion of derivative data. [2025-09-16T18:53:35.897630] [DEBUG] [LOCAL ] HYAC: No derivative data file found to delete. [2025-09-16T18:53:35.897751] [INFO ] [MAIN ] Update Progress: 1072/2369 symbols completed. HYAC took 16.6s. Estimated remaining: 2h 17m 23s. Rate: 9.4 symbols/min [2025-09-16T18:53:36.080761] [INFO ] [MAIN ] HYT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:36.106905] [DEBUG] [LOCAL ] HYT: Loaded 24150 existing records from primary storage. [2025-09-16T18:53:36.107040] [INFO ] [MAIN ] HYT: Loaded 24150 existing records. [2025-09-16T18:53:36.283712] [WARN ] [MAIN ] HYT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:53:36.283945] [INFO ] [MAIN ] HYT: Wiping existing primary data before full refresh. [2025-09-16T18:53:36.283993] [INFO ] [LOCAL ] HYT: Starting deletion of primary data. [2025-09-16T18:53:36.284622] [DEBUG] [LOCAL ] HYT: Deleted primary data file. [2025-09-16T18:53:36.284699] [INFO ] [MAIN ] HYT: Starting full primary refresh. [2025-09-16T18:53:36.383128] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-06. [2025-09-16T18:53:36.639711] [INFO ] [REMOTE ] HYT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:36.720221] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-09-16T18:53:37.174914] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-04. [2025-09-16T18:53:37.974904] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-07. [2025-09-16T18:53:38.010412] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-09-16T18:53:38.281520] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-07 (0.31 seconds). [2025-09-16T18:53:38.774923] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-05. [2025-09-16T18:53:39.410051] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-09-16T18:53:39.574903] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-06. [2025-09-16T18:53:40.358443] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T18:53:40.374852] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-07. [2025-09-16T18:53:41.046072] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-09-16T18:53:41.174891] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-02. [2025-09-16T18:53:41.974870] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-08. [2025-09-16T18:53:42.434683] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-09-16T18:53:42.567524] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-09-16T18:53:42.774889] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-03. [2025-09-16T18:53:43.574879] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-08. [2025-09-16T18:53:43.893382] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-09-16T18:53:44.010499] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-09-16T18:53:44.374912] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-09. [2025-09-16T18:53:44.820199] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-09-16T18:53:44.820888] [INFO ] [REMOTE ] HYBX: Full fetch complete (487 records). [2025-09-16T18:53:44.820954] [DEBUG] [VALIDATOR ] HYBX: Validating primary data. [2025-09-16T18:53:45.007346] [WARN ] [VALIDATOR ] HYBX: Validation failed - insufficient historical data. Have 134 unique days, need 150 days [2025-09-16T18:53:45.007562] [FATAL] [MAIN ] HYBX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:53:45.007620] [WARN ] [MAIN ] HYBX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:53:45.007752] [INFO ] [LOCAL ] HYBX: Starting deletion of primary data. [2025-09-16T18:53:45.007843] [DEBUG] [LOCAL ] HYBX: No primary data file found to delete. [2025-09-16T18:53:45.007890] [INFO ] [LOCAL ] HYBX: Starting deletion of derivative data. [2025-09-16T18:53:45.007949] [DEBUG] [LOCAL ] HYBX: No derivative data file found to delete. [2025-09-16T18:53:45.008028] [INFO ] [MAIN ] Update Progress: 1073/2369 symbols completed. HYBX took 20.8s. Estimated remaining: 2h 17m 20s. Rate: 9.4 symbols/min [2025-09-16T18:53:45.180472] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-09. [2025-09-16T18:53:45.183610] [INFO ] [MAIN ] HYTR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:45.188449] [DEBUG] [LOCAL ] HYTR: Loaded 2215 existing records from primary storage. [2025-09-16T18:53:45.188689] [INFO ] [MAIN ] HYTR: Loaded 2215 existing records. [2025-09-16T18:53:45.375380] [WARN ] [MAIN ] HYTR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:53:45.375666] [INFO ] [MAIN ] HYTR: Wiping existing primary data before full refresh. [2025-09-16T18:53:45.375720] [INFO ] [LOCAL ] HYTR: Starting deletion of primary data. [2025-09-16T18:53:45.376116] [DEBUG] [LOCAL ] HYTR: Deleted primary data file. [2025-09-16T18:53:45.376165] [INFO ] [MAIN ] HYTR: Starting full primary refresh. [2025-09-16T18:53:45.744514] [INFO ] [REMOTE ] HYTR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:45.842809] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-09-16T18:53:45.844080] [INFO ] [REMOTE ] HYI: Full fetch complete (9678 records). [2025-09-16T18:53:45.844171] [DEBUG] [VALIDATOR ] HYI: Validating primary data. [2025-09-16T18:53:45.980035] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-04. [2025-09-16T18:53:46.219372] [DEBUG] [VALIDATOR ] HYI: All expected trading days are present. [2025-09-16T18:53:46.219449] [DEBUG] [VALIDATOR ] HYI: Primary data validation passed. [2025-09-16T18:53:46.219714] [INFO ] [MAIN ] HYI: Fetched primary data is valid. [2025-09-16T18:53:46.219773] [DEBUG] [LOCAL ] HYI: Saving primary data. [2025-09-16T18:53:46.220213] [DEBUG] [LOCAL ] HYI: Trimmed 9678 records to 6467 (keeping 150 days of data) [2025-09-16T18:53:46.259923] [DEBUG] [LOCAL ] HYI: Primary data saved (6467 records). [2025-09-16T18:53:46.260029] [INFO ] [MAIN ] HYI: Full history saved. [2025-09-16T18:53:46.260121] [INFO ] [MAIN ] Update Progress: 1074/2369 symbols completed. HYI took 20.0s. Estimated remaining: 2h 17m 7s. Rate: 9.4 symbols/min [2025-09-16T18:53:46.429872] [INFO ] [MAIN ] HZO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:46.449246] [DEBUG] [LOCAL ] HZO: Loaded 20526 existing records from primary storage. [2025-09-16T18:53:46.449370] [INFO ] [MAIN ] HZO: Loaded 20526 existing records. [2025-09-16T18:53:46.629767] [WARN ] [MAIN ] HZO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:53:46.630032] [INFO ] [MAIN ] HZO: Wiping existing primary data before full refresh. [2025-09-16T18:53:46.630089] [INFO ] [LOCAL ] HZO: Starting deletion of primary data. [2025-09-16T18:53:46.630691] [DEBUG] [LOCAL ] HZO: Deleted primary data file. [2025-09-16T18:53:46.630742] [INFO ] [MAIN ] HZO: Starting full primary refresh. [2025-09-16T18:53:46.775968] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-02. [2025-09-16T18:53:46.986254] [INFO ] [REMOTE ] HZO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:47.462710] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T18:53:47.531293] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-09-16T18:53:47.574875] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-05. [2025-09-16T18:53:48.374904] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-03. [2025-09-16T18:53:48.925116] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T18:53:48.935437] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-09-16T18:53:49.174932] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-06. [2025-09-16T18:53:49.974871] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-02. [2025-09-16T18:53:50.422452] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T18:53:50.774974] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-07. [2025-09-16T18:53:50.803419] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-09-16T18:53:51.574928] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-03. [2025-09-16T18:53:52.216579] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T18:53:52.374925] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-08. [2025-09-16T18:53:52.441381] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-09-16T18:53:53.174917] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-04. [2025-09-16T18:53:53.491513] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T18:53:53.729861] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T18:53:53.974895] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-08. [2025-09-16T18:53:54.774939] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-09. [2025-09-16T18:53:55.452346] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T18:53:55.574867] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-09. [2025-09-16T18:53:55.835937] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-09-16T18:53:55.839925] [INFO ] [REMOTE ] HYT: Full fetch complete (36032 records). [2025-09-16T18:53:55.840011] [DEBUG] [VALIDATOR ] HYT: Validating primary data. [2025-09-16T18:53:56.227438] [DEBUG] [VALIDATOR ] HYT: All expected trading days are present. [2025-09-16T18:53:56.227525] [DEBUG] [VALIDATOR ] HYT: Primary data validation passed. [2025-09-16T18:53:56.228240] [INFO ] [MAIN ] HYT: Fetched primary data is valid. [2025-09-16T18:53:56.228302] [DEBUG] [LOCAL ] HYT: Saving primary data. [2025-09-16T18:53:56.229234] [DEBUG] [LOCAL ] HYT: Trimmed 36032 records to 24402 (keeping 150 days of data) [2025-09-16T18:53:56.378226] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-04. [2025-09-16T18:53:56.387379] [DEBUG] [LOCAL ] HYT: Primary data saved (24402 records). [2025-09-16T18:53:56.387503] [INFO ] [MAIN ] HYT: Full history saved. [2025-09-16T18:53:56.387599] [INFO ] [MAIN ] Update Progress: 1075/2369 symbols completed. HYT took 20.5s. Estimated remaining: 2h 17m 5s. Rate: 9.4 symbols/min [2025-09-16T18:53:56.563933] [INFO ] [MAIN ] IAE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:56.570188] [DEBUG] [LOCAL ] IAE: Loaded 4215 existing records from primary storage. [2025-09-16T18:53:56.570333] [INFO ] [MAIN ] IAE: Loaded 4215 existing records. [2025-09-16T18:53:56.752171] [WARN ] [MAIN ] IAE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:53:56.752414] [INFO ] [MAIN ] IAE: Wiping existing primary data before full refresh. [2025-09-16T18:53:56.752464] [INFO ] [LOCAL ] IAE: Starting deletion of primary data. [2025-09-16T18:53:56.752711] [DEBUG] [LOCAL ] IAE: Deleted primary data file. [2025-09-16T18:53:56.752758] [INFO ] [MAIN ] IAE: Starting full primary refresh. [2025-09-16T18:53:57.126016] [INFO ] [REMOTE ] IAE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:57.141202] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T18:53:57.146800] [INFO ] [REMOTE ] HXL: Full fetch complete (50044 records). [2025-09-16T18:53:57.146930] [DEBUG] [VALIDATOR ] HXL: Validating primary data. [2025-09-16T18:53:57.179981] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-02. [2025-09-16T18:53:57.536718] [DEBUG] [VALIDATOR ] HXL: All expected trading days are present. [2025-09-16T18:53:57.536805] [DEBUG] [VALIDATOR ] HXL: Primary data validation passed. [2025-09-16T18:53:57.537623] [INFO ] [MAIN ] HXL: Fetched primary data is valid. [2025-09-16T18:53:57.537703] [DEBUG] [LOCAL ] HXL: Saving primary data. [2025-09-16T18:53:57.538822] [DEBUG] [LOCAL ] HXL: Trimmed 50044 records to 34210 (keeping 150 days of data) [2025-09-16T18:53:57.752844] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-09-16T18:53:57.783291] [DEBUG] [LOCAL ] HXL: Primary data saved (34210 records). [2025-09-16T18:53:57.783468] [INFO ] [MAIN ] HXL: Full history saved. [2025-09-16T18:53:57.783580] [INFO ] [MAIN ] Update Progress: 1076/2369 symbols completed. HXL took 53.8s. Estimated remaining: 2h 16m 53s. Rate: 9.4 symbols/min [2025-09-16T18:53:57.911164] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-09-16T18:53:57.968202] [INFO ] [MAIN ] IAG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:53:57.974867] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-03. [2025-09-16T18:53:58.006016] [DEBUG] [LOCAL ] IAG: Loaded 39458 existing records from primary storage. [2025-09-16T18:53:58.006160] [INFO ] [MAIN ] IAG: Loaded 39458 existing records. [2025-09-16T18:53:58.187974] [WARN ] [MAIN ] IAG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:53:58.188244] [INFO ] [MAIN ] IAG: Wiping existing primary data before full refresh. [2025-09-16T18:53:58.188292] [INFO ] [LOCAL ] IAG: Starting deletion of primary data. [2025-09-16T18:53:58.189163] [DEBUG] [LOCAL ] IAG: Deleted primary data file. [2025-09-16T18:53:58.189215] [INFO ] [MAIN ] IAG: Starting full primary refresh. [2025-09-16T18:53:58.541634] [INFO ] [REMOTE ] IAG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:53:58.634371] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-09-16T18:53:58.774923] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-04. [2025-09-16T18:53:59.411358] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-09-16T18:53:59.574869] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-05. [2025-09-16T18:54:00.170002] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-09-16T18:54:00.374957] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-06. [2025-09-16T18:54:01.174919] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-02. [2025-09-16T18:54:01.492246] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T18:54:01.974950] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-07. [2025-09-16T18:54:02.715265] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-09-16T18:54:02.779977] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-05. [2025-09-16T18:54:02.799504] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T18:54:03.574864] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-03. [2025-09-16T18:54:03.903669] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T18:54:04.374903] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-05. [2025-09-16T18:54:04.853526] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-09-16T18:54:05.174943] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-06. [2025-09-16T18:54:05.306628] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T18:54:05.974866] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-04. [2025-09-16T18:54:06.384420] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T18:54:06.774950] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-07. [2025-09-16T18:54:07.426146] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T18:54:07.574898] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-05. [2025-09-16T18:54:08.068745] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-09-16T18:54:08.374965] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-06. [2025-09-16T18:54:08.810244] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-09-16T18:54:09.174967] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-08. [2025-09-16T18:54:09.297220] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-09-16T18:54:09.777625] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T18:54:09.974891] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-06. [2025-09-16T18:54:10.774924] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-07. [2025-09-16T18:54:11.310543] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-09-16T18:54:11.574922] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-08. [2025-09-16T18:54:11.718035] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T18:54:12.374930] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-07. [2025-09-16T18:54:12.729219] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T18:54:13.174891] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-08. [2025-09-16T18:54:13.642731] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-09-16T18:54:13.974901] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-09. [2025-09-16T18:54:14.188979] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T18:54:14.774874] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-08. [2025-09-16T18:54:15.128485] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T18:54:15.131795] [INFO ] [REMOTE ] HZO: Full fetch complete (31518 records). [2025-09-16T18:54:15.131878] [DEBUG] [VALIDATOR ] HZO: Validating primary data. [2025-09-16T18:54:15.515516] [DEBUG] [VALIDATOR ] HZO: All expected trading days are present. [2025-09-16T18:54:15.515596] [DEBUG] [VALIDATOR ] HZO: Primary data validation passed. [2025-09-16T18:54:15.516202] [INFO ] [MAIN ] HZO: Fetched primary data is valid. [2025-09-16T18:54:15.516247] [DEBUG] [LOCAL ] HZO: Saving primary data. [2025-09-16T18:54:15.517080] [DEBUG] [LOCAL ] HZO: Trimmed 31518 records to 20719 (keeping 150 days of data) [2025-09-16T18:54:15.576912] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-09. [2025-09-16T18:54:15.642361] [DEBUG] [LOCAL ] HZO: Primary data saved (20719 records). [2025-09-16T18:54:15.642491] [INFO ] [MAIN ] HZO: Full history saved. [2025-09-16T18:54:15.642580] [INFO ] [MAIN ] Update Progress: 1077/2369 symbols completed. HZO took 29.4s. Estimated remaining: 2h 17m 1s. Rate: 9.4 symbols/min [2025-09-16T18:54:16.012412] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-09-16T18:54:16.013337] [INFO ] [REMOTE ] HYTR: Full fetch complete (3465 records). [2025-09-16T18:54:16.013404] [DEBUG] [VALIDATOR ] HYTR: Validating primary data. [2025-09-16T18:54:16.377384] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-09. [2025-09-16T18:54:16.411519] [DEBUG] [VALIDATOR ] HYTR: All expected trading days are present. [2025-09-16T18:54:16.411589] [DEBUG] [VALIDATOR ] HYTR: Primary data validation passed. [2025-09-16T18:54:16.411725] [INFO ] [MAIN ] HYTR: Fetched primary data is valid. [2025-09-16T18:54:16.411849] [DEBUG] [LOCAL ] HYTR: Saving primary data. [2025-09-16T18:54:16.412310] [DEBUG] [LOCAL ] HYTR: Trimmed 3465 records to 2235 (keeping 150 days of data) [2025-09-16T18:54:16.428082] [DEBUG] [LOCAL ] HYTR: Primary data saved (2235 records). [2025-09-16T18:54:16.428280] [INFO ] [MAIN ] HYTR: Full history saved. [2025-09-16T18:54:16.428366] [INFO ] [MAIN ] Update Progress: 1078/2369 symbols completed. HYTR took 31.4s. Estimated remaining: 2h 16m 47s. Rate: 9.4 symbols/min [2025-09-16T18:54:16.459830] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T18:54:17.118157] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-09-16T18:54:17.119486] [INFO ] [REMOTE ] IAE: Full fetch complete (6375 records). [2025-09-16T18:54:17.119558] [DEBUG] [VALIDATOR ] IAE: Validating primary data. [2025-09-16T18:54:17.184982] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-09. [2025-09-16T18:54:17.492705] [DEBUG] [VALIDATOR ] IAE: All expected trading days are present. [2025-09-16T18:54:17.492794] [DEBUG] [VALIDATOR ] IAE: Primary data validation passed. [2025-09-16T18:54:17.493064] [INFO ] [MAIN ] IAE: Fetched primary data is valid. [2025-09-16T18:54:17.493106] [DEBUG] [LOCAL ] IAE: Saving primary data. [2025-09-16T18:54:17.493551] [DEBUG] [LOCAL ] IAE: Trimmed 6375 records to 4267 (keeping 150 days of data) [2025-09-16T18:54:17.519696] [DEBUG] [LOCAL ] IAE: Primary data saved (4267 records). [2025-09-16T18:54:17.519812] [INFO ] [MAIN ] IAE: Full history saved. [2025-09-16T18:54:17.519930] [INFO ] [MAIN ] Update Progress: 1079/2369 symbols completed. IAE took 21.1s. Estimated remaining: 2h 16m 35s. Rate: 9.4 symbols/min [2025-09-16T18:54:19.097941] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-09-16T18:54:19.106348] [INFO ] [REMOTE ] IAG: Full fetch complete (58099 records). [2025-09-16T18:54:19.106555] [DEBUG] [VALIDATOR ] IAG: Validating primary data. [2025-09-16T18:54:19.505280] [DEBUG] [VALIDATOR ] IAG: All expected trading days are present. [2025-09-16T18:54:19.505392] [DEBUG] [VALIDATOR ] IAG: Primary data validation passed. [2025-09-16T18:54:19.506290] [INFO ] [MAIN ] IAG: Fetched primary data is valid. [2025-09-16T18:54:19.506348] [DEBUG] [LOCAL ] IAG: Saving primary data. [2025-09-16T18:54:19.507897] [DEBUG] [LOCAL ] IAG: Trimmed 58099 records to 39848 (keeping 150 days of data) [2025-09-16T18:54:19.743793] [DEBUG] [LOCAL ] IAG: Primary data saved (39848 records). [2025-09-16T18:54:19.743977] [INFO ] [MAIN ] IAG: Full history saved. [2025-09-16T18:54:19.744089] [INFO ] [MAIN ] Update Progress: 1080/2369 symbols completed. IAG took 22.0s. Estimated remaining: 2h 16m 24s. Rate: 9.5 symbols/min [2025-09-16T18:54:20.362113] [INFO ] [MAIN ] IBN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:54:20.389001] [INFO ] [MAIN ] IBP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:54:20.389821] [INFO ] [MAIN ] IBM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:54:20.440632] [INFO ] [MAIN ] IBTA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:54:20.467826] [DEBUG] [LOCAL ] IBP: Loaded 21527 existing records from primary storage. [2025-09-16T18:54:20.468170] [INFO ] [MAIN ] IBP: Loaded 21527 existing records. [2025-09-16T18:54:20.606352] [DEBUG] [LOCAL ] IBTA: Loaded 25982 existing records from primary storage. [2025-09-16T18:54:20.662263] [INFO ] [MAIN ] IBTA: Loaded 25982 existing records. [2025-09-16T18:54:20.735157] [DEBUG] [LOCAL ] IBN: Loaded 38999 existing records from primary storage. [2025-09-16T18:54:20.740480] [INFO ] [MAIN ] IBN: Loaded 38999 existing records. [2025-09-16T18:54:20.741070] [DEBUG] [LOCAL ] IBM: Loaded 39615 existing records from primary storage. [2025-09-16T18:54:20.699976] [WARN ] [MAIN ] IBP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:54:20.751591] [INFO ] [MAIN ] IBM: Loaded 39615 existing records. [2025-09-16T18:54:20.785236] [INFO ] [MAIN ] IBP: Wiping existing primary data before full refresh. [2025-09-16T18:54:20.886838] [INFO ] [LOCAL ] IBP: Starting deletion of primary data. [2025-09-16T18:54:20.950047] [DEBUG] [LOCAL ] IBP: Deleted primary data file. [2025-09-16T18:54:20.980084] [INFO ] [MAIN ] IBP: Starting full primary refresh. [2025-09-16T18:54:21.279627] [WARN ] [MAIN ] IBTA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:54:21.332134] [WARN ] [MAIN ] IBN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:54:21.337843] [WARN ] [MAIN ] IBM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:54:21.359241] [INFO ] [MAIN ] IBN: Wiping existing primary data before full refresh. [2025-09-16T18:54:21.359460] [INFO ] [LOCAL ] IBN: Starting deletion of primary data. [2025-09-16T18:54:21.370348] [DEBUG] [LOCAL ] IBN: Deleted primary data file. [2025-09-16T18:54:21.359599] [INFO ] [MAIN ] IBTA: Wiping existing primary data before full refresh. [2025-09-16T18:54:21.380699] [INFO ] [LOCAL ] IBTA: Starting deletion of primary data. [2025-09-16T18:54:21.385924] [INFO ] [MAIN ] IBN: Starting full primary refresh. [2025-09-16T18:54:21.370312] [INFO ] [MAIN ] IBM: Wiping existing primary data before full refresh. [2025-09-16T18:54:21.429322] [INFO ] [LOCAL ] IBM: Starting deletion of primary data. [2025-09-16T18:54:21.429258] [DEBUG] [LOCAL ] IBTA: Deleted primary data file. [2025-09-16T18:54:21.462192] [INFO ] [MAIN ] IBTA: Starting full primary refresh. [2025-09-16T18:54:21.462107] [DEBUG] [LOCAL ] IBM: Deleted primary data file. [2025-09-16T18:54:21.591359] [INFO ] [MAIN ] IBM: Starting full primary refresh. [2025-09-16T18:54:22.101529] [INFO ] [REMOTE ] IBP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:54:22.162488] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-02. [2025-09-16T18:54:22.577781] [INFO ] [REMOTE ] IBN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:54:22.581732] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-02. [2025-09-16T18:54:22.737510] [INFO ] [REMOTE ] IBM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:54:22.741846] [INFO ] [REMOTE ] IBTA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:54:22.742596] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-02. [2025-09-16T18:54:22.742304] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-02. [2025-09-16T18:54:23.803735] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:54:23.809425] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-03. [2025-09-16T18:54:23.907360] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-09-16T18:54:23.912315] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-03. [2025-09-16T18:54:24.084349] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-09-16T18:54:24.094092] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-03. [2025-09-16T18:54:24.608328] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-09-16T18:54:24.618607] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-03. [2025-09-16T18:54:25.095109] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-09-16T18:54:25.101755] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-04. [2025-09-16T18:54:25.537886] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T18:54:25.548045] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-04. [2025-09-16T18:54:25.701784] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T18:54:25.974875] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-04. [2025-09-16T18:54:26.283261] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T18:54:26.322265] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-09-16T18:54:26.774914] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-04. [2025-09-16T18:54:27.137372] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-09-16T18:54:27.238902] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-09-16T18:54:27.574886] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-05. [2025-09-16T18:54:28.374910] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-05. [2025-09-16T18:54:29.092445] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-04 (2.32 seconds). [2025-09-16T18:54:29.174865] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-05. [2025-09-16T18:54:29.451997] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-09-16T18:54:29.643094] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T18:54:29.974908] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-05. [2025-09-16T18:54:30.451819] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-09-16T18:54:30.774956] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-06. [2025-09-16T18:54:31.574894] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-06. [2025-09-16T18:54:31.972913] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-09-16T18:54:32.031438] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T18:54:32.374953] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-07. [2025-09-16T18:54:33.174913] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-06. [2025-09-16T18:54:33.275600] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T18:54:33.667674] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-09-16T18:54:33.974917] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-07. [2025-09-16T18:54:34.429114] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T18:54:34.774951] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-06. [2025-09-16T18:54:35.574913] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-08. [2025-09-16T18:54:35.886996] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T18:54:36.374906] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-08. [2025-09-16T18:54:36.722760] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T18:54:36.838548] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-09-16T18:54:37.174896] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-07. [2025-09-16T18:54:37.974888] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-07. [2025-09-16T18:54:38.190548] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T18:54:38.774910] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-09. [2025-09-16T18:54:38.920625] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-09-16T18:54:39.584696] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-09. [2025-09-16T18:54:39.606608] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-07 (2.43 seconds). [2025-09-16T18:54:39.664081] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T18:54:39.668156] [INFO ] [REMOTE ] IBP: Full fetch complete (32122 records). [2025-09-16T18:54:39.668331] [DEBUG] [VALIDATOR ] IBP: Validating primary data. [2025-09-16T18:54:40.072557] [DEBUG] [VALIDATOR ] IBP: All expected trading days are present. [2025-09-16T18:54:40.072636] [DEBUG] [VALIDATOR ] IBP: Primary data validation passed. [2025-09-16T18:54:40.073424] [INFO ] [MAIN ] IBP: Fetched primary data is valid. [2025-09-16T18:54:40.073496] [DEBUG] [LOCAL ] IBP: Saving primary data. [2025-09-16T18:54:40.074296] [DEBUG] [LOCAL ] IBP: Trimmed 32122 records to 21669 (keeping 150 days of data) [2025-09-16T18:54:40.219833] [DEBUG] [LOCAL ] IBP: Primary data saved (21669 records). [2025-09-16T18:54:40.219969] [INFO ] [MAIN ] IBP: Full history saved. [2025-09-16T18:54:40.220097] [INFO ] [MAIN ] Update Progress: 1081/2369 symbols completed. IBP took 20.5s. Estimated remaining: 2h 16m 34s. Rate: 9.4 symbols/min [2025-09-16T18:54:40.386998] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-08. [2025-09-16T18:54:40.390121] [INFO ] [MAIN ] ICE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:54:40.433687] [DEBUG] [LOCAL ] ICE: Loaded 39028 existing records from primary storage. [2025-09-16T18:54:40.433823] [INFO ] [MAIN ] ICE: Loaded 39028 existing records. [2025-09-16T18:54:40.609742] [WARN ] [MAIN ] ICE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:54:40.609985] [INFO ] [MAIN ] ICE: Wiping existing primary data before full refresh. [2025-09-16T18:54:40.610038] [INFO ] [LOCAL ] ICE: Starting deletion of primary data. [2025-09-16T18:54:40.610800] [DEBUG] [LOCAL ] ICE: Deleted primary data file. [2025-09-16T18:54:40.610889] [INFO ] [MAIN ] ICE: Starting full primary refresh. [2025-09-16T18:54:40.964323] [INFO ] [REMOTE ] ICE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:54:41.174885] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-02. [2025-09-16T18:54:41.278789] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-09-16T18:54:41.285445] [INFO ] [REMOTE ] IBN: Full fetch complete (57536 records). [2025-09-16T18:54:41.285550] [DEBUG] [VALIDATOR ] IBN: Validating primary data. [2025-09-16T18:54:41.661768] [DEBUG] [VALIDATOR ] IBN: All expected trading days are present. [2025-09-16T18:54:41.661844] [DEBUG] [VALIDATOR ] IBN: Primary data validation passed. [2025-09-16T18:54:41.662593] [INFO ] [MAIN ] IBN: Fetched primary data is valid. [2025-09-16T18:54:41.662634] [DEBUG] [LOCAL ] IBN: Saving primary data. [2025-09-16T18:54:41.663723] [DEBUG] [LOCAL ] IBN: Trimmed 57536 records to 39380 (keeping 150 days of data) [2025-09-16T18:54:41.909555] [DEBUG] [LOCAL ] IBN: Primary data saved (39380 records). [2025-09-16T18:54:41.909707] [INFO ] [MAIN ] IBN: Full history saved. [2025-09-16T18:54:41.909814] [INFO ] [MAIN ] Update Progress: 1082/2369 symbols completed. IBN took 22.2s. Estimated remaining: 2h 16m 22s. Rate: 9.4 symbols/min [2025-09-16T18:54:41.980020] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-08. [2025-09-16T18:54:42.084253] [INFO ] [MAIN ] ICL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:54:42.109918] [DEBUG] [LOCAL ] ICL: Loaded 28261 existing records from primary storage. [2025-09-16T18:54:42.110075] [INFO ] [MAIN ] ICL: Loaded 28261 existing records. [2025-09-16T18:54:42.289329] [WARN ] [MAIN ] ICL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:54:42.289570] [INFO ] [MAIN ] ICL: Wiping existing primary data before full refresh. [2025-09-16T18:54:42.289625] [INFO ] [LOCAL ] ICL: Starting deletion of primary data. [2025-09-16T18:54:42.290303] [DEBUG] [LOCAL ] ICL: Deleted primary data file. [2025-09-16T18:54:42.290353] [INFO ] [MAIN ] ICL: Starting full primary refresh. [2025-09-16T18:54:42.640418] [INFO ] [REMOTE ] ICL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:54:42.774850] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-02. [2025-09-16T18:54:42.857534] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-08 (2.47 seconds). [2025-09-16T18:54:43.003336] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T18:54:43.574876] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-03. [2025-09-16T18:54:43.697431] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-09-16T18:54:43.960220] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T18:54:44.374949] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-09. [2025-09-16T18:54:45.022759] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T18:54:45.174858] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-04. [2025-09-16T18:54:45.974911] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-09. [2025-09-16T18:54:46.655607] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-09 (2.28 seconds). [2025-09-16T18:54:46.661880] [INFO ] [REMOTE ] IBM: Full fetch complete (58278 records). [2025-09-16T18:54:46.661987] [DEBUG] [VALIDATOR ] IBM: Validating primary data. [2025-09-16T18:54:46.784737] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-03. [2025-09-16T18:54:47.055275] [DEBUG] [VALIDATOR ] IBM: All expected trading days are present. [2025-09-16T18:54:47.055441] [DEBUG] [VALIDATOR ] IBM: Primary data validation passed. [2025-09-16T18:54:47.056608] [INFO ] [MAIN ] IBM: Fetched primary data is valid. [2025-09-16T18:54:47.056769] [DEBUG] [LOCAL ] IBM: Saving primary data. [2025-09-16T18:54:47.072431] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-09-16T18:54:47.090223] [DEBUG] [LOCAL ] IBM: Trimmed 58278 records to 40001 (keeping 150 days of data) [2025-09-16T18:54:47.106267] [INFO ] [REMOTE ] IBTA: Full fetch complete (40635 records). [2025-09-16T18:54:47.109934] [DEBUG] [VALIDATOR ] IBTA: Validating primary data. [2025-09-16T18:54:47.113094] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-09-16T18:54:47.588022] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-05. [2025-09-16T18:54:47.646309] [DEBUG] [VALIDATOR ] IBTA: All expected trading days are present. [2025-09-16T18:54:47.646674] [DEBUG] [VALIDATOR ] IBTA: Primary data validation passed. [2025-09-16T18:54:47.648261] [INFO ] [MAIN ] IBTA: Fetched primary data is valid. [2025-09-16T18:54:47.649191] [DEBUG] [LOCAL ] IBTA: Saving primary data. [2025-09-16T18:54:47.652991] [DEBUG] [LOCAL ] IBTA: Trimmed 40635 records to 26290 (keeping 150 days of data) [2025-09-16T18:54:47.923142] [DEBUG] [LOCAL ] IBTA: Primary data saved (26290 records). [2025-09-16T18:54:47.924216] [INFO ] [MAIN ] IBTA: Full history saved. [2025-09-16T18:54:47.924580] [INFO ] [MAIN ] Update Progress: 1083/2369 symbols completed. IBTA took 28.2s. Estimated remaining: 2h 16m 15s. Rate: 9.4 symbols/min [2025-09-16T18:54:48.106283] [INFO ] [MAIN ] IDA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:54:48.142431] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T18:54:48.170868] [DEBUG] [LOCAL ] IDA: Loaded 24568 existing records from primary storage. [2025-09-16T18:54:48.172619] [INFO ] [MAIN ] IDA: Loaded 24568 existing records. [2025-09-16T18:54:48.336265] [DEBUG] [LOCAL ] IBM: Primary data saved (40001 records). [2025-09-16T18:54:48.339241] [INFO ] [MAIN ] IBM: Full history saved. [2025-09-16T18:54:48.339663] [INFO ] [MAIN ] Update Progress: 1084/2369 symbols completed. IBM took 28.6s. Estimated remaining: 2h 16m 2s. Rate: 9.4 symbols/min [2025-09-16T18:54:48.380077] [WARN ] [MAIN ] IDA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:54:48.379952] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-04. [2025-09-16T18:54:48.423380] [INFO ] [MAIN ] IDA: Wiping existing primary data before full refresh. [2025-09-16T18:54:48.460048] [INFO ] [LOCAL ] IDA: Starting deletion of primary data. [2025-09-16T18:54:48.468273] [DEBUG] [LOCAL ] IDA: Deleted primary data file. [2025-09-16T18:54:48.483692] [INFO ] [MAIN ] IDA: Starting full primary refresh. [2025-09-16T18:54:48.627462] [INFO ] [MAIN ] IDE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:54:48.755338] [DEBUG] [LOCAL ] IDE: Loaded 6534 existing records from primary storage. [2025-09-16T18:54:48.792779] [INFO ] [MAIN ] IDE: Loaded 6534 existing records. [2025-09-16T18:54:49.027236] [INFO ] [REMOTE ] IDA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:54:49.084301] [WARN ] [MAIN ] IDE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:54:49.084560] [INFO ] [MAIN ] IDE: Wiping existing primary data before full refresh. [2025-09-16T18:54:49.084618] [INFO ] [LOCAL ] IDE: Starting deletion of primary data. [2025-09-16T18:54:49.084979] [DEBUG] [LOCAL ] IDE: Deleted primary data file. [2025-09-16T18:54:49.085026] [INFO ] [MAIN ] IDE: Starting full primary refresh. [2025-09-16T18:54:49.183917] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-02. [2025-09-16T18:54:49.445718] [INFO ] [REMOTE ] IDE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:54:49.506806] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-09-16T18:54:49.507929] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-09-16T18:54:49.974951] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-05. [2025-09-16T18:54:50.105780] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-09-16T18:54:50.774905] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-03. [2025-09-16T18:54:51.449128] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T18:54:51.574942] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-06. [2025-09-16T18:54:51.738362] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-09-16T18:54:52.374906] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-06. [2025-09-16T18:54:53.174915] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-04. [2025-09-16T18:54:53.317822] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T18:54:53.386612] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T18:54:53.974903] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-02. [2025-09-16T18:54:54.170660] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-09-16T18:54:54.674140] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-09-16T18:54:54.774958] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-05. [2025-09-16T18:54:55.574957] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-07. [2025-09-16T18:54:56.245204] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-09-16T18:54:56.374874] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-06. [2025-09-16T18:54:56.981978] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-09-16T18:54:57.174891] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-03. [2025-09-16T18:54:57.700465] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-09-16T18:54:57.722701] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T18:54:57.974898] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-07. [2025-09-16T18:54:58.774908] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-07. [2025-09-16T18:54:59.487914] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T18:54:59.574931] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-08. [2025-09-16T18:55:00.252221] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T18:55:00.374954] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-08. [2025-09-16T18:55:00.977200] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-09-16T18:55:01.174996] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-08. [2025-09-16T18:55:01.715481] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T18:55:01.974908] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-09. [2025-09-16T18:55:02.774908] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-04. [2025-09-16T18:55:02.920666] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T18:55:02.968673] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T18:55:02.973422] [INFO ] [REMOTE ] ICL: Full fetch complete (42366 records). [2025-09-16T18:55:02.973538] [DEBUG] [VALIDATOR ] ICL: Validating primary data. [2025-09-16T18:55:03.357606] [DEBUG] [VALIDATOR ] ICL: All expected trading days are present. [2025-09-16T18:55:03.357707] [DEBUG] [VALIDATOR ] ICL: Primary data validation passed. [2025-09-16T18:55:03.358441] [INFO ] [MAIN ] ICL: Fetched primary data is valid. [2025-09-16T18:55:03.358493] [DEBUG] [LOCAL ] ICL: Saving primary data. [2025-09-16T18:55:03.359494] [DEBUG] [LOCAL ] ICL: Trimmed 42366 records to 28534 (keeping 150 days of data) [2025-09-16T18:55:03.526490] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-09-16T18:55:03.533599] [DEBUG] [LOCAL ] ICL: Primary data saved (28534 records). [2025-09-16T18:55:03.534066] [INFO ] [MAIN ] ICL: Full history saved. [2025-09-16T18:55:03.534165] [INFO ] [MAIN ] Update Progress: 1085/2369 symbols completed. ICL took 21.6s. Estimated remaining: 2h 16m 6s. Rate: 9.4 symbols/min [2025-09-16T18:55:03.579952] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-09. [2025-09-16T18:55:03.709233] [INFO ] [MAIN ] IDT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:55:03.723473] [DEBUG] [LOCAL ] IDT: Loaded 13084 existing records from primary storage. [2025-09-16T18:55:03.723612] [INFO ] [MAIN ] IDT: Loaded 13084 existing records. [2025-09-16T18:55:03.917929] [WARN ] [MAIN ] IDT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:55:03.918185] [INFO ] [MAIN ] IDT: Wiping existing primary data before full refresh. [2025-09-16T18:55:03.918232] [INFO ] [LOCAL ] IDT: Starting deletion of primary data. [2025-09-16T18:55:03.918616] [DEBUG] [LOCAL ] IDT: Deleted primary data file. [2025-09-16T18:55:03.918686] [INFO ] [MAIN ] IDT: Starting full primary refresh. [2025-09-16T18:55:04.276991] [INFO ] [REMOTE ] IDT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:55:04.374912] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-05. [2025-09-16T18:55:04.608614] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-09-16T18:55:04.611984] [INFO ] [REMOTE ] IDA: Full fetch complete (34568 records). [2025-09-16T18:55:04.612083] [DEBUG] [VALIDATOR ] IDA: Validating primary data. [2025-09-16T18:55:04.940180] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-09-16T18:55:05.005576] [DEBUG] [VALIDATOR ] IDA: All expected trading days are present. [2025-09-16T18:55:05.005678] [DEBUG] [VALIDATOR ] IDA: Primary data validation passed. [2025-09-16T18:55:05.006386] [INFO ] [MAIN ] IDA: Fetched primary data is valid. [2025-09-16T18:55:05.006429] [DEBUG] [LOCAL ] IDA: Saving primary data. [2025-09-16T18:55:05.007261] [DEBUG] [LOCAL ] IDA: Trimmed 34568 records to 24824 (keeping 150 days of data) [2025-09-16T18:55:05.165319] [DEBUG] [LOCAL ] IDA: Primary data saved (24824 records). [2025-09-16T18:55:05.165467] [INFO ] [MAIN ] IDA: Full history saved. [2025-09-16T18:55:05.165565] [INFO ] [MAIN ] Update Progress: 1086/2369 symbols completed. IDA took 17.2s. Estimated remaining: 2h 15m 54s. Rate: 9.4 symbols/min [2025-09-16T18:55:05.180012] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-09. [2025-09-16T18:55:05.352523] [INFO ] [MAIN ] IEX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:55:05.380286] [DEBUG] [LOCAL ] IEX: Loaded 28450 existing records from primary storage. [2025-09-16T18:55:05.380420] [INFO ] [MAIN ] IEX: Loaded 28450 existing records. [2025-09-16T18:55:05.562887] [WARN ] [MAIN ] IEX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:55:05.563108] [INFO ] [MAIN ] IEX: Wiping existing primary data before full refresh. [2025-09-16T18:55:05.563201] [INFO ] [LOCAL ] IEX: Starting deletion of primary data. [2025-09-16T18:55:05.563985] [DEBUG] [LOCAL ] IEX: Deleted primary data file. [2025-09-16T18:55:05.564038] [INFO ] [MAIN ] IEX: Starting full primary refresh. [2025-09-16T18:55:05.920975] [INFO ] [REMOTE ] IEX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:55:05.974859] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-02. [2025-09-16T18:55:06.774943] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-02. [2025-09-16T18:55:06.934134] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-09-16T18:55:06.940358] [INFO ] [REMOTE ] ICE: Full fetch complete (57650 records). [2025-09-16T18:55:06.940464] [DEBUG] [VALIDATOR ] ICE: Validating primary data. [2025-09-16T18:55:07.333966] [DEBUG] [VALIDATOR ] ICE: All expected trading days are present. [2025-09-16T18:55:07.334122] [DEBUG] [VALIDATOR ] ICE: Primary data validation passed. [2025-09-16T18:55:07.335005] [INFO ] [MAIN ] ICE: Fetched primary data is valid. [2025-09-16T18:55:07.335069] [DEBUG] [LOCAL ] ICE: Saving primary data. [2025-09-16T18:55:07.336434] [DEBUG] [LOCAL ] ICE: Trimmed 57650 records to 39416 (keeping 150 days of data) [2025-09-16T18:55:07.463128] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T18:55:07.510399] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-09-16T18:55:07.576674] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-03. [2025-09-16T18:55:07.649265] [DEBUG] [LOCAL ] ICE: Primary data saved (39416 records). [2025-09-16T18:55:07.649414] [INFO ] [MAIN ] ICE: Full history saved. [2025-09-16T18:55:07.650491] [INFO ] [MAIN ] Update Progress: 1087/2369 symbols completed. ICE took 27.4s. Estimated remaining: 2h 15m 43s. Rate: 9.4 symbols/min [2025-09-16T18:55:07.820725] [INFO ] [MAIN ] IFF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:55:07.860108] [DEBUG] [LOCAL ] IFF: Loaded 37546 existing records from primary storage. [2025-09-16T18:55:07.860249] [INFO ] [MAIN ] IFF: Loaded 37546 existing records. [2025-09-16T18:55:08.041702] [WARN ] [MAIN ] IFF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:55:08.041959] [INFO ] [MAIN ] IFF: Wiping existing primary data before full refresh. [2025-09-16T18:55:08.042006] [INFO ] [LOCAL ] IFF: Starting deletion of primary data. [2025-09-16T18:55:08.042763] [DEBUG] [LOCAL ] IFF: Deleted primary data file. [2025-09-16T18:55:08.042813] [INFO ] [MAIN ] IFF: Starting full primary refresh. [2025-09-16T18:55:08.379050] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-03. [2025-09-16T18:55:08.402531] [INFO ] [REMOTE ] IFF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:55:08.619002] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-09-16T18:55:09.174906] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-02. [2025-09-16T18:55:09.909679] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T18:55:09.974841] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-04. [2025-09-16T18:55:10.780070] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-06. [2025-09-16T18:55:10.793908] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T18:55:11.477959] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-09-16T18:55:11.553063] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T18:55:11.574846] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-05. [2025-09-16T18:55:12.374936] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-04. [2025-09-16T18:55:12.776267] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-09-16T18:55:13.174906] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-06. [2025-09-16T18:55:13.353211] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-09-16T18:55:13.974934] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-07. [2025-09-16T18:55:14.357714] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-09-16T18:55:14.774962] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-07. [2025-09-16T18:55:14.810638] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-09-16T18:55:15.574970] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-08. [2025-09-16T18:55:16.172795] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T18:55:16.309009] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-09-16T18:55:16.374902] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-05. [2025-09-16T18:55:17.174959] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-08. [2025-09-16T18:55:17.373508] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T18:55:17.974945] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-06. [2025-09-16T18:55:18.679128] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T18:55:18.774882] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-09. [2025-09-16T18:55:19.094849] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T18:55:19.574922] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-03. [2025-09-16T18:55:20.106081] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-09-16T18:55:20.113311] [INFO ] [REMOTE ] IEX: Full fetch complete (41299 records). [2025-09-16T18:55:20.113499] [DEBUG] [VALIDATOR ] IEX: Validating primary data. [2025-09-16T18:55:20.380008] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-07. [2025-09-16T18:55:20.493605] [DEBUG] [VALIDATOR ] IEX: All expected trading days are present. [2025-09-16T18:55:20.493699] [DEBUG] [VALIDATOR ] IEX: Primary data validation passed. [2025-09-16T18:55:20.494392] [INFO ] [MAIN ] IEX: Fetched primary data is valid. [2025-09-16T18:55:20.494441] [DEBUG] [LOCAL ] IEX: Saving primary data. [2025-09-16T18:55:20.495360] [DEBUG] [LOCAL ] IEX: Trimmed 41299 records to 28751 (keeping 150 days of data) [2025-09-16T18:55:20.672127] [DEBUG] [LOCAL ] IEX: Primary data saved (28751 records). [2025-09-16T18:55:20.672267] [INFO ] [MAIN ] IEX: Full history saved. [2025-09-16T18:55:20.672351] [INFO ] [MAIN ] Update Progress: 1088/2369 symbols completed. IEX took 15.5s. Estimated remaining: 2h 15m 45s. Rate: 9.4 symbols/min [2025-09-16T18:55:20.842186] [INFO ] [MAIN ] IFN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:55:20.856194] [DEBUG] [LOCAL ] IFN: Loaded 13576 existing records from primary storage. [2025-09-16T18:55:20.856316] [INFO ] [MAIN ] IFN: Loaded 13576 existing records. [2025-09-16T18:55:21.031533] [WARN ] [MAIN ] IFN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:55:21.031822] [INFO ] [MAIN ] IFN: Wiping existing primary data before full refresh. [2025-09-16T18:55:21.031873] [INFO ] [LOCAL ] IFN: Starting deletion of primary data. [2025-09-16T18:55:21.032270] [DEBUG] [LOCAL ] IFN: Deleted primary data file. [2025-09-16T18:55:21.032314] [INFO ] [MAIN ] IFN: Starting full primary refresh. [2025-09-16T18:55:21.183352] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-09. [2025-09-16T18:55:21.395187] [INFO ] [REMOTE ] IFN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:55:21.424089] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-09-16T18:55:21.527151] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T18:55:21.974972] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-02. [2025-09-16T18:55:21.985968] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T18:55:21.987924] [INFO ] [REMOTE ] IDE: Full fetch complete (9017 records). [2025-09-16T18:55:21.988017] [DEBUG] [VALIDATOR ] IDE: Validating primary data. [2025-09-16T18:55:22.369511] [DEBUG] [VALIDATOR ] IDE: All expected trading days are present. [2025-09-16T18:55:22.369628] [DEBUG] [VALIDATOR ] IDE: Primary data validation passed. [2025-09-16T18:55:22.369892] [INFO ] [MAIN ] IDE: Fetched primary data is valid. [2025-09-16T18:55:22.369933] [DEBUG] [LOCAL ] IDE: Saving primary data. [2025-09-16T18:55:22.370375] [DEBUG] [LOCAL ] IDE: Trimmed 9017 records to 6599 (keeping 150 days of data) [2025-09-16T18:55:22.414622] [DEBUG] [LOCAL ] IDE: Primary data saved (6599 records). [2025-09-16T18:55:22.414791] [INFO ] [MAIN ] IDE: Full history saved. [2025-09-16T18:55:22.414907] [INFO ] [MAIN ] Update Progress: 1089/2369 symbols completed. IDE took 34.1s. Estimated remaining: 2h 15m 33s. Rate: 9.4 symbols/min [2025-09-16T18:55:22.586628] [INFO ] [MAIN ] IFS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:55:22.608861] [DEBUG] [LOCAL ] IFS: Loaded 16343 existing records from primary storage. [2025-09-16T18:55:22.609024] [INFO ] [MAIN ] IFS: Loaded 16343 existing records. [2025-09-16T18:55:22.782766] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-08. [2025-09-16T18:55:22.787924] [WARN ] [MAIN ] IFS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:55:22.789155] [INFO ] [MAIN ] IFS: Wiping existing primary data before full refresh. [2025-09-16T18:55:22.789257] [INFO ] [LOCAL ] IFS: Starting deletion of primary data. [2025-09-16T18:55:22.789773] [DEBUG] [LOCAL ] IFS: Deleted primary data file. [2025-09-16T18:55:22.789835] [INFO ] [MAIN ] IFS: Starting full primary refresh. [2025-09-16T18:55:23.145713] [INFO ] [REMOTE ] IFS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:55:23.158673] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-09-16T18:55:23.574947] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-04. [2025-09-16T18:55:23.771519] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T18:55:24.374925] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-02. [2025-09-16T18:55:25.174918] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-09. [2025-09-16T18:55:25.233066] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T18:55:25.470445] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T18:55:25.974914] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-03. [2025-09-16T18:55:26.101450] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T18:55:26.103997] [INFO ] [REMOTE ] IDT: Full fetch complete (18233 records). [2025-09-16T18:55:26.104073] [DEBUG] [VALIDATOR ] IDT: Validating primary data. [2025-09-16T18:55:26.480625] [DEBUG] [VALIDATOR ] IDT: All expected trading days are present. [2025-09-16T18:55:26.480766] [DEBUG] [VALIDATOR ] IDT: Primary data validation passed. [2025-09-16T18:55:26.481201] [INFO ] [MAIN ] IDT: Fetched primary data is valid. [2025-09-16T18:55:26.481245] [DEBUG] [LOCAL ] IDT: Saving primary data. [2025-09-16T18:55:26.481886] [DEBUG] [LOCAL ] IDT: Trimmed 18233 records to 13221 (keeping 150 days of data) [2025-09-16T18:55:26.561870] [DEBUG] [LOCAL ] IDT: Primary data saved (13221 records). [2025-09-16T18:55:26.562017] [INFO ] [MAIN ] IDT: Full history saved. [2025-09-16T18:55:26.562128] [INFO ] [MAIN ] Update Progress: 1090/2369 symbols completed. IDT took 23.0s. Estimated remaining: 2h 15m 24s. Rate: 9.4 symbols/min [2025-09-16T18:55:26.741031] [INFO ] [MAIN ] IGA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:55:26.749768] [DEBUG] [LOCAL ] IGA: Loaded 6917 existing records from primary storage. [2025-09-16T18:55:26.749898] [INFO ] [MAIN ] IGA: Loaded 6917 existing records. [2025-09-16T18:55:26.780100] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-03. [2025-09-16T18:55:26.933819] [WARN ] [MAIN ] IGA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:55:26.934104] [INFO ] [MAIN ] IGA: Wiping existing primary data before full refresh. [2025-09-16T18:55:26.934153] [INFO ] [LOCAL ] IGA: Starting deletion of primary data. [2025-09-16T18:55:26.934434] [DEBUG] [LOCAL ] IGA: Deleted primary data file. [2025-09-16T18:55:26.934480] [INFO ] [MAIN ] IGA: Starting full primary refresh. [2025-09-16T18:55:27.295882] [INFO ] [REMOTE ] IGA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:55:27.310384] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-09-16T18:55:27.574885] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-02. [2025-09-16T18:55:27.838548] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-09-16T18:55:28.286115] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-09-16T18:55:28.374887] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-03. [2025-09-16T18:55:29.134333] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-09-16T18:55:29.174873] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-04. [2025-09-16T18:55:29.974907] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-05. [2025-09-16T18:55:30.129131] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-09-16T18:55:30.774902] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-05. [2025-09-16T18:55:31.330054] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-09-16T18:55:31.575427] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-04. [2025-09-16T18:55:31.581379] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-09-16T18:55:32.374871] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-06. [2025-09-16T18:55:32.435869] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-09-16T18:55:33.174906] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-06. [2025-09-16T18:55:33.882021] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-09-16T18:55:33.981579] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-04. [2025-09-16T18:55:34.000755] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T18:55:34.774898] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-07. [2025-09-16T18:55:35.110434] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-09-16T18:55:35.574851] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-07. [2025-09-16T18:55:35.574768] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-09-16T18:55:36.374897] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-08. [2025-09-16T18:55:37.095753] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-09-16T18:55:37.174880] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-05. [2025-09-16T18:55:37.254930] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-09-16T18:55:37.974896] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-05. [2025-09-16T18:55:38.171676] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T18:55:38.774898] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-08. [2025-09-16T18:55:39.035830] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-09-16T18:55:39.574904] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-09. [2025-09-16T18:55:40.360085] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T18:55:40.375525] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-06. [2025-09-16T18:55:40.376622] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T18:55:40.379757] [INFO ] [REMOTE ] IGA: Full fetch complete (10390 records). [2025-09-16T18:55:40.379874] [DEBUG] [VALIDATOR ] IGA: Validating primary data. [2025-09-16T18:55:40.757821] [DEBUG] [VALIDATOR ] IGA: All expected trading days are present. [2025-09-16T18:55:40.758030] [DEBUG] [VALIDATOR ] IGA: Primary data validation passed. [2025-09-16T18:55:40.758334] [INFO ] [MAIN ] IGA: Fetched primary data is valid. [2025-09-16T18:55:40.758396] [DEBUG] [LOCAL ] IGA: Saving primary data. [2025-09-16T18:55:40.758959] [DEBUG] [LOCAL ] IGA: Trimmed 10390 records to 7019 (keeping 150 days of data) [2025-09-16T18:55:40.802103] [DEBUG] [LOCAL ] IGA: Primary data saved (7019 records). [2025-09-16T18:55:40.802222] [INFO ] [MAIN ] IGA: Full history saved. [2025-09-16T18:55:40.802306] [INFO ] [MAIN ] Update Progress: 1091/2369 symbols completed. IGA took 14.2s. Estimated remaining: 2h 15m 27s. Rate: 9.4 symbols/min [2025-09-16T18:55:40.977100] [INFO ] [MAIN ] IGD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:55:40.991611] [DEBUG] [LOCAL ] IGD: Loaded 13795 existing records from primary storage. [2025-09-16T18:55:40.991772] [INFO ] [MAIN ] IGD: Loaded 13795 existing records. [2025-09-16T18:55:41.165406] [WARN ] [MAIN ] IGD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:55:41.165729] [INFO ] [MAIN ] IGD: Wiping existing primary data before full refresh. [2025-09-16T18:55:41.165777] [INFO ] [LOCAL ] IGD: Starting deletion of primary data. [2025-09-16T18:55:41.166311] [DEBUG] [LOCAL ] IGD: Deleted primary data file. [2025-09-16T18:55:41.166357] [INFO ] [MAIN ] IGD: Starting full primary refresh. [2025-09-16T18:55:41.180003] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-06. [2025-09-16T18:55:41.515593] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:55:41.552919] [INFO ] [REMOTE ] IGD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:55:41.974916] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-09. [2025-09-16T18:55:42.526969] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T18:55:42.774900] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-02. [2025-09-16T18:55:43.574878] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-07. [2025-09-16T18:55:43.641610] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-09-16T18:55:43.650945] [INFO ] [REMOTE ] IFF: Full fetch complete (55399 records). [2025-09-16T18:55:43.651156] [DEBUG] [VALIDATOR ] IFF: Validating primary data. [2025-09-16T18:55:43.952506] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-09-16T18:55:44.079780] [DEBUG] [VALIDATOR ] IFF: All expected trading days are present. [2025-09-16T18:55:44.079859] [DEBUG] [VALIDATOR ] IFF: Primary data validation passed. [2025-09-16T18:55:44.080594] [INFO ] [MAIN ] IFF: Fetched primary data is valid. [2025-09-16T18:55:44.080637] [DEBUG] [LOCAL ] IFF: Saving primary data. [2025-09-16T18:55:44.081763] [DEBUG] [LOCAL ] IFF: Trimmed 55399 records to 37927 (keeping 150 days of data) [2025-09-16T18:55:44.318546] [DEBUG] [LOCAL ] IFF: Primary data saved (37927 records). [2025-09-16T18:55:44.318746] [INFO ] [MAIN ] IFF: Full history saved. [2025-09-16T18:55:44.319666] [INFO ] [MAIN ] Update Progress: 1092/2369 symbols completed. IFF took 36.7s. Estimated remaining: 2h 15m 17s. Rate: 9.4 symbols/min [2025-09-16T18:55:44.382173] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-03. [2025-09-16T18:55:44.492581] [INFO ] [MAIN ] IGI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:55:44.497883] [DEBUG] [LOCAL ] IGI: Loaded 2713 existing records from primary storage. [2025-09-16T18:55:44.497990] [INFO ] [MAIN ] IGI: Loaded 2713 existing records. [2025-09-16T18:55:44.673658] [WARN ] [MAIN ] IGI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:55:44.673996] [INFO ] [MAIN ] IGI: Wiping existing primary data before full refresh. [2025-09-16T18:55:44.674198] [INFO ] [LOCAL ] IGI: Starting deletion of primary data. [2025-09-16T18:55:44.674445] [DEBUG] [LOCAL ] IGI: Deleted primary data file. [2025-09-16T18:55:44.674538] [INFO ] [MAIN ] IGI: Starting full primary refresh. [2025-09-16T18:55:44.794242] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-09-16T18:55:45.053297] [INFO ] [REMOTE ] IGI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:55:45.174932] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-02. [2025-09-16T18:55:45.544730] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T18:55:45.624306] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-09-16T18:55:45.974964] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-03. [2025-09-16T18:55:46.564761] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-09-16T18:55:46.774912] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-08. [2025-09-16T18:55:47.574917] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-04. [2025-09-16T18:55:47.899247] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T18:55:48.374947] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-04. [2025-09-16T18:55:48.607078] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T18:55:48.916578] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-09-16T18:55:49.174941] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-09. [2025-09-16T18:55:49.974911] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-05. [2025-09-16T18:55:50.157261] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-09-16T18:55:50.159695] [INFO ] [REMOTE ] IFN: Full fetch complete (20556 records). [2025-09-16T18:55:50.159770] [DEBUG] [VALIDATOR ] IFN: Validating primary data. [2025-09-16T18:55:50.446067] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-09-16T18:55:50.530087] [DEBUG] [VALIDATOR ] IFN: All expected trading days are present. [2025-09-16T18:55:50.530172] [DEBUG] [VALIDATOR ] IFN: Primary data validation passed. [2025-09-16T18:55:50.530636] [INFO ] [MAIN ] IFN: Fetched primary data is valid. [2025-09-16T18:55:50.530708] [DEBUG] [LOCAL ] IFN: Saving primary data. [2025-09-16T18:55:50.531308] [DEBUG] [LOCAL ] IFN: Trimmed 20556 records to 13709 (keeping 150 days of data) [2025-09-16T18:55:50.614141] [DEBUG] [LOCAL ] IFN: Primary data saved (13709 records). [2025-09-16T18:55:50.614282] [INFO ] [MAIN ] IFN: Full history saved. [2025-09-16T18:55:50.614364] [INFO ] [MAIN ] Update Progress: 1093/2369 symbols completed. IFN took 29.9s. Estimated remaining: 2h 15m 11s. Rate: 9.4 symbols/min [2025-09-16T18:55:50.782446] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-05. [2025-09-16T18:55:50.785066] [INFO ] [MAIN ] IGR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:55:50.806670] [DEBUG] [LOCAL ] IGR: Loaded 22380 existing records from primary storage. [2025-09-16T18:55:50.806799] [INFO ] [MAIN ] IGR: Loaded 22380 existing records. [2025-09-16T18:55:50.984341] [WARN ] [MAIN ] IGR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:55:50.984596] [INFO ] [MAIN ] IGR: Wiping existing primary data before full refresh. [2025-09-16T18:55:50.984660] [INFO ] [LOCAL ] IGR: Starting deletion of primary data. [2025-09-16T18:55:50.985238] [DEBUG] [LOCAL ] IGR: Deleted primary data file. [2025-09-16T18:55:50.985287] [INFO ] [MAIN ] IGR: Starting full primary refresh. [2025-09-16T18:55:51.350264] [INFO ] [REMOTE ] IGR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:55:51.574914] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-02. [2025-09-16T18:55:51.750792] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T18:55:52.374870] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-06. [2025-09-16T18:55:52.843440] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T18:55:53.174906] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-06. [2025-09-16T18:55:53.391704] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-09-16T18:55:53.501969] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-09-16T18:55:53.974943] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-07. [2025-09-16T18:55:54.774937] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-07. [2025-09-16T18:55:54.925543] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-09-16T18:55:55.574967] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-03. [2025-09-16T18:55:55.632165] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-09-16T18:55:56.374946] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-08. [2025-09-16T18:55:56.818843] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-09-16T18:55:57.174903] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-04. [2025-09-16T18:55:57.509227] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T18:55:57.974949] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-08. [2025-09-16T18:55:58.450395] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-09-16T18:55:58.774901] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-07. [2025-09-16T18:55:58.948541] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-09-16T18:55:59.368179] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-09-16T18:55:59.574940] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-09. [2025-09-16T18:56:00.374883] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-09. [2025-09-16T18:56:00.577060] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T18:56:00.580079] [INFO ] [REMOTE ] IFS: Full fetch complete (24972 records). [2025-09-16T18:56:00.580166] [DEBUG] [VALIDATOR ] IFS: Validating primary data. [2025-09-16T18:56:00.952869] [DEBUG] [VALIDATOR ] IFS: All expected trading days are present. [2025-09-16T18:56:00.952972] [DEBUG] [VALIDATOR ] IFS: Primary data validation passed. [2025-09-16T18:56:00.953565] [INFO ] [MAIN ] IFS: Fetched primary data is valid. [2025-09-16T18:56:00.953619] [DEBUG] [LOCAL ] IFS: Saving primary data. [2025-09-16T18:56:00.954711] [DEBUG] [LOCAL ] IFS: Trimmed 24972 records to 16478 (keeping 150 days of data) [2025-09-16T18:56:01.055851] [DEBUG] [LOCAL ] IFS: Primary data saved (16478 records). [2025-09-16T18:56:01.056016] [INFO ] [MAIN ] IFS: Full history saved. [2025-09-16T18:56:01.056125] [INFO ] [MAIN ] Update Progress: 1094/2369 symbols completed. IFS took 38.6s. Estimated remaining: 2h 15m 9s. Rate: 9.4 symbols/min [2025-09-16T18:56:01.180034] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-05. [2025-09-16T18:56:01.227763] [INFO ] [MAIN ] IH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:01.231420] [DEBUG] [LOCAL ] IH: Loaded 2002 existing records from primary storage. [2025-09-16T18:56:01.231505] [INFO ] [MAIN ] IH: Loaded 2002 existing records. [2025-09-16T18:56:01.412824] [WARN ] [MAIN ] IH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:01.413118] [INFO ] [MAIN ] IH: Wiping existing primary data before full refresh. [2025-09-16T18:56:01.413290] [INFO ] [LOCAL ] IH: Starting deletion of primary data. [2025-09-16T18:56:01.419216] [DEBUG] [LOCAL ] IH: Deleted primary data file. [2025-09-16T18:56:01.419438] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-09-16T18:56:01.419582] [INFO ] [MAIN ] IH: Starting full primary refresh. [2025-09-16T18:56:01.447671] [INFO ] [REMOTE ] IGD: Full fetch complete (21212 records). [2025-09-16T18:56:01.489817] [DEBUG] [VALIDATOR ] IGD: Validating primary data. [2025-09-16T18:56:01.980092] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-08. [2025-09-16T18:56:02.047051] [INFO ] [REMOTE ] IH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:02.194121] [DEBUG] [VALIDATOR ] IGD: All expected trading days are present. [2025-09-16T18:56:02.194229] [DEBUG] [VALIDATOR ] IGD: Primary data validation passed. [2025-09-16T18:56:02.194768] [INFO ] [MAIN ] IGD: Fetched primary data is valid. [2025-09-16T18:56:02.194815] [DEBUG] [LOCAL ] IGD: Saving primary data. [2025-09-16T18:56:02.195461] [DEBUG] [LOCAL ] IGD: Trimmed 21212 records to 13912 (keeping 150 days of data) [2025-09-16T18:56:02.281009] [DEBUG] [LOCAL ] IGD: Primary data saved (13912 records). [2025-09-16T18:56:02.281166] [INFO ] [MAIN ] IGD: Full history saved. [2025-09-16T18:56:02.281269] [INFO ] [MAIN ] Update Progress: 1095/2369 symbols completed. IGD took 21.5s. Estimated remaining: 2h 14m 57s. Rate: 9.4 symbols/min [2025-09-16T18:56:02.462661] [INFO ] [MAIN ] IHD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:02.469594] [DEBUG] [LOCAL ] IHD: Loaded 5148 existing records from primary storage. [2025-09-16T18:56:02.469734] [INFO ] [MAIN ] IHD: Loaded 5148 existing records. [2025-09-16T18:56:02.662518] [WARN ] [MAIN ] IHD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:02.662930] [INFO ] [MAIN ] IHD: Wiping existing primary data before full refresh. [2025-09-16T18:56:02.663007] [INFO ] [LOCAL ] IHD: Starting deletion of primary data. [2025-09-16T18:56:02.663278] [DEBUG] [LOCAL ] IHD: Deleted primary data file. [2025-09-16T18:56:02.663323] [INFO ] [MAIN ] IHD: Starting full primary refresh. [2025-09-16T18:56:02.780137] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-02. [2025-09-16T18:56:02.782128] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-09-16T18:56:02.882939] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T18:56:03.079108] [INFO ] [REMOTE ] IHD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:03.411955] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-09-16T18:56:03.574966] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-02. [2025-09-16T18:56:04.362674] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T18:56:04.374845] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-03. [2025-09-16T18:56:05.174916] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-09. [2025-09-16T18:56:05.197921] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-09-16T18:56:05.845601] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-09-16T18:56:05.846532] [INFO ] [REMOTE ] IGI: Full fetch complete (3976 records). [2025-09-16T18:56:05.846625] [DEBUG] [VALIDATOR ] IGI: Validating primary data. [2025-09-16T18:56:05.979962] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-03. [2025-09-16T18:56:06.217157] [DEBUG] [VALIDATOR ] IGI: All expected trading days are present. [2025-09-16T18:56:06.217264] [DEBUG] [VALIDATOR ] IGI: Primary data validation passed. [2025-09-16T18:56:06.217400] [INFO ] [MAIN ] IGI: Fetched primary data is valid. [2025-09-16T18:56:06.217440] [DEBUG] [LOCAL ] IGI: Saving primary data. [2025-09-16T18:56:06.217842] [DEBUG] [LOCAL ] IGI: Trimmed 3976 records to 2729 (keeping 150 days of data) [2025-09-16T18:56:06.235561] [DEBUG] [LOCAL ] IGI: Primary data saved (2729 records). [2025-09-16T18:56:06.235630] [INFO ] [MAIN ] IGI: Full history saved. [2025-09-16T18:56:06.235732] [INFO ] [MAIN ] Update Progress: 1096/2369 symbols completed. IGI took 21.9s. Estimated remaining: 2h 14m 48s. Rate: 9.4 symbols/min [2025-09-16T18:56:06.412784] [INFO ] [MAIN ] IHG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:06.432834] [DEBUG] [LOCAL ] IHG: Loaded 17602 existing records from primary storage. [2025-09-16T18:56:06.432979] [INFO ] [MAIN ] IHG: Loaded 17602 existing records. [2025-09-16T18:56:06.614700] [WARN ] [MAIN ] IHG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:06.614971] [INFO ] [MAIN ] IHG: Wiping existing primary data before full refresh. [2025-09-16T18:56:06.615029] [INFO ] [LOCAL ] IHG: Starting deletion of primary data. [2025-09-16T18:56:06.615615] [DEBUG] [LOCAL ] IHG: Deleted primary data file. [2025-09-16T18:56:06.615696] [INFO ] [MAIN ] IHG: Starting full primary refresh. [2025-09-16T18:56:06.792859] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-06. [2025-09-16T18:56:06.914458] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-09-16T18:56:06.997928] [INFO ] [REMOTE ] IHG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:07.574881] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-04. [2025-09-16T18:56:07.936846] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T18:56:08.024891] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-09-16T18:56:08.374911] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-07. [2025-09-16T18:56:09.174934] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-05. [2025-09-16T18:56:09.602969] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-09-16T18:56:09.937159] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-09-16T18:56:09.974863] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-04. [2025-09-16T18:56:10.682246] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-09-16T18:56:10.774915] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-08. [2025-09-16T18:56:11.574902] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-02. [2025-09-16T18:56:12.198046] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T18:56:12.374870] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-05. [2025-09-16T18:56:12.787509] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-09-16T18:56:13.136327] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-09-16T18:56:13.174871] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-06. [2025-09-16T18:56:13.783556] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-09-16T18:56:13.974907] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-07. [2025-09-16T18:56:14.668939] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-09-16T18:56:14.774902] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-08. [2025-09-16T18:56:15.358506] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-09-16T18:56:15.574895] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-09. [2025-09-16T18:56:16.217596] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-09-16T18:56:16.219178] [INFO ] [REMOTE ] IHD: Full fetch complete (8138 records). [2025-09-16T18:56:16.219256] [DEBUG] [VALIDATOR ] IHD: Validating primary data. [2025-09-16T18:56:16.380250] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-09. [2025-09-16T18:56:16.580699] [DEBUG] [VALIDATOR ] IHD: All expected trading days are present. [2025-09-16T18:56:16.580776] [DEBUG] [VALIDATOR ] IHD: Primary data validation passed. [2025-09-16T18:56:16.580981] [INFO ] [MAIN ] IHD: Fetched primary data is valid. [2025-09-16T18:56:16.581021] [DEBUG] [LOCAL ] IHD: Saving primary data. [2025-09-16T18:56:16.581416] [DEBUG] [LOCAL ] IHD: Trimmed 8138 records to 5213 (keeping 150 days of data) [2025-09-16T18:56:16.611491] [DEBUG] [LOCAL ] IHD: Primary data saved (5213 records). [2025-09-16T18:56:16.611561] [INFO ] [MAIN ] IHD: Full history saved. [2025-09-16T18:56:16.611666] [INFO ] [MAIN ] Update Progress: 1097/2369 symbols completed. IHD took 14.3s. Estimated remaining: 2h 14m 46s. Rate: 9.4 symbols/min [2025-09-16T18:56:16.783060] [INFO ] [MAIN ] IHS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:16.810069] [DEBUG] [LOCAL ] IHS: Loaded 29102 existing records from primary storage. [2025-09-16T18:56:16.810194] [INFO ] [MAIN ] IHS: Loaded 29102 existing records. [2025-09-16T18:56:16.984444] [WARN ] [MAIN ] IHS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:16.984725] [INFO ] [MAIN ] IHS: Wiping existing primary data before full refresh. [2025-09-16T18:56:16.984774] [INFO ] [LOCAL ] IHS: Starting deletion of primary data. [2025-09-16T18:56:16.985446] [DEBUG] [LOCAL ] IHS: Deleted primary data file. [2025-09-16T18:56:16.985490] [INFO ] [MAIN ] IHS: Starting full primary refresh. [2025-09-16T18:56:17.180379] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-03. [2025-09-16T18:56:17.337801] [INFO ] [REMOTE ] IHS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:17.660344] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-09-16T18:56:17.664749] [INFO ] [REMOTE ] IGR: Full fetch complete (33284 records). [2025-09-16T18:56:17.664847] [DEBUG] [VALIDATOR ] IGR: Validating primary data. [2025-09-16T18:56:17.980115] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-02. [2025-09-16T18:56:18.066049] [DEBUG] [VALIDATOR ] IGR: All expected trading days are present. [2025-09-16T18:56:18.066149] [DEBUG] [VALIDATOR ] IGR: Primary data validation passed. [2025-09-16T18:56:18.066860] [INFO ] [MAIN ] IGR: Fetched primary data is valid. [2025-09-16T18:56:18.066927] [DEBUG] [LOCAL ] IGR: Saving primary data. [2025-09-16T18:56:18.068018] [DEBUG] [LOCAL ] IGR: Trimmed 33284 records to 22600 (keeping 150 days of data) [2025-09-16T18:56:18.205354] [DEBUG] [LOCAL ] IGR: Primary data saved (22600 records). [2025-09-16T18:56:18.205492] [INFO ] [MAIN ] IGR: Full history saved. [2025-09-16T18:56:18.205603] [INFO ] [MAIN ] Update Progress: 1098/2369 symbols completed. IGR took 27.6s. Estimated remaining: 2h 14m 34s. Rate: 9.4 symbols/min [2025-09-16T18:56:18.385858] [INFO ] [MAIN ] IIF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:18.391866] [DEBUG] [LOCAL ] IIF: Loaded 4027 existing records from primary storage. [2025-09-16T18:56:18.391981] [INFO ] [MAIN ] IIF: Loaded 4027 existing records. [2025-09-16T18:56:18.511944] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T18:56:18.597694] [WARN ] [MAIN ] IIF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:18.597965] [INFO ] [MAIN ] IIF: Wiping existing primary data before full refresh. [2025-09-16T18:56:18.598011] [INFO ] [LOCAL ] IIF: Starting deletion of primary data. [2025-09-16T18:56:18.598298] [DEBUG] [LOCAL ] IIF: Deleted primary data file. [2025-09-16T18:56:18.598352] [INFO ] [MAIN ] IIF: Starting full primary refresh. [2025-09-16T18:56:18.784548] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-04. [2025-09-16T18:56:18.976714] [INFO ] [REMOTE ] IIF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:18.992912] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-09-16T18:56:19.574952] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-03. [2025-09-16T18:56:20.325407] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-09-16T18:56:20.374873] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-05. [2025-09-16T18:56:20.991218] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-09-16T18:56:21.174929] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-06. [2025-09-16T18:56:21.591217] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-09-16T18:56:21.700133] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T18:56:21.974901] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-06. [2025-09-16T18:56:22.774881] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-04. [2025-09-16T18:56:23.127200] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-09-16T18:56:23.574912] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-02. [2025-09-16T18:56:24.126993] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T18:56:24.324468] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T18:56:24.374912] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-03. [2025-09-16T18:56:24.961453] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-09-16T18:56:25.174913] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-07. [2025-09-16T18:56:25.974923] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-05. [2025-09-16T18:56:26.541701] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T18:56:26.774935] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-07. [2025-09-16T18:56:27.219285] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-09-16T18:56:27.475154] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-09-16T18:56:27.574863] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-08. [2025-09-16T18:56:27.959661] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-09-16T18:56:28.374922] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-06. [2025-09-16T18:56:29.174916] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-09. [2025-09-16T18:56:29.720928] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-09-16T18:56:29.721984] [INFO ] [REMOTE ] IH: Full fetch complete (4235 records). [2025-09-16T18:56:29.722056] [DEBUG] [VALIDATOR ] IH: Validating primary data. [2025-09-16T18:56:29.977334] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-08. [2025-09-16T18:56:30.113976] [DEBUG] [VALIDATOR ] IH: All expected trading days are present. [2025-09-16T18:56:30.114379] [DEBUG] [VALIDATOR ] IH: Primary data validation passed. [2025-09-16T18:56:30.114552] [INFO ] [MAIN ] IH: Fetched primary data is valid. [2025-09-16T18:56:30.116297] [DEBUG] [LOCAL ] IH: Saving primary data. [2025-09-16T18:56:30.116202] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T18:56:30.116931] [DEBUG] [LOCAL ] IH: Trimmed 4235 records to 2033 (keeping 150 days of data) [2025-09-16T18:56:30.137671] [DEBUG] [LOCAL ] IH: Primary data saved (2033 records). [2025-09-16T18:56:30.137738] [INFO ] [MAIN ] IH: Full history saved. [2025-09-16T18:56:30.137823] [INFO ] [MAIN ] Update Progress: 1099/2369 symbols completed. IH took 29.1s. Estimated remaining: 2h 14m 34s. Rate: 9.4 symbols/min [2025-09-16T18:56:30.315307] [INFO ] [MAIN ] IIIN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:30.331223] [DEBUG] [LOCAL ] IIIN: Loaded 11671 existing records from primary storage. [2025-09-16T18:56:30.331345] [INFO ] [MAIN ] IIIN: Loaded 11671 existing records. [2025-09-16T18:56:30.506215] [WARN ] [MAIN ] IIIN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:30.506491] [INFO ] [MAIN ] IIIN: Wiping existing primary data before full refresh. [2025-09-16T18:56:30.506548] [INFO ] [LOCAL ] IIIN: Starting deletion of primary data. [2025-09-16T18:56:30.506992] [DEBUG] [LOCAL ] IIIN: Deleted primary data file. [2025-09-16T18:56:30.507055] [INFO ] [MAIN ] IIIN: Starting full primary refresh. [2025-09-16T18:56:30.780856] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-04. [2025-09-16T18:56:30.857753] [INFO ] [REMOTE ] IIIN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:31.174589] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-09-16T18:56:31.492561] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-09-16T18:56:31.574890] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-05. [2025-09-16T18:56:32.195435] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-09-16T18:56:32.374903] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-09. [2025-09-16T18:56:33.174908] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-06. [2025-09-16T18:56:33.401092] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-09-16T18:56:33.403984] [INFO ] [REMOTE ] IHG: Full fetch complete (27406 records). [2025-09-16T18:56:33.404081] [DEBUG] [VALIDATOR ] IHG: Validating primary data. [2025-09-16T18:56:33.771052] [DEBUG] [VALIDATOR ] IHG: All expected trading days are present. [2025-09-16T18:56:33.771133] [DEBUG] [VALIDATOR ] IHG: Primary data validation passed. [2025-09-16T18:56:33.771754] [INFO ] [MAIN ] IHG: Fetched primary data is valid. [2025-09-16T18:56:33.771799] [DEBUG] [LOCAL ] IHG: Saving primary data. [2025-09-16T18:56:33.772458] [DEBUG] [LOCAL ] IHG: Trimmed 27406 records to 17709 (keeping 150 days of data) [2025-09-16T18:56:33.864271] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-09-16T18:56:33.897387] [DEBUG] [LOCAL ] IHG: Primary data saved (17709 records). [2025-09-16T18:56:33.897527] [INFO ] [MAIN ] IHG: Full history saved. [2025-09-16T18:56:33.897631] [INFO ] [MAIN ] Update Progress: 1100/2369 symbols completed. IHG took 27.7s. Estimated remaining: 2h 14m 25s. Rate: 9.4 symbols/min [2025-09-16T18:56:33.980117] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-07. [2025-09-16T18:56:34.080401] [INFO ] [MAIN ] IIM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:34.092799] [DEBUG] [LOCAL ] IIM: Loaded 11317 existing records from primary storage. [2025-09-16T18:56:34.092901] [INFO ] [MAIN ] IIM: Loaded 11317 existing records. [2025-09-16T18:56:34.276752] [WARN ] [MAIN ] IIM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:34.277011] [INFO ] [MAIN ] IIM: Wiping existing primary data before full refresh. [2025-09-16T18:56:34.277058] [INFO ] [LOCAL ] IIM: Starting deletion of primary data. [2025-09-16T18:56:34.277408] [DEBUG] [LOCAL ] IIM: Deleted primary data file. [2025-09-16T18:56:34.277452] [INFO ] [MAIN ] IIM: Starting full primary refresh. [2025-09-16T18:56:34.670133] [INFO ] [REMOTE ] IIM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:34.774899] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-07. [2025-09-16T18:56:35.521519] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-09-16T18:56:35.547576] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T18:56:35.574859] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-08. [2025-09-16T18:56:36.374931] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-02. [2025-09-16T18:56:36.933734] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-09-16T18:56:37.174869] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-09. [2025-09-16T18:56:37.467553] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T18:56:37.974898] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-02. [2025-09-16T18:56:38.474012] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T18:56:38.479695] [INFO ] [REMOTE ] IHS: Full fetch complete (41405 records). [2025-09-16T18:56:38.479828] [DEBUG] [VALIDATOR ] IHS: Validating primary data. [2025-09-16T18:56:38.783113] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-03. [2025-09-16T18:56:38.912051] [DEBUG] [VALIDATOR ] IHS: All expected trading days are present. [2025-09-16T18:56:38.912144] [DEBUG] [VALIDATOR ] IHS: Primary data validation passed. [2025-09-16T18:56:38.912911] [INFO ] [MAIN ] IHS: Fetched primary data is valid. [2025-09-16T18:56:38.912961] [DEBUG] [LOCAL ] IHS: Saving primary data. [2025-09-16T18:56:38.914308] [DEBUG] [LOCAL ] IHS: Trimmed 41405 records to 29352 (keeping 150 days of data) [2025-09-16T18:56:39.036134] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T18:56:39.101522] [DEBUG] [LOCAL ] IHS: Primary data saved (29352 records). [2025-09-16T18:56:39.101690] [INFO ] [MAIN ] IHS: Full history saved. [2025-09-16T18:56:39.101784] [INFO ] [MAIN ] Update Progress: 1101/2369 symbols completed. IHS took 22.5s. Estimated remaining: 2h 14m 17s. Rate: 9.4 symbols/min [2025-09-16T18:56:39.272839] [INFO ] [MAIN ] IIPR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:39.292540] [DEBUG] [LOCAL ] IIPR: Loaded 20582 existing records from primary storage. [2025-09-16T18:56:39.292691] [INFO ] [MAIN ] IIPR: Loaded 20582 existing records. [2025-09-16T18:56:39.473890] [WARN ] [MAIN ] IIPR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:39.474163] [INFO ] [MAIN ] IIPR: Wiping existing primary data before full refresh. [2025-09-16T18:56:39.474212] [INFO ] [LOCAL ] IIPR: Starting deletion of primary data. [2025-09-16T18:56:39.474838] [DEBUG] [LOCAL ] IIPR: Deleted primary data file. [2025-09-16T18:56:39.474889] [INFO ] [MAIN ] IIPR: Starting full primary refresh. [2025-09-16T18:56:39.575801] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-03. [2025-09-16T18:56:39.827938] [INFO ] [REMOTE ] IIPR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:39.853853] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-09-16T18:56:40.374898] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-02. [2025-09-16T18:56:40.622575] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-09-16T18:56:41.174907] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-08. [2025-09-16T18:56:41.635788] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-09-16T18:56:41.709807] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-09-16T18:56:41.974910] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-04. [2025-09-16T18:56:42.774900] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-03. [2025-09-16T18:56:42.891963] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-09-16T18:56:43.574887] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-05. [2025-09-16T18:56:44.286848] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-09-16T18:56:44.308261] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T18:56:44.374848] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-06. [2025-09-16T18:56:45.174907] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-04. [2025-09-16T18:56:45.291480] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-09-16T18:56:45.974914] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-04. [2025-09-16T18:56:46.562843] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T18:56:46.587736] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-09-16T18:56:46.774931] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-05. [2025-09-16T18:56:47.514356] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-09-16T18:56:47.575038] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-05. [2025-09-16T18:56:48.374911] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-06. [2025-09-16T18:56:48.848311] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T18:56:49.174909] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-09. [2025-09-16T18:56:49.339698] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-09-16T18:56:49.799557] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-09-16T18:56:49.800576] [INFO ] [REMOTE ] IIF: Full fetch complete (6482 records). [2025-09-16T18:56:49.800670] [DEBUG] [VALIDATOR ] IIF: Validating primary data. [2025-09-16T18:56:49.980321] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-07. [2025-09-16T18:56:50.159985] [DEBUG] [VALIDATOR ] IIF: All expected trading days are present. [2025-09-16T18:56:50.160087] [DEBUG] [VALIDATOR ] IIF: Primary data validation passed. [2025-09-16T18:56:50.160279] [INFO ] [MAIN ] IIF: Fetched primary data is valid. [2025-09-16T18:56:50.160321] [DEBUG] [LOCAL ] IIF: Saving primary data. [2025-09-16T18:56:50.160915] [DEBUG] [LOCAL ] IIF: Trimmed 6482 records to 4047 (keeping 150 days of data) [2025-09-16T18:56:50.185152] [DEBUG] [LOCAL ] IIF: Primary data saved (4047 records). [2025-09-16T18:56:50.185270] [INFO ] [MAIN ] IIF: Full history saved. [2025-09-16T18:56:50.185381] [INFO ] [MAIN ] Update Progress: 1102/2369 symbols completed. IIF took 32.0s. Estimated remaining: 2h 14m 16s. Rate: 9.4 symbols/min [2025-09-16T18:56:50.354245] [INFO ] [MAIN ] IMAX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:50.391788] [DEBUG] [LOCAL ] IMAX: Loaded 31543 existing records from primary storage. [2025-09-16T18:56:50.391961] [INFO ] [MAIN ] IMAX: Loaded 31543 existing records. [2025-09-16T18:56:50.564402] [WARN ] [MAIN ] IMAX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:50.564628] [INFO ] [MAIN ] IMAX: Wiping existing primary data before full refresh. [2025-09-16T18:56:50.564713] [INFO ] [LOCAL ] IMAX: Starting deletion of primary data. [2025-09-16T18:56:50.565455] [DEBUG] [LOCAL ] IMAX: Deleted primary data file. [2025-09-16T18:56:50.565501] [INFO ] [MAIN ] IMAX: Starting full primary refresh. [2025-09-16T18:56:50.781261] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-06. [2025-09-16T18:56:50.917154] [INFO ] [REMOTE ] IMAX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:51.045961] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-09-16T18:56:51.574897] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-08. [2025-09-16T18:56:52.112201] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T18:56:52.175882] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T18:56:52.374900] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-09. [2025-09-16T18:56:53.100120] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T18:56:53.102799] [INFO ] [REMOTE ] IIM: Full fetch complete (18436 records). [2025-09-16T18:56:53.102871] [DEBUG] [VALIDATOR ] IIM: Validating primary data. [2025-09-16T18:56:53.180015] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-07. [2025-09-16T18:56:53.466830] [DEBUG] [VALIDATOR ] IIM: All expected trading days are present. [2025-09-16T18:56:53.466907] [DEBUG] [VALIDATOR ] IIM: Primary data validation passed. [2025-09-16T18:56:53.467290] [INFO ] [MAIN ] IIM: Fetched primary data is valid. [2025-09-16T18:56:53.467330] [DEBUG] [LOCAL ] IIM: Saving primary data. [2025-09-16T18:56:53.468000] [DEBUG] [LOCAL ] IIM: Trimmed 18436 records to 11438 (keeping 150 days of data) [2025-09-16T18:56:53.537614] [DEBUG] [LOCAL ] IIM: Primary data saved (11438 records). [2025-09-16T18:56:53.537757] [INFO ] [MAIN ] IIM: Full history saved. [2025-09-16T18:56:53.537850] [INFO ] [MAIN ] Update Progress: 1103/2369 symbols completed. IIM took 19.6s. Estimated remaining: 2h 14m 6s. Rate: 9.4 symbols/min [2025-09-16T18:56:53.707236] [INFO ] [MAIN ] INFA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:56:53.750306] [DEBUG] [LOCAL ] INFA: Loaded 36912 existing records from primary storage. [2025-09-16T18:56:53.750449] [INFO ] [MAIN ] INFA: Loaded 36912 existing records. [2025-09-16T18:56:53.930428] [WARN ] [MAIN ] INFA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:56:53.930724] [INFO ] [MAIN ] INFA: Wiping existing primary data before full refresh. [2025-09-16T18:56:53.930773] [INFO ] [LOCAL ] INFA: Starting deletion of primary data. [2025-09-16T18:56:53.931590] [DEBUG] [LOCAL ] INFA: Deleted primary data file. [2025-09-16T18:56:53.931667] [INFO ] [MAIN ] INFA: Starting full primary refresh. [2025-09-16T18:56:53.986515] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-02. [2025-09-16T18:56:54.292959] [INFO ] [REMOTE ] INFA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:56:54.578158] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-09-16T18:56:54.774923] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-08. [2025-09-16T18:56:55.446610] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-09-16T18:56:55.574926] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-07. [2025-09-16T18:56:56.165575] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-09-16T18:56:56.374923] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-02. [2025-09-16T18:56:56.591632] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T18:56:57.174902] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-08. [2025-09-16T18:56:57.950551] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T18:56:57.974857] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-03. [2025-09-16T18:56:57.997046] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-09-16T18:56:58.774907] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-03. [2025-09-16T18:56:59.574898] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-09. [2025-09-16T18:56:59.717348] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T18:57:00.235116] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T18:57:00.374869] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-04. [2025-09-16T18:57:00.478164] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-09-16T18:57:00.479995] [INFO ] [REMOTE ] IIIN: Full fetch complete (18454 records). [2025-09-16T18:57:00.480065] [DEBUG] [VALIDATOR ] IIIN: Validating primary data. [2025-09-16T18:57:00.849136] [DEBUG] [VALIDATOR ] IIIN: All expected trading days are present. [2025-09-16T18:57:00.849215] [DEBUG] [VALIDATOR ] IIIN: Primary data validation passed. [2025-09-16T18:57:00.849660] [INFO ] [MAIN ] IIIN: Fetched primary data is valid. [2025-09-16T18:57:00.849712] [DEBUG] [LOCAL ] IIIN: Saving primary data. [2025-09-16T18:57:00.850365] [DEBUG] [LOCAL ] IIIN: Trimmed 18454 records to 11751 (keeping 150 days of data) [2025-09-16T18:57:00.925610] [DEBUG] [LOCAL ] IIIN: Primary data saved (11751 records). [2025-09-16T18:57:00.925772] [INFO ] [MAIN ] IIIN: Full history saved. [2025-09-16T18:57:00.925865] [INFO ] [MAIN ] Update Progress: 1104/2369 symbols completed. IIIN took 30.8s. Estimated remaining: 2h 14m 1s. Rate: 9.4 symbols/min [2025-09-16T18:57:01.100743] [INFO ] [MAIN ] INFY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:01.146726] [DEBUG] [LOCAL ] INFY: Loaded 39568 existing records from primary storage. [2025-09-16T18:57:01.146867] [INFO ] [MAIN ] INFY: Loaded 39568 existing records. [2025-09-16T18:57:01.180021] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-09. [2025-09-16T18:57:01.326633] [WARN ] [MAIN ] INFY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:01.326891] [INFO ] [MAIN ] INFY: Wiping existing primary data before full refresh. [2025-09-16T18:57:01.326939] [INFO ] [LOCAL ] INFY: Starting deletion of primary data. [2025-09-16T18:57:01.327911] [DEBUG] [LOCAL ] INFY: Deleted primary data file. [2025-09-16T18:57:01.327963] [INFO ] [MAIN ] INFY: Starting full primary refresh. [2025-09-16T18:57:01.679326] [INFO ] [REMOTE ] INFY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:01.974906] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-02. [2025-09-16T18:57:02.238760] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-09-16T18:57:02.541765] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T18:57:02.545484] [INFO ] [REMOTE ] IIPR: Full fetch complete (32119 records). [2025-09-16T18:57:02.545602] [DEBUG] [VALIDATOR ] IIPR: Validating primary data. [2025-09-16T18:57:02.780084] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-04. [2025-09-16T18:57:02.925035] [DEBUG] [VALIDATOR ] IIPR: All expected trading days are present. [2025-09-16T18:57:02.925114] [DEBUG] [VALIDATOR ] IIPR: Primary data validation passed. [2025-09-16T18:57:02.925716] [INFO ] [MAIN ] IIPR: Fetched primary data is valid. [2025-09-16T18:57:02.925760] [DEBUG] [LOCAL ] IIPR: Saving primary data. [2025-09-16T18:57:02.926486] [DEBUG] [LOCAL ] IIPR: Trimmed 32119 records to 20792 (keeping 150 days of data) [2025-09-16T18:57:03.050452] [DEBUG] [LOCAL ] IIPR: Primary data saved (20792 records). [2025-09-16T18:57:03.050582] [INFO ] [MAIN ] IIPR: Full history saved. [2025-09-16T18:57:03.050695] [INFO ] [MAIN ] Update Progress: 1105/2369 symbols completed. IIPR took 23.9s. Estimated remaining: 2h 13m 50s. Rate: 9.4 symbols/min [2025-09-16T18:57:03.234480] [INFO ] [MAIN ] ING: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:03.272095] [DEBUG] [LOCAL ] ING: Loaded 37270 existing records from primary storage. [2025-09-16T18:57:03.272270] [INFO ] [MAIN ] ING: Loaded 37270 existing records. [2025-09-16T18:57:03.454666] [WARN ] [MAIN ] ING: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:03.455024] [INFO ] [MAIN ] ING: Wiping existing primary data before full refresh. [2025-09-16T18:57:03.455105] [INFO ] [LOCAL ] ING: Starting deletion of primary data. [2025-09-16T18:57:03.456059] [DEBUG] [LOCAL ] ING: Deleted primary data file. [2025-09-16T18:57:03.456121] [INFO ] [MAIN ] ING: Starting full primary refresh. [2025-09-16T18:57:03.576967] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-05. [2025-09-16T18:57:03.720820] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T18:57:03.871726] [INFO ] [REMOTE ] ING: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:04.027320] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-09-16T18:57:04.374930] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-02. [2025-09-16T18:57:05.174915] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-03. [2025-09-16T18:57:05.365291] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T18:57:05.494065] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-09-16T18:57:05.974930] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-03. [2025-09-16T18:57:06.699492] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T18:57:06.774905] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-04. [2025-09-16T18:57:07.581772] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-06. [2025-09-16T18:57:07.600915] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T18:57:08.374869] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-04. [2025-09-16T18:57:08.788186] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-09-16T18:57:08.854745] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-09-16T18:57:09.174915] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-05. [2025-09-16T18:57:09.777048] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T18:57:09.974874] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-07. [2025-09-16T18:57:10.774897] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-05. [2025-09-16T18:57:10.890499] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-09-16T18:57:11.574891] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-05. [2025-09-16T18:57:11.821771] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T18:57:12.374914] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-08. [2025-09-16T18:57:12.652252] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-09-16T18:57:12.871312] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-09-16T18:57:13.174909] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-06. [2025-09-16T18:57:13.663023] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T18:57:13.974905] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-09. [2025-09-16T18:57:14.626890] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T18:57:14.774898] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-07. [2025-09-16T18:57:15.447139] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T18:57:15.453159] [INFO ] [REMOTE ] INFA: Full fetch complete (54722 records). [2025-09-16T18:57:15.453271] [DEBUG] [VALIDATOR ] INFA: Validating primary data. [2025-09-16T18:57:15.580062] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-06. [2025-09-16T18:57:15.836773] [DEBUG] [VALIDATOR ] INFA: All expected trading days are present. [2025-09-16T18:57:15.836850] [DEBUG] [VALIDATOR ] INFA: Primary data validation passed. [2025-09-16T18:57:15.837597] [INFO ] [MAIN ] INFA: Fetched primary data is valid. [2025-09-16T18:57:15.837654] [DEBUG] [LOCAL ] INFA: Saving primary data. [2025-09-16T18:57:15.838742] [DEBUG] [LOCAL ] INFA: Trimmed 54722 records to 37288 (keeping 150 days of data) [2025-09-16T18:57:16.062222] [DEBUG] [LOCAL ] INFA: Primary data saved (37288 records). [2025-09-16T18:57:16.062367] [INFO ] [MAIN ] INFA: Full history saved. [2025-09-16T18:57:16.063296] [INFO ] [MAIN ] Update Progress: 1106/2369 symbols completed. INFA took 22.5s. Estimated remaining: 2h 13m 51s. Rate: 9.4 symbols/min [2025-09-16T18:57:16.233758] [INFO ] [MAIN ] INGM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:16.255293] [DEBUG] [LOCAL ] INGM: Loaded 20005 existing records from primary storage. [2025-09-16T18:57:16.255427] [INFO ] [MAIN ] INGM: Loaded 20005 existing records. [2025-09-16T18:57:16.379748] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-06. [2025-09-16T18:57:16.442018] [WARN ] [MAIN ] INGM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:16.442503] [INFO ] [MAIN ] INGM: Wiping existing primary data before full refresh. [2025-09-16T18:57:16.442615] [INFO ] [LOCAL ] INGM: Starting deletion of primary data. [2025-09-16T18:57:16.443280] [DEBUG] [LOCAL ] INGM: Deleted primary data file. [2025-09-16T18:57:16.443330] [INFO ] [MAIN ] INGM: Starting full primary refresh. [2025-09-16T18:57:16.625270] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T18:57:16.833789] [INFO ] [REMOTE ] INGM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:17.174872] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-08. [2025-09-16T18:57:17.266536] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T18:57:17.979956] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-07. [2025-09-16T18:57:17.996825] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T18:57:18.509726] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-09-16T18:57:18.774931] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-07. [2025-09-16T18:57:19.574904] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-09. [2025-09-16T18:57:19.932010] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-09-16T18:57:20.299112] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-09-16T18:57:20.374928] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-02. [2025-09-16T18:57:20.976772] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T18:57:20.982240] [INFO ] [REMOTE ] IMAX: Full fetch complete (46257 records). [2025-09-16T18:57:20.982388] [DEBUG] [VALIDATOR ] IMAX: Validating primary data. [2025-09-16T18:57:21.183243] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-08. [2025-09-16T18:57:21.365771] [DEBUG] [VALIDATOR ] IMAX: All expected trading days are present. [2025-09-16T18:57:21.365859] [DEBUG] [VALIDATOR ] IMAX: Primary data validation passed. [2025-09-16T18:57:21.366613] [INFO ] [MAIN ] IMAX: Fetched primary data is valid. [2025-09-16T18:57:21.366674] [DEBUG] [LOCAL ] IMAX: Saving primary data. [2025-09-16T18:57:21.367665] [DEBUG] [LOCAL ] IMAX: Trimmed 46257 records to 31866 (keeping 150 days of data) [2025-09-16T18:57:21.558340] [DEBUG] [LOCAL ] IMAX: Primary data saved (31866 records). [2025-09-16T18:57:21.558615] [INFO ] [MAIN ] IMAX: Full history saved. [2025-09-16T18:57:21.558799] [INFO ] [MAIN ] Update Progress: 1107/2369 symbols completed. IMAX took 31.4s. Estimated remaining: 2h 13m 44s. Rate: 9.4 symbols/min [2025-09-16T18:57:21.739844] [INFO ] [MAIN ] INGR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:21.758782] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T18:57:21.791696] [DEBUG] [LOCAL ] INGR: Loaded 23872 existing records from primary storage. [2025-09-16T18:57:21.791917] [INFO ] [MAIN ] INGR: Loaded 23872 existing records. [2025-09-16T18:57:21.973301] [WARN ] [MAIN ] INGR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:21.973552] [INFO ] [MAIN ] INGR: Wiping existing primary data before full refresh. [2025-09-16T18:57:21.973598] [INFO ] [LOCAL ] INGR: Starting deletion of primary data. [2025-09-16T18:57:21.974277] [DEBUG] [LOCAL ] INGR: Deleted primary data file. [2025-09-16T18:57:21.974324] [INFO ] [MAIN ] INGR: Starting full primary refresh. [2025-09-16T18:57:21.985434] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-08. [2025-09-16T18:57:22.331113] [INFO ] [REMOTE ] INGR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:22.774941] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-02. [2025-09-16T18:57:22.980347] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-09-16T18:57:23.574904] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-09. [2025-09-16T18:57:23.911548] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T18:57:24.183675] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-09-16T18:57:24.374908] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-03. [2025-09-16T18:57:25.174878] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-03. [2025-09-16T18:57:25.192171] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T18:57:25.198563] [INFO ] [REMOTE ] ING: Full fetch complete (55237 records). [2025-09-16T18:57:25.198693] [DEBUG] [VALIDATOR ] ING: Validating primary data. [2025-09-16T18:57:25.597194] [DEBUG] [VALIDATOR ] ING: All expected trading days are present. [2025-09-16T18:57:25.597353] [DEBUG] [VALIDATOR ] ING: Primary data validation passed. [2025-09-16T18:57:25.598222] [INFO ] [MAIN ] ING: Fetched primary data is valid. [2025-09-16T18:57:25.598319] [DEBUG] [LOCAL ] ING: Saving primary data. [2025-09-16T18:57:25.599822] [DEBUG] [LOCAL ] ING: Trimmed 55237 records to 37640 (keeping 150 days of data) [2025-09-16T18:57:25.724434] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T18:57:25.861796] [DEBUG] [LOCAL ] ING: Primary data saved (37640 records). [2025-09-16T18:57:25.861941] [INFO ] [MAIN ] ING: Full history saved. [2025-09-16T18:57:25.862023] [INFO ] [MAIN ] Update Progress: 1108/2369 symbols completed. ING took 22.8s. Estimated remaining: 2h 13m 35s. Rate: 9.4 symbols/min [2025-09-16T18:57:25.979980] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-09. [2025-09-16T18:57:26.035410] [INFO ] [MAIN ] INN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:26.064678] [DEBUG] [LOCAL ] INN: Loaded 31262 existing records from primary storage. [2025-09-16T18:57:26.064815] [INFO ] [MAIN ] INN: Loaded 31262 existing records. [2025-09-16T18:57:26.240963] [WARN ] [MAIN ] INN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:26.241235] [INFO ] [MAIN ] INN: Wiping existing primary data before full refresh. [2025-09-16T18:57:26.241279] [INFO ] [LOCAL ] INN: Starting deletion of primary data. [2025-09-16T18:57:26.242016] [DEBUG] [LOCAL ] INN: Deleted primary data file. [2025-09-16T18:57:26.242066] [INFO ] [MAIN ] INN: Starting full primary refresh. [2025-09-16T18:57:26.602316] [INFO ] [REMOTE ] INN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:26.623321] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T18:57:26.774905] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-04. [2025-09-16T18:57:27.574918] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-02. [2025-09-16T18:57:27.762290] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T18:57:27.768697] [INFO ] [REMOTE ] INFY: Full fetch complete (58272 records). [2025-09-16T18:57:27.768798] [DEBUG] [VALIDATOR ] INFY: Validating primary data. [2025-09-16T18:57:28.146286] [DEBUG] [VALIDATOR ] INFY: All expected trading days are present. [2025-09-16T18:57:28.146427] [DEBUG] [VALIDATOR ] INFY: Primary data validation passed. [2025-09-16T18:57:28.147270] [INFO ] [MAIN ] INFY: Fetched primary data is valid. [2025-09-16T18:57:28.147327] [DEBUG] [LOCAL ] INFY: Saving primary data. [2025-09-16T18:57:28.148605] [DEBUG] [LOCAL ] INFY: Trimmed 58272 records to 39957 (keeping 150 days of data) [2025-09-16T18:57:28.257997] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T18:57:28.391525] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-05. [2025-09-16T18:57:28.427533] [DEBUG] [LOCAL ] INFY: Primary data saved (39957 records). [2025-09-16T18:57:28.427709] [INFO ] [MAIN ] INFY: Full history saved. [2025-09-16T18:57:28.427836] [INFO ] [MAIN ] Update Progress: 1109/2369 symbols completed. INFY took 27.5s. Estimated remaining: 2h 13m 25s. Rate: 9.4 symbols/min [2025-09-16T18:57:28.599324] [INFO ] [MAIN ] INR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:28.618333] [DEBUG] [LOCAL ] INR: Loaded 19051 existing records from primary storage. [2025-09-16T18:57:28.618466] [INFO ] [MAIN ] INR: Loaded 19051 existing records. [2025-09-16T18:57:28.791544] [WARN ] [MAIN ] INR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:28.791788] [INFO ] [MAIN ] INR: Wiping existing primary data before full refresh. [2025-09-16T18:57:28.791834] [INFO ] [LOCAL ] INR: Starting deletion of primary data. [2025-09-16T18:57:28.792407] [DEBUG] [LOCAL ] INR: Deleted primary data file. [2025-09-16T18:57:28.792453] [INFO ] [MAIN ] INR: Starting full primary refresh. [2025-09-16T18:57:29.196543] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-04. [2025-09-16T18:57:29.206366] [INFO ] [REMOTE ] INR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:29.211243] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:57:29.782302] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T18:57:29.974920] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-02. [2025-09-16T18:57:30.024830] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-09-16T18:57:30.774928] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-05. [2025-09-16T18:57:31.253282] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-09-16T18:57:31.574906] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-03. [2025-09-16T18:57:31.861952] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-09-16T18:57:32.374910] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-03. [2025-09-16T18:57:32.845867] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T18:57:33.174917] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-04. [2025-09-16T18:57:33.680517] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T18:57:33.974902] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-04. [2025-09-16T18:57:34.579741] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T18:57:34.774928] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-05. [2025-09-16T18:57:35.028916] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-09-16T18:57:35.574905] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-05. [2025-09-16T18:57:35.980762] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-09-16T18:57:36.374871] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-06. [2025-09-16T18:57:36.690236] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T18:57:37.174906] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-06. [2025-09-16T18:57:37.637924] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T18:57:37.974911] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-07. [2025-09-16T18:57:38.184623] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T18:57:38.774903] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-06. [2025-09-16T18:57:39.079485] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-09-16T18:57:39.574914] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-07. [2025-09-16T18:57:39.945423] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-09-16T18:57:40.374876] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-07. [2025-09-16T18:57:40.830232] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T18:57:41.174874] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-08. [2025-09-16T18:57:41.357418] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-09-16T18:57:41.974871] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-06. [2025-09-16T18:57:42.387509] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-09-16T18:57:42.774890] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-09. [2025-09-16T18:57:43.299289] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T18:57:43.574899] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-08. [2025-09-16T18:57:44.015823] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T18:57:44.019308] [INFO ] [REMOTE ] INR: Full fetch complete (29750 records). [2025-09-16T18:57:44.019382] [DEBUG] [VALIDATOR ] INR: Validating primary data. [2025-09-16T18:57:44.384713] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-07. [2025-09-16T18:57:44.397886] [DEBUG] [VALIDATOR ] INR: All expected trading days are present. [2025-09-16T18:57:44.397947] [DEBUG] [VALIDATOR ] INR: Primary data validation passed. [2025-09-16T18:57:44.398619] [INFO ] [MAIN ] INR: Fetched primary data is valid. [2025-09-16T18:57:44.398779] [DEBUG] [LOCAL ] INR: Saving primary data. [2025-09-16T18:57:44.399600] [DEBUG] [LOCAL ] INR: Trimmed 29750 records to 19261 (keeping 150 days of data) [2025-09-16T18:57:44.513497] [DEBUG] [LOCAL ] INR: Primary data saved (19261 records). [2025-09-16T18:57:44.513679] [INFO ] [MAIN ] INR: Full history saved. [2025-09-16T18:57:44.513786] [INFO ] [MAIN ] Update Progress: 1110/2369 symbols completed. INR took 16.1s. Estimated remaining: 2h 13m 29s. Rate: 9.4 symbols/min [2025-09-16T18:57:44.696083] [INFO ] [MAIN ] INSP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:44.720623] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T18:57:44.750132] [DEBUG] [LOCAL ] INSP: Loaded 27714 existing records from primary storage. [2025-09-16T18:57:44.750264] [INFO ] [MAIN ] INSP: Loaded 27714 existing records. [2025-09-16T18:57:44.924587] [WARN ] [MAIN ] INSP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:44.924852] [INFO ] [MAIN ] INSP: Wiping existing primary data before full refresh. [2025-09-16T18:57:44.924898] [INFO ] [LOCAL ] INSP: Starting deletion of primary data. [2025-09-16T18:57:44.925580] [DEBUG] [LOCAL ] INSP: Deleted primary data file. [2025-09-16T18:57:44.925633] [INFO ] [MAIN ] INSP: Starting full primary refresh. [2025-09-16T18:57:45.180024] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-08. [2025-09-16T18:57:45.297287] [INFO ] [REMOTE ] INSP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:45.511680] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-09-16T18:57:45.974859] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-08. [2025-09-16T18:57:46.433325] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-09-16T18:57:46.774906] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-09. [2025-09-16T18:57:47.360258] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-09-16T18:57:47.574870] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-02. [2025-09-16T18:57:47.905615] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T18:57:47.909117] [INFO ] [REMOTE ] INGM: Full fetch complete (31545 records). [2025-09-16T18:57:47.909202] [DEBUG] [VALIDATOR ] INGM: Validating primary data. [2025-09-16T18:57:48.277535] [DEBUG] [VALIDATOR ] INGM: All expected trading days are present. [2025-09-16T18:57:48.277612] [DEBUG] [VALIDATOR ] INGM: Primary data validation passed. [2025-09-16T18:57:48.278293] [INFO ] [MAIN ] INGM: Fetched primary data is valid. [2025-09-16T18:57:48.278340] [DEBUG] [LOCAL ] INGM: Saving primary data. [2025-09-16T18:57:48.279161] [DEBUG] [LOCAL ] INGM: Trimmed 31545 records to 20171 (keeping 150 days of data) [2025-09-16T18:57:48.379729] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-09. [2025-09-16T18:57:48.410071] [DEBUG] [LOCAL ] INGM: Primary data saved (20171 records). [2025-09-16T18:57:48.410218] [INFO ] [MAIN ] INGM: Full history saved. [2025-09-16T18:57:48.410317] [INFO ] [MAIN ] Update Progress: 1111/2369 symbols completed. INGM took 32.3s. Estimated remaining: 2h 13m 20s. Rate: 9.4 symbols/min [2025-09-16T18:57:48.580950] [INFO ] [MAIN ] INSW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:48.611727] [DEBUG] [LOCAL ] INSW: Loaded 29210 existing records from primary storage. [2025-09-16T18:57:48.611903] [INFO ] [MAIN ] INSW: Loaded 29210 existing records. [2025-09-16T18:57:48.786787] [WARN ] [MAIN ] INSW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:48.787041] [INFO ] [MAIN ] INSW: Wiping existing primary data before full refresh. [2025-09-16T18:57:48.787092] [INFO ] [LOCAL ] INSW: Starting deletion of primary data. [2025-09-16T18:57:48.787936] [DEBUG] [LOCAL ] INSW: Deleted primary data file. [2025-09-16T18:57:48.787990] [INFO ] [MAIN ] INSW: Starting full primary refresh. [2025-09-16T18:57:49.154932] [INFO ] [REMOTE ] INSW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:49.174368] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T18:57:49.181687] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-03. [2025-09-16T18:57:49.343299] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T18:57:49.348727] [INFO ] [REMOTE ] INGR: Full fetch complete (36808 records). [2025-09-16T18:57:49.348883] [DEBUG] [VALIDATOR ] INGR: Validating primary data. [2025-09-16T18:57:49.722512] [DEBUG] [VALIDATOR ] INGR: All expected trading days are present. [2025-09-16T18:57:49.722593] [DEBUG] [VALIDATOR ] INGR: Primary data validation passed. [2025-09-16T18:57:49.723246] [INFO ] [MAIN ] INGR: Fetched primary data is valid. [2025-09-16T18:57:49.723301] [DEBUG] [LOCAL ] INGR: Saving primary data. [2025-09-16T18:57:49.724179] [DEBUG] [LOCAL ] INGR: Trimmed 36808 records to 24186 (keeping 150 days of data) [2025-09-16T18:57:49.875243] [DEBUG] [LOCAL ] INGR: Primary data saved (24186 records). [2025-09-16T18:57:49.875394] [INFO ] [MAIN ] INGR: Full history saved. [2025-09-16T18:57:49.875480] [INFO ] [MAIN ] Update Progress: 1112/2369 symbols completed. INGR took 28.3s. Estimated remaining: 2h 13m 8s. Rate: 9.4 symbols/min [2025-09-16T18:57:49.980092] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-09. [2025-09-16T18:57:50.057066] [INFO ] [MAIN ] INVH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:50.095141] [DEBUG] [LOCAL ] INVH: Loaded 39223 existing records from primary storage. [2025-09-16T18:57:50.095318] [INFO ] [MAIN ] INVH: Loaded 39223 existing records. [2025-09-16T18:57:50.276658] [WARN ] [MAIN ] INVH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:50.277011] [INFO ] [MAIN ] INVH: Wiping existing primary data before full refresh. [2025-09-16T18:57:50.277064] [INFO ] [LOCAL ] INVH: Starting deletion of primary data. [2025-09-16T18:57:50.277856] [DEBUG] [LOCAL ] INVH: Deleted primary data file. [2025-09-16T18:57:50.277906] [INFO ] [MAIN ] INVH: Starting full primary refresh. [2025-09-16T18:57:50.442917] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T18:57:50.660138] [INFO ] [REMOTE ] INVH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:50.774949] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-02. [2025-09-16T18:57:51.319112] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T18:57:51.323898] [INFO ] [REMOTE ] INN: Full fetch complete (45094 records). [2025-09-16T18:57:51.324019] [DEBUG] [VALIDATOR ] INN: Validating primary data. [2025-09-16T18:57:51.579976] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-02. [2025-09-16T18:57:51.703267] [DEBUG] [VALIDATOR ] INN: All expected trading days are present. [2025-09-16T18:57:51.703360] [DEBUG] [VALIDATOR ] INN: Primary data validation passed. [2025-09-16T18:57:51.704109] [INFO ] [MAIN ] INN: Fetched primary data is valid. [2025-09-16T18:57:51.704158] [DEBUG] [LOCAL ] INN: Saving primary data. [2025-09-16T18:57:51.705295] [DEBUG] [LOCAL ] INN: Trimmed 45094 records to 31561 (keeping 150 days of data) [2025-09-16T18:57:51.898445] [DEBUG] [LOCAL ] INN: Primary data saved (31561 records). [2025-09-16T18:57:51.898593] [INFO ] [MAIN ] INN: Full history saved. [2025-09-16T18:57:51.898694] [INFO ] [MAIN ] Update Progress: 1113/2369 symbols completed. INN took 26.0s. Estimated remaining: 2h 12m 57s. Rate: 9.4 symbols/min [2025-09-16T18:57:52.074483] [INFO ] [MAIN ] INVX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:57:52.094984] [DEBUG] [LOCAL ] INVX: Loaded 21242 existing records from primary storage. [2025-09-16T18:57:52.095135] [INFO ] [MAIN ] INVX: Loaded 21242 existing records. [2025-09-16T18:57:52.273406] [WARN ] [MAIN ] INVX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:57:52.273733] [INFO ] [MAIN ] INVX: Wiping existing primary data before full refresh. [2025-09-16T18:57:52.273800] [INFO ] [LOCAL ] INVX: Starting deletion of primary data. [2025-09-16T18:57:52.274329] [DEBUG] [LOCAL ] INVX: Deleted primary data file. [2025-09-16T18:57:52.274380] [INFO ] [MAIN ] INVX: Starting full primary refresh. [2025-09-16T18:57:52.377734] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-04. [2025-09-16T18:57:52.648821] [INFO ] [REMOTE ] INVX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:57:52.677542] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-09-16T18:57:52.728062] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-09-16T18:57:53.174966] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-03. [2025-09-16T18:57:53.807622] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T18:57:53.974907] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-05. [2025-09-16T18:57:54.774929] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-02. [2025-09-16T18:57:54.968570] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-09-16T18:57:55.415258] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T18:57:55.574902] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-06. [2025-09-16T18:57:55.747968] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-09-16T18:57:56.374927] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-03. [2025-09-16T18:57:56.890454] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T18:57:57.174962] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-07. [2025-09-16T18:57:57.574925] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T18:57:57.974916] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-04. [2025-09-16T18:57:58.195127] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T18:57:58.774956] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-03. [2025-09-16T18:57:59.452129] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T18:57:59.574982] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-05. [2025-09-16T18:58:00.380058] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-04. [2025-09-16T18:58:00.394136] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T18:58:01.084621] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-09-16T18:58:01.174948] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-04. [2025-09-16T18:58:01.974930] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-08. [2025-09-16T18:58:02.407428] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-09-16T18:58:02.774924] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-06. [2025-09-16T18:58:02.817011] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T18:58:03.473705] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T18:58:03.574894] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-05. [2025-09-16T18:58:03.611094] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-09-16T18:58:04.374941] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-09. [2025-09-16T18:58:05.036363] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T18:58:05.174906] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-06. [2025-09-16T18:58:05.486166] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T18:58:05.491740] [INFO ] [REMOTE ] INSP: Full fetch complete (39660 records). [2025-09-16T18:58:05.491861] [DEBUG] [VALIDATOR ] INSP: Validating primary data. [2025-09-16T18:58:05.870101] [DEBUG] [VALIDATOR ] INSP: All expected trading days are present. [2025-09-16T18:58:05.870183] [DEBUG] [VALIDATOR ] INSP: Primary data validation passed. [2025-09-16T18:58:05.870921] [INFO ] [MAIN ] INSP: Fetched primary data is valid. [2025-09-16T18:58:05.870982] [DEBUG] [LOCAL ] INSP: Saving primary data. [2025-09-16T18:58:05.872006] [DEBUG] [LOCAL ] INSP: Trimmed 39660 records to 28015 (keeping 150 days of data) [2025-09-16T18:58:05.979704] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-07. [2025-09-16T18:58:06.043277] [DEBUG] [LOCAL ] INSP: Primary data saved (28015 records). [2025-09-16T18:58:06.043415] [INFO ] [MAIN ] INSP: Full history saved. [2025-09-16T18:58:06.043537] [INFO ] [MAIN ] Update Progress: 1114/2369 symbols completed. INSP took 21.5s. Estimated remaining: 2h 12m 59s. Rate: 9.4 symbols/min [2025-09-16T18:58:06.215089] [INFO ] [MAIN ] IONQ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:58:06.252352] [DEBUG] [LOCAL ] IONQ: Loaded 39748 existing records from primary storage. [2025-09-16T18:58:06.252496] [INFO ] [MAIN ] IONQ: Loaded 39748 existing records. [2025-09-16T18:58:06.428904] [WARN ] [MAIN ] IONQ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:58:06.429131] [INFO ] [MAIN ] IONQ: Wiping existing primary data before full refresh. [2025-09-16T18:58:06.429176] [INFO ] [LOCAL ] IONQ: Starting deletion of primary data. [2025-09-16T18:58:06.429899] [DEBUG] [LOCAL ] IONQ: Deleted primary data file. [2025-09-16T18:58:06.429949] [INFO ] [MAIN ] IONQ: Starting full primary refresh. [2025-09-16T18:58:06.776983] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-05. [2025-09-16T18:58:06.796846] [INFO ] [REMOTE ] IONQ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:58:06.822364] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T18:58:06.849442] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T18:58:07.574896] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-08. [2025-09-16T18:58:08.374978] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-07. [2025-09-16T18:58:08.566176] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T18:58:08.592989] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-09-16T18:58:09.174950] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-09. [2025-09-16T18:58:09.967327] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T18:58:09.975338] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-06. [2025-09-16T18:58:10.271768] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-09-16T18:58:10.275134] [INFO ] [REMOTE ] INVX: Full fetch complete (31855 records). [2025-09-16T18:58:10.275289] [DEBUG] [VALIDATOR ] INVX: Validating primary data. [2025-09-16T18:58:10.645851] [DEBUG] [VALIDATOR ] INVX: All expected trading days are present. [2025-09-16T18:58:10.645936] [DEBUG] [VALIDATOR ] INVX: Primary data validation passed. [2025-09-16T18:58:10.646561] [INFO ] [MAIN ] INVX: Fetched primary data is valid. [2025-09-16T18:58:10.646602] [DEBUG] [LOCAL ] INVX: Saving primary data. [2025-09-16T18:58:10.647460] [DEBUG] [LOCAL ] INVX: Trimmed 31855 records to 21435 (keeping 150 days of data) [2025-09-16T18:58:10.776513] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-08. [2025-09-16T18:58:10.777071] [DEBUG] [LOCAL ] INVX: Primary data saved (21435 records). [2025-09-16T18:58:10.778446] [INFO ] [MAIN ] INVX: Full history saved. [2025-09-16T18:58:10.778577] [INFO ] [MAIN ] Update Progress: 1115/2369 symbols completed. INVX took 18.9s. Estimated remaining: 2h 12m 51s. Rate: 9.4 symbols/min [2025-09-16T18:58:10.960076] [INFO ] [MAIN ] IOT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:58:10.996926] [DEBUG] [LOCAL ] IOT: Loaded 39479 existing records from primary storage. [2025-09-16T18:58:10.997063] [INFO ] [MAIN ] IOT: Loaded 39479 existing records. [2025-09-16T18:58:11.175832] [WARN ] [MAIN ] IOT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:58:11.176052] [INFO ] [MAIN ] IOT: Wiping existing primary data before full refresh. [2025-09-16T18:58:11.176101] [INFO ] [LOCAL ] IOT: Starting deletion of primary data. [2025-09-16T18:58:11.176930] [DEBUG] [LOCAL ] IOT: Deleted primary data file. [2025-09-16T18:58:11.176984] [INFO ] [MAIN ] IOT: Starting full primary refresh. [2025-09-16T18:58:11.546824] [INFO ] [REMOTE ] IOT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:58:11.574854] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-02. [2025-09-16T18:58:11.677713] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T18:58:12.220321] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T18:58:12.374904] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-02. [2025-09-16T18:58:13.174895] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-07. [2025-09-16T18:58:13.321984] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T18:58:13.974941] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-09. [2025-09-16T18:58:14.774955] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-03. [2025-09-16T18:58:14.964022] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-09-16T18:58:15.003311] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-02 (2.63 seconds). [2025-09-16T18:58:15.456608] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-09-16T18:58:15.461286] [INFO ] [REMOTE ] INSW: Full fetch complete (44610 records). [2025-09-16T18:58:15.461377] [DEBUG] [VALIDATOR ] INSW: Validating primary data. [2025-09-16T18:58:15.580029] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-08. [2025-09-16T18:58:15.837770] [DEBUG] [VALIDATOR ] INSW: All expected trading days are present. [2025-09-16T18:58:15.837846] [DEBUG] [VALIDATOR ] INSW: Primary data validation passed. [2025-09-16T18:58:15.838517] [INFO ] [MAIN ] INSW: Fetched primary data is valid. [2025-09-16T18:58:15.838557] [DEBUG] [LOCAL ] INSW: Saving primary data. [2025-09-16T18:58:15.839489] [DEBUG] [LOCAL ] INSW: Trimmed 44610 records to 29451 (keeping 150 days of data) [2025-09-16T18:58:16.018564] [DEBUG] [LOCAL ] INSW: Primary data saved (29451 records). [2025-09-16T18:58:16.018716] [INFO ] [MAIN ] INSW: Full history saved. [2025-09-16T18:58:16.019392] [INFO ] [MAIN ] Update Progress: 1116/2369 symbols completed. INSW took 27.6s. Estimated remaining: 2h 12m 44s. Rate: 9.4 symbols/min [2025-09-16T18:58:16.195335] [INFO ] [MAIN ] IP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:58:16.236343] [DEBUG] [LOCAL ] IP: Loaded 39314 existing records from primary storage. [2025-09-16T18:58:16.236487] [INFO ] [MAIN ] IP: Loaded 39314 existing records. [2025-09-16T18:58:16.380110] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-03. [2025-09-16T18:58:16.416635] [WARN ] [MAIN ] IP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:58:16.416879] [INFO ] [MAIN ] IP: Wiping existing primary data before full refresh. [2025-09-16T18:58:16.416936] [INFO ] [LOCAL ] IP: Starting deletion of primary data. [2025-09-16T18:58:16.417751] [DEBUG] [LOCAL ] IP: Deleted primary data file. [2025-09-16T18:58:16.417804] [INFO ] [MAIN ] IP: Starting full primary refresh. [2025-09-16T18:58:16.771732] [INFO ] [REMOTE ] IP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:58:16.805629] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-09-16T18:58:17.174861] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-04. [2025-09-16T18:58:17.268673] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T18:58:17.974902] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-09. [2025-09-16T18:58:18.774872] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-02. [2025-09-16T18:58:18.955716] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-03 (2.58 seconds). [2025-09-16T18:58:18.995139] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T18:58:19.247605] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T18:58:19.254045] [INFO ] [REMOTE ] INVH: Full fetch complete (57745 records). [2025-09-16T18:58:19.254175] [DEBUG] [VALIDATOR ] INVH: Validating primary data. [2025-09-16T18:58:19.580030] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-05. [2025-09-16T18:58:19.661965] [DEBUG] [VALIDATOR ] INVH: All expected trading days are present. [2025-09-16T18:58:19.662060] [DEBUG] [VALIDATOR ] INVH: Primary data validation passed. [2025-09-16T18:58:19.662868] [INFO ] [MAIN ] INVH: Fetched primary data is valid. [2025-09-16T18:58:19.662913] [DEBUG] [LOCAL ] INVH: Saving primary data. [2025-09-16T18:58:19.664152] [DEBUG] [LOCAL ] INVH: Trimmed 57745 records to 39610 (keeping 150 days of data) [2025-09-16T18:58:19.906971] [DEBUG] [LOCAL ] INVH: Primary data saved (39610 records). [2025-09-16T18:58:19.907114] [INFO ] [MAIN ] INVH: Full history saved. [2025-09-16T18:58:19.907198] [INFO ] [MAIN ] Update Progress: 1117/2369 symbols completed. INVH took 30.0s. Estimated remaining: 2h 12m 35s. Rate: 9.4 symbols/min [2025-09-16T18:58:20.374875] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-04. [2025-09-16T18:58:20.413002] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T18:58:21.174917] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-03. [2025-09-16T18:58:21.466221] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-09-16T18:58:21.974958] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-06. [2025-09-16T18:58:22.739070] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-04 (2.36 seconds). [2025-09-16T18:58:22.774856] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-05. [2025-09-16T18:58:22.829709] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-09-16T18:58:23.413115] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T18:58:23.574908] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-07. [2025-09-16T18:58:24.374931] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-04. [2025-09-16T18:58:25.413903] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-05 (2.64 seconds). [2025-09-16T18:58:25.425386] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-06. [2025-09-16T18:58:25.616170] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-09-16T18:58:25.974952] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-08. [2025-09-16T18:58:26.181716] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T18:58:26.774927] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-05. [2025-09-16T18:58:27.749030] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-06 (2.32 seconds). [2025-09-16T18:58:27.759885] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-07. [2025-09-16T18:58:27.909630] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T18:58:28.374925] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-09. [2025-09-16T18:58:28.688804] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-09-16T18:58:29.174930] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-06. [2025-09-16T18:58:30.312703] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-09-16T18:58:30.319048] [INFO ] [REMOTE ] IOT: Full fetch complete (58129 records). [2025-09-16T18:58:30.319150] [DEBUG] [VALIDATOR ] IOT: Validating primary data. [2025-09-16T18:58:30.709686] [DEBUG] [VALIDATOR ] IOT: All expected trading days are present. [2025-09-16T18:58:30.709819] [DEBUG] [VALIDATOR ] IOT: Primary data validation passed. [2025-09-16T18:58:30.710613] [INFO ] [MAIN ] IOT: Fetched primary data is valid. [2025-09-16T18:58:30.710764] [DEBUG] [LOCAL ] IOT: Saving primary data. [2025-09-16T18:58:30.712367] [DEBUG] [LOCAL ] IOT: Trimmed 58129 records to 39868 (keeping 150 days of data) [2025-09-16T18:58:30.857946] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-07 (3.10 seconds). [2025-09-16T18:58:30.876382] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T18:58:30.887483] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-08. [2025-09-16T18:58:30.890995] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-07. [2025-09-16T18:58:31.035498] [DEBUG] [LOCAL ] IOT: Primary data saved (39868 records). [2025-09-16T18:58:31.035659] [INFO ] [MAIN ] IOT: Full history saved. [2025-09-16T18:58:31.035758] [INFO ] [MAIN ] Update Progress: 1118/2369 symbols completed. IOT took 20.3s. Estimated remaining: 2h 12m 34s. Rate: 9.4 symbols/min [2025-09-16T18:58:32.804822] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T18:58:32.816322] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-08. [2025-09-16T18:58:33.135718] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-09-16T18:58:33.147513] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-09. [2025-09-16T18:58:34.638426] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T18:58:34.649891] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-09. [2025-09-16T18:58:35.748241] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-09 (2.60 seconds). [2025-09-16T18:58:35.755618] [INFO ] [REMOTE ] IONQ: Full fetch complete (58468 records). [2025-09-16T18:58:35.755753] [DEBUG] [VALIDATOR ] IONQ: Validating primary data. [2025-09-16T18:58:36.148758] [DEBUG] [VALIDATOR ] IONQ: All expected trading days are present. [2025-09-16T18:58:36.148837] [DEBUG] [VALIDATOR ] IONQ: Primary data validation passed. [2025-09-16T18:58:36.149631] [INFO ] [MAIN ] IONQ: Fetched primary data is valid. [2025-09-16T18:58:36.149692] [DEBUG] [LOCAL ] IONQ: Saving primary data. [2025-09-16T18:58:36.151095] [DEBUG] [LOCAL ] IONQ: Trimmed 58468 records to 40138 (keeping 150 days of data) [2025-09-16T18:58:36.392531] [DEBUG] [LOCAL ] IONQ: Primary data saved (40138 records). [2025-09-16T18:58:36.392704] [INFO ] [MAIN ] IONQ: Full history saved. [2025-09-16T18:58:36.392829] [INFO ] [MAIN ] Update Progress: 1119/2369 symbols completed. IONQ took 30.3s. Estimated remaining: 2h 12m 26s. Rate: 9.4 symbols/min [2025-09-16T18:58:36.448027] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-09-16T18:58:36.454287] [INFO ] [REMOTE ] IP: Full fetch complete (58004 records). [2025-09-16T18:58:36.454380] [DEBUG] [VALIDATOR ] IP: Validating primary data. [2025-09-16T18:58:36.872197] [DEBUG] [VALIDATOR ] IP: All expected trading days are present. [2025-09-16T18:58:36.872293] [DEBUG] [VALIDATOR ] IP: Primary data validation passed. [2025-09-16T18:58:36.873094] [INFO ] [MAIN ] IP: Fetched primary data is valid. [2025-09-16T18:58:36.873146] [DEBUG] [LOCAL ] IP: Saving primary data. [2025-09-16T18:58:36.874591] [DEBUG] [LOCAL ] IP: Trimmed 58004 records to 39704 (keeping 150 days of data) [2025-09-16T18:58:37.117658] [DEBUG] [LOCAL ] IP: Primary data saved (39704 records). [2025-09-16T18:58:37.117799] [INFO ] [MAIN ] IP: Full history saved. [2025-09-16T18:58:37.118454] [INFO ] [MAIN ] Update Progress: 1120/2369 symbols completed. IP took 21.1s. Estimated remaining: 2h 12m 13s. Rate: 9.4 symbols/min [2025-09-16T18:58:37.806489] [INFO ] [MAIN ] IQI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:58:37.798117] [INFO ] [MAIN ] IPI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:58:37.799010] [INFO ] [MAIN ] IPG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:58:37.837938] [INFO ] [MAIN ] IQV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:58:37.856839] [DEBUG] [LOCAL ] IPI: Loaded 15432 existing records from primary storage. [2025-09-16T18:58:37.857138] [INFO ] [MAIN ] IPI: Loaded 15432 existing records. [2025-09-16T18:58:37.904551] [DEBUG] [LOCAL ] IQI: Loaded 13437 existing records from primary storage. [2025-09-16T18:58:37.964740] [INFO ] [MAIN ] IQI: Loaded 13437 existing records. [2025-09-16T18:58:38.176771] [WARN ] [MAIN ] IPI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:58:38.199332] [DEBUG] [LOCAL ] IPG: Loaded 39313 existing records from primary storage. [2025-09-16T18:58:38.199563] [INFO ] [MAIN ] IPG: Loaded 39313 existing records. [2025-09-16T18:58:38.200078] [DEBUG] [LOCAL ] IQV: Loaded 37209 existing records from primary storage. [2025-09-16T18:58:38.244925] [INFO ] [MAIN ] IQV: Loaded 37209 existing records. [2025-09-16T18:58:38.384401] [INFO ] [MAIN ] IPI: Wiping existing primary data before full refresh. [2025-09-16T18:58:38.456063] [INFO ] [LOCAL ] IPI: Starting deletion of primary data. [2025-09-16T18:58:38.438628] [WARN ] [MAIN ] IQI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:58:38.478617] [DEBUG] [LOCAL ] IPI: Deleted primary data file. [2025-09-16T18:58:38.493278] [INFO ] [MAIN ] IPI: Starting full primary refresh. [2025-09-16T18:58:38.503578] [INFO ] [MAIN ] IQI: Wiping existing primary data before full refresh. [2025-09-16T18:58:38.518993] [INFO ] [LOCAL ] IQI: Starting deletion of primary data. [2025-09-16T18:58:38.663458] [DEBUG] [LOCAL ] IQI: Deleted primary data file. [2025-09-16T18:58:38.693083] [INFO ] [MAIN ] IQI: Starting full primary refresh. [2025-09-16T18:58:38.692864] [WARN ] [MAIN ] IQV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:58:38.693003] [WARN ] [MAIN ] IPG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:58:38.723857] [INFO ] [MAIN ] IPG: Wiping existing primary data before full refresh. [2025-09-16T18:58:38.739016] [INFO ] [LOCAL ] IPG: Starting deletion of primary data. [2025-09-16T18:58:38.744441] [INFO ] [MAIN ] IQV: Wiping existing primary data before full refresh. [2025-09-16T18:58:38.744559] [INFO ] [LOCAL ] IQV: Starting deletion of primary data. [2025-09-16T18:58:38.753266] [DEBUG] [LOCAL ] IPG: Deleted primary data file. [2025-09-16T18:58:38.785113] [DEBUG] [LOCAL ] IQV: Deleted primary data file. [2025-09-16T18:58:38.768846] [INFO ] [MAIN ] IPG: Starting full primary refresh. [2025-09-16T18:58:38.800719] [INFO ] [MAIN ] IQV: Starting full primary refresh. [2025-09-16T18:58:39.836046] [INFO ] [REMOTE ] IPI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:58:39.904079] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-02. [2025-09-16T18:58:40.037999] [INFO ] [REMOTE ] IQI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:58:40.076513] [INFO ] [REMOTE ] IPG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:58:40.077158] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-02. [2025-09-16T18:58:40.088037] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-02. [2025-09-16T18:58:40.150547] [INFO ] [REMOTE ] IQV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:58:40.151476] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-02. [2025-09-16T18:58:41.120293] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T18:58:41.124868] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-03. [2025-09-16T18:58:41.220515] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-09-16T18:58:41.225485] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-03. [2025-09-16T18:58:41.718800] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T18:58:41.728115] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-03. [2025-09-16T18:58:41.782121] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T18:58:41.792541] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-03. [2025-09-16T18:58:41.804154] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T18:58:41.808236] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-04. [2025-09-16T18:58:41.907591] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-09-16T18:58:41.912759] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-04. [2025-09-16T18:58:42.831022] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-09-16T18:58:42.836326] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-05. [2025-09-16T18:58:42.901918] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-09-16T18:58:42.907365] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-05. [2025-09-16T18:58:43.078587] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T18:58:43.565000] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T18:58:43.575716] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-04. [2025-09-16T18:58:43.704748] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-09-16T18:58:43.731614] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T18:58:44.374899] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-06. [2025-09-16T18:58:45.174907] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-06. [2025-09-16T18:58:45.376320] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-09-16T18:58:45.399540] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-09-16T18:58:45.715268] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-09-16T18:58:45.974918] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-04. [2025-09-16T18:58:46.774915] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-07. [2025-09-16T18:58:47.579694] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-07. [2025-09-16T18:58:47.598332] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T18:58:47.770852] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T18:58:48.374993] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-05. [2025-09-16T18:58:48.709765] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-09-16T18:58:49.174961] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-08. [2025-09-16T18:58:49.892715] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T18:58:49.974952] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-06. [2025-09-16T18:58:50.521195] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-09-16T18:58:50.774952] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-09. [2025-09-16T18:58:51.574894] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-08. [2025-09-16T18:58:51.728076] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-09-16T18:58:51.814203] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-09-16T18:58:51.819540] [INFO ] [REMOTE ] IQI: Full fetch complete (20074 records). [2025-09-16T18:58:51.820161] [DEBUG] [VALIDATOR ] IQI: Validating primary data. [2025-09-16T18:58:52.364268] [DEBUG] [VALIDATOR ] IQI: All expected trading days are present. [2025-09-16T18:58:52.364405] [DEBUG] [VALIDATOR ] IQI: Primary data validation passed. [2025-09-16T18:58:52.365183] [INFO ] [MAIN ] IQI: Fetched primary data is valid. [2025-09-16T18:58:52.365284] [DEBUG] [LOCAL ] IQI: Saving primary data. [2025-09-16T18:58:52.366599] [DEBUG] [LOCAL ] IQI: Trimmed 20074 records to 13543 (keeping 150 days of data) [2025-09-16T18:58:52.419606] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-05. [2025-09-16T18:58:52.497010] [DEBUG] [LOCAL ] IQI: Primary data saved (13543 records). [2025-09-16T18:58:52.497173] [INFO ] [MAIN ] IQI: Full history saved. [2025-09-16T18:58:52.497334] [INFO ] [MAIN ] Update Progress: 1121/2369 symbols completed. IQI took 15.4s. Estimated remaining: 2h 12m 17s. Rate: 9.4 symbols/min [2025-09-16T18:58:52.681027] [INFO ] [MAIN ] IR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:58:52.697756] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-09-16T18:58:52.743935] [DEBUG] [LOCAL ] IR: Loaded 38888 existing records from primary storage. [2025-09-16T18:58:52.744125] [INFO ] [MAIN ] IR: Loaded 38888 existing records. [2025-09-16T18:58:52.926585] [WARN ] [MAIN ] IR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:58:52.926884] [INFO ] [MAIN ] IR: Wiping existing primary data before full refresh. [2025-09-16T18:58:52.926933] [INFO ] [LOCAL ] IR: Starting deletion of primary data. [2025-09-16T18:58:52.927995] [DEBUG] [LOCAL ] IR: Deleted primary data file. [2025-09-16T18:58:52.928049] [INFO ] [MAIN ] IR: Starting full primary refresh. [2025-09-16T18:58:53.180181] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-07. [2025-09-16T18:58:53.292202] [INFO ] [REMOTE ] IR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:58:53.974995] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-09. [2025-09-16T18:58:54.225307] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T18:58:54.774908] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-06. [2025-09-16T18:58:54.829350] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T18:58:54.920855] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T18:58:54.923877] [INFO ] [REMOTE ] IPI: Full fetch complete (22884 records). [2025-09-16T18:58:54.924082] [DEBUG] [VALIDATOR ] IPI: Validating primary data. [2025-09-16T18:58:55.302602] [DEBUG] [VALIDATOR ] IPI: All expected trading days are present. [2025-09-16T18:58:55.302696] [DEBUG] [VALIDATOR ] IPI: Primary data validation passed. [2025-09-16T18:58:55.303220] [INFO ] [MAIN ] IPI: Fetched primary data is valid. [2025-09-16T18:58:55.303266] [DEBUG] [LOCAL ] IPI: Saving primary data. [2025-09-16T18:58:55.303860] [DEBUG] [LOCAL ] IPI: Trimmed 22884 records to 15552 (keeping 150 days of data) [2025-09-16T18:58:55.397350] [DEBUG] [LOCAL ] IPI: Primary data saved (15552 records). [2025-09-16T18:58:55.397504] [INFO ] [MAIN ] IPI: Full history saved. [2025-09-16T18:58:55.397617] [INFO ] [MAIN ] Update Progress: 1122/2369 symbols completed. IPI took 18.3s. Estimated remaining: 2h 12m 7s. Rate: 9.4 symbols/min [2025-09-16T18:58:55.576458] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-02. [2025-09-16T18:58:55.578037] [INFO ] [MAIN ] IRM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:58:55.614987] [DEBUG] [LOCAL ] IRM: Loaded 36011 existing records from primary storage. [2025-09-16T18:58:55.615152] [INFO ] [MAIN ] IRM: Loaded 36011 existing records. [2025-09-16T18:58:55.798780] [WARN ] [MAIN ] IRM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:58:55.799116] [INFO ] [MAIN ] IRM: Wiping existing primary data before full refresh. [2025-09-16T18:58:55.799173] [INFO ] [LOCAL ] IRM: Starting deletion of primary data. [2025-09-16T18:58:55.800133] [DEBUG] [LOCAL ] IRM: Deleted primary data file. [2025-09-16T18:58:55.800185] [INFO ] [MAIN ] IRM: Starting full primary refresh. [2025-09-16T18:58:56.169386] [INFO ] [REMOTE ] IRM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:58:56.374894] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-08. [2025-09-16T18:58:56.450324] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T18:58:57.183186] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-07. [2025-09-16T18:58:57.201421] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T18:58:57.714085] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T18:58:57.974912] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-09. [2025-09-16T18:58:58.703879] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-09-16T18:58:58.774935] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-08. [2025-09-16T18:58:59.574913] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-03. [2025-09-16T18:58:59.646099] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-09-16T18:58:59.652980] [INFO ] [REMOTE ] IPG: Full fetch complete (58003 records). [2025-09-16T18:58:59.653136] [DEBUG] [VALIDATOR ] IPG: Validating primary data. [2025-09-16T18:59:00.062403] [DEBUG] [VALIDATOR ] IPG: All expected trading days are present. [2025-09-16T18:59:00.062498] [DEBUG] [VALIDATOR ] IPG: Primary data validation passed. [2025-09-16T18:59:00.063312] [INFO ] [MAIN ] IPG: Fetched primary data is valid. [2025-09-16T18:59:00.063359] [DEBUG] [LOCAL ] IPG: Saving primary data. [2025-09-16T18:59:00.064560] [DEBUG] [LOCAL ] IPG: Trimmed 58003 records to 39697 (keeping 150 days of data) [2025-09-16T18:59:00.317936] [DEBUG] [LOCAL ] IPG: Primary data saved (39697 records). [2025-09-16T18:59:00.318094] [INFO ] [MAIN ] IPG: Full history saved. [2025-09-16T18:59:00.318184] [INFO ] [MAIN ] Update Progress: 1123/2369 symbols completed. IPG took 23.2s. Estimated remaining: 2h 11m 59s. Rate: 9.4 symbols/min [2025-09-16T18:59:00.390371] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-02. [2025-09-16T18:59:00.528824] [INFO ] [MAIN ] IRS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:00.564070] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T18:59:00.580688] [DEBUG] [LOCAL ] IRS: Loaded 16167 existing records from primary storage. [2025-09-16T18:59:00.580890] [INFO ] [MAIN ] IRS: Loaded 16167 existing records. [2025-09-16T18:59:00.756368] [WARN ] [MAIN ] IRS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:00.756595] [INFO ] [MAIN ] IRS: Wiping existing primary data before full refresh. [2025-09-16T18:59:00.756659] [INFO ] [LOCAL ] IRS: Starting deletion of primary data. [2025-09-16T18:59:00.757185] [DEBUG] [LOCAL ] IRS: Deleted primary data file. [2025-09-16T18:59:00.757234] [INFO ] [MAIN ] IRS: Starting full primary refresh. [2025-09-16T18:59:01.111204] [INFO ] [REMOTE ] IRS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:01.145545] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-09-16T18:59:01.175011] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-02. [2025-09-16T18:59:01.974887] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-09. [2025-09-16T18:59:02.020038] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T18:59:02.089412] [DEBUG] [REMOTE ] IQV: Error fetching primary data for 2025-09: 'API call returned an empty response.' [2025-09-16T18:59:02.089785] [WARN ] [REMOTE ] IQV: No data returned for month 2025-09 [2025-09-16T18:59:02.089873] [INFO ] [REMOTE ] IQV: Full fetch complete (51148 records). [2025-09-16T18:59:02.089942] [DEBUG] [VALIDATOR ] IQV: Validating primary data. [2025-09-16T18:59:02.091481] [FATAL] [VALIDATOR ] IQV: Data is stale - most recent date (2025-08-29) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T18:59:02.091561] [FATAL] [MAIN ] IQV: Fetched primary data is invalid. Refusing to save it. [2025-09-16T18:59:02.091604] [WARN ] [MAIN ] IQV: Deleting existing primary and derivative data due to validation failure. [2025-09-16T18:59:02.091667] [INFO ] [LOCAL ] IQV: Starting deletion of primary data. [2025-09-16T18:59:02.091727] [DEBUG] [LOCAL ] IQV: No primary data file found to delete. [2025-09-16T18:59:02.091764] [INFO ] [LOCAL ] IQV: Starting deletion of derivative data. [2025-09-16T18:59:02.092099] [DEBUG] [LOCAL ] IQV: Deleted derivative data file. [2025-09-16T18:59:02.092185] [INFO ] [MAIN ] Update Progress: 1124/2369 symbols completed. IQV took 25.0s. Estimated remaining: 2h 11m 48s. Rate: 9.4 symbols/min [2025-09-16T18:59:02.268738] [INFO ] [MAIN ] IRT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:02.281345] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-09-16T18:59:02.325661] [DEBUG] [LOCAL ] IRT: Loaded 37200 existing records from primary storage. [2025-09-16T18:59:02.325809] [INFO ] [MAIN ] IRT: Loaded 37200 existing records. [2025-09-16T18:59:02.498678] [WARN ] [MAIN ] IRT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:02.498914] [INFO ] [MAIN ] IRT: Wiping existing primary data before full refresh. [2025-09-16T18:59:02.498959] [INFO ] [LOCAL ] IRT: Starting deletion of primary data. [2025-09-16T18:59:02.499893] [DEBUG] [LOCAL ] IRT: Deleted primary data file. [2025-09-16T18:59:02.499953] [INFO ] [MAIN ] IRT: Starting full primary refresh. [2025-09-16T18:59:02.779972] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-04. [2025-09-16T18:59:02.852709] [INFO ] [REMOTE ] IRT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:03.574893] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-03. [2025-09-16T18:59:04.374922] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-02. [2025-09-16T18:59:04.558048] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-09-16T18:59:05.174952] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-05. [2025-09-16T18:59:05.362542] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-09-16T18:59:05.974908] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-04. [2025-09-16T18:59:06.475053] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-02 (2.10 seconds). [2025-09-16T18:59:06.745677] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T18:59:06.774857] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-06. [2025-09-16T18:59:07.394661] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T18:59:07.574856] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-05. [2025-09-16T18:59:08.091631] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T18:59:08.374897] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-07. [2025-09-16T18:59:09.174904] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-03. [2025-09-16T18:59:09.438258] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-09-16T18:59:09.974880] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-03. [2025-09-16T18:59:10.031807] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-09-16T18:59:10.775004] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-08. [2025-09-16T18:59:10.842849] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T18:59:11.186709] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-09-16T18:59:11.574882] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-04. [2025-09-16T18:59:12.374928] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-04. [2025-09-16T18:59:12.432369] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T18:59:13.174942] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-06. [2025-09-16T18:59:13.307398] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T18:59:13.347055] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-09-16T18:59:13.974900] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-05. [2025-09-16T18:59:14.774878] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-09. [2025-09-16T18:59:14.962774] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-09-16T18:59:15.574880] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-05. [2025-09-16T18:59:15.655964] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T18:59:16.374896] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-07. [2025-09-16T18:59:16.456042] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T18:59:16.462305] [INFO ] [REMOTE ] IR: Full fetch complete (57330 records). [2025-09-16T18:59:16.462407] [DEBUG] [VALIDATOR ] IR: Validating primary data. [2025-09-16T18:59:16.804153] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-09-16T18:59:16.877734] [DEBUG] [VALIDATOR ] IR: All expected trading days are present. [2025-09-16T18:59:16.877808] [DEBUG] [VALIDATOR ] IR: Primary data validation passed. [2025-09-16T18:59:16.878567] [INFO ] [MAIN ] IR: Fetched primary data is valid. [2025-09-16T18:59:16.878610] [DEBUG] [LOCAL ] IR: Saving primary data. [2025-09-16T18:59:16.879750] [DEBUG] [LOCAL ] IR: Trimmed 57330 records to 39275 (keeping 150 days of data) [2025-09-16T18:59:17.125559] [DEBUG] [LOCAL ] IR: Primary data saved (39275 records). [2025-09-16T18:59:17.125731] [INFO ] [MAIN ] IR: Full history saved. [2025-09-16T18:59:17.125841] [INFO ] [MAIN ] Update Progress: 1125/2369 symbols completed. IR took 24.6s. Estimated remaining: 2h 11m 51s. Rate: 9.4 symbols/min [2025-09-16T18:59:17.183697] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-06. [2025-09-16T18:59:17.341838] [INFO ] [MAIN ] ISD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:17.353762] [DEBUG] [LOCAL ] ISD: Loaded 11068 existing records from primary storage. [2025-09-16T18:59:17.353882] [INFO ] [MAIN ] ISD: Loaded 11068 existing records. [2025-09-16T18:59:17.534465] [WARN ] [MAIN ] ISD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:17.534881] [INFO ] [MAIN ] ISD: Wiping existing primary data before full refresh. [2025-09-16T18:59:17.535088] [INFO ] [LOCAL ] ISD: Starting deletion of primary data. [2025-09-16T18:59:17.535583] [DEBUG] [LOCAL ] ISD: Deleted primary data file. [2025-09-16T18:59:17.535811] [INFO ] [MAIN ] ISD: Starting full primary refresh. [2025-09-16T18:59:17.926851] [INFO ] [REMOTE ] ISD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:17.928898] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-09-16T18:59:17.974852] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-08. [2025-09-16T18:59:18.208897] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-09-16T18:59:18.774906] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-02. [2025-09-16T18:59:19.574898] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-06. [2025-09-16T18:59:19.621845] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-09-16T18:59:19.771807] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-09-16T18:59:20.374920] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-09. [2025-09-16T18:59:21.174913] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-03. [2025-09-16T18:59:21.250726] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T18:59:21.953008] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-09-16T18:59:21.959262] [INFO ] [REMOTE ] IRM: Full fetch complete (54142 records). [2025-09-16T18:59:21.959370] [DEBUG] [VALIDATOR ] IRM: Validating primary data. [2025-09-16T18:59:21.980707] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-07. [2025-09-16T18:59:22.343490] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T18:59:22.366014] [DEBUG] [VALIDATOR ] IRM: All expected trading days are present. [2025-09-16T18:59:22.366115] [DEBUG] [VALIDATOR ] IRM: Primary data validation passed. [2025-09-16T18:59:22.367041] [INFO ] [MAIN ] IRM: Fetched primary data is valid. [2025-09-16T18:59:22.367092] [DEBUG] [LOCAL ] IRM: Saving primary data. [2025-09-16T18:59:22.368445] [DEBUG] [LOCAL ] IRM: Trimmed 54142 records to 36386 (keeping 150 days of data) [2025-09-16T18:59:22.601007] [DEBUG] [LOCAL ] IRM: Primary data saved (36386 records). [2025-09-16T18:59:22.601168] [INFO ] [MAIN ] IRM: Full history saved. [2025-09-16T18:59:22.601267] [INFO ] [MAIN ] Update Progress: 1126/2369 symbols completed. IRM took 27.2s. Estimated remaining: 2h 11m 43s. Rate: 9.4 symbols/min [2025-09-16T18:59:22.775531] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-04. [2025-09-16T18:59:22.775450] [INFO ] [MAIN ] IT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:22.809156] [DEBUG] [LOCAL ] IT: Loaded 30721 existing records from primary storage. [2025-09-16T18:59:22.809305] [INFO ] [MAIN ] IT: Loaded 30721 existing records. [2025-09-16T18:59:22.987570] [WARN ] [MAIN ] IT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:22.987825] [INFO ] [MAIN ] IT: Wiping existing primary data before full refresh. [2025-09-16T18:59:22.987876] [INFO ] [LOCAL ] IT: Starting deletion of primary data. [2025-09-16T18:59:22.988717] [DEBUG] [LOCAL ] IT: Deleted primary data file. [2025-09-16T18:59:22.988766] [INFO ] [MAIN ] IT: Starting full primary refresh. [2025-09-16T18:59:23.296362] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-09-16T18:59:23.362149] [INFO ] [REMOTE ] IT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:23.513247] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-09-16T18:59:23.574859] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-02. [2025-09-16T18:59:24.374928] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-05. [2025-09-16T18:59:24.935155] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T18:59:25.174940] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-07. [2025-09-16T18:59:25.316277] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-09-16T18:59:25.974904] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-03. [2025-09-16T18:59:26.774939] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-06. [2025-09-16T18:59:26.913196] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T18:59:27.474519] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-09-16T18:59:27.574920] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-04. [2025-09-16T18:59:27.766006] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-09-16T18:59:28.374924] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-08. [2025-09-16T18:59:29.174868] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-07. [2025-09-16T18:59:29.237603] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T18:59:29.344985] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T18:59:29.974938] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-09. [2025-09-16T18:59:30.102007] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-09-16T18:59:30.774913] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-08. [2025-09-16T18:59:31.152080] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T18:59:31.156497] [INFO ] [REMOTE ] IRS: Full fetch complete (22838 records). [2025-09-16T18:59:31.156628] [DEBUG] [VALIDATOR ] IRS: Validating primary data. [2025-09-16T18:59:31.469326] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-09-16T18:59:31.538544] [DEBUG] [VALIDATOR ] IRS: All expected trading days are present. [2025-09-16T18:59:31.538620] [DEBUG] [VALIDATOR ] IRS: Primary data validation passed. [2025-09-16T18:59:31.539157] [INFO ] [MAIN ] IRS: Fetched primary data is valid. [2025-09-16T18:59:31.539203] [DEBUG] [LOCAL ] IRS: Saving primary data. [2025-09-16T18:59:31.539892] [DEBUG] [LOCAL ] IRS: Trimmed 22838 records to 16397 (keeping 150 days of data) [2025-09-16T18:59:31.580885] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-09. [2025-09-16T18:59:31.636295] [DEBUG] [LOCAL ] IRS: Primary data saved (16397 records). [2025-09-16T18:59:31.636434] [INFO ] [MAIN ] IRS: Full history saved. [2025-09-16T18:59:31.636523] [INFO ] [MAIN ] Update Progress: 1127/2369 symbols completed. IRS took 31.3s. Estimated remaining: 2h 11m 40s. Rate: 9.4 symbols/min [2025-09-16T18:59:31.808568] [INFO ] [MAIN ] ITGR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:31.830630] [DEBUG] [LOCAL ] ITGR: Loaded 22027 existing records from primary storage. [2025-09-16T18:59:31.830789] [INFO ] [MAIN ] ITGR: Loaded 22027 existing records. [2025-09-16T18:59:32.005205] [WARN ] [MAIN ] ITGR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:32.005447] [INFO ] [MAIN ] ITGR: Wiping existing primary data before full refresh. [2025-09-16T18:59:32.005495] [INFO ] [LOCAL ] ITGR: Starting deletion of primary data. [2025-09-16T18:59:32.006023] [DEBUG] [LOCAL ] ITGR: Deleted primary data file. [2025-09-16T18:59:32.006074] [INFO ] [MAIN ] ITGR: Starting full primary refresh. [2025-09-16T18:59:32.357786] [INFO ] [REMOTE ] ITGR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:32.374847] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-02. [2025-09-16T18:59:32.534913] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T18:59:32.537008] [INFO ] [REMOTE ] ISD: Full fetch complete (17302 records). [2025-09-16T18:59:32.537098] [DEBUG] [VALIDATOR ] ISD: Validating primary data. [2025-09-16T18:59:32.899284] [DEBUG] [VALIDATOR ] ISD: All expected trading days are present. [2025-09-16T18:59:32.899360] [DEBUG] [VALIDATOR ] ISD: Primary data validation passed. [2025-09-16T18:59:32.899709] [INFO ] [MAIN ] ISD: Fetched primary data is valid. [2025-09-16T18:59:32.899752] [DEBUG] [LOCAL ] ISD: Saving primary data. [2025-09-16T18:59:32.900310] [DEBUG] [LOCAL ] ISD: Trimmed 17302 records to 11160 (keeping 150 days of data) [2025-09-16T18:59:32.969182] [DEBUG] [LOCAL ] ISD: Primary data saved (11160 records). [2025-09-16T18:59:32.969314] [INFO ] [MAIN ] ISD: Full history saved. [2025-09-16T18:59:32.969398] [INFO ] [MAIN ] Update Progress: 1128/2369 symbols completed. ISD took 15.8s. Estimated remaining: 2h 11m 28s. Rate: 9.4 symbols/min [2025-09-16T18:59:33.144166] [INFO ] [MAIN ] ITT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:33.167691] [DEBUG] [LOCAL ] ITT: Loaded 23217 existing records from primary storage. [2025-09-16T18:59:33.167821] [INFO ] [MAIN ] ITT: Loaded 23217 existing records. [2025-09-16T18:59:33.180780] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-05. [2025-09-16T18:59:33.354614] [WARN ] [MAIN ] ITT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:33.355020] [INFO ] [MAIN ] ITT: Wiping existing primary data before full refresh. [2025-09-16T18:59:33.355083] [INFO ] [LOCAL ] ITT: Starting deletion of primary data. [2025-09-16T18:59:33.356039] [DEBUG] [LOCAL ] ITT: Deleted primary data file. [2025-09-16T18:59:33.356113] [INFO ] [MAIN ] ITT: Starting full primary refresh. [2025-09-16T18:59:33.711189] [INFO ] [REMOTE ] ITT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:33.725214] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-09-16T18:59:33.974855] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-03. [2025-09-16T18:59:34.774874] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-08. [2025-09-16T18:59:34.808211] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-09-16T18:59:35.277142] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T18:59:35.574892] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-06. [2025-09-16T18:59:36.351417] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T18:59:36.374840] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-09. [2025-09-16T18:59:36.925522] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T18:59:37.174908] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-04. [2025-09-16T18:59:37.965863] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T18:59:37.971881] [INFO ] [REMOTE ] IRT: Full fetch complete (54788 records). [2025-09-16T18:59:37.971977] [DEBUG] [VALIDATOR ] IRT: Validating primary data. [2025-09-16T18:59:37.983852] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-07. [2025-09-16T18:59:38.361582] [DEBUG] [VALIDATOR ] IRT: All expected trading days are present. [2025-09-16T18:59:38.361731] [DEBUG] [VALIDATOR ] IRT: Primary data validation passed. [2025-09-16T18:59:38.362497] [INFO ] [MAIN ] IRT: Fetched primary data is valid. [2025-09-16T18:59:38.362543] [DEBUG] [LOCAL ] IRT: Saving primary data. [2025-09-16T18:59:38.363786] [DEBUG] [LOCAL ] IRT: Trimmed 54788 records to 37574 (keeping 150 days of data) [2025-09-16T18:59:38.606444] [DEBUG] [LOCAL ] IRT: Primary data saved (37574 records). [2025-09-16T18:59:38.606652] [INFO ] [MAIN ] IRT: Full history saved. [2025-09-16T18:59:38.607631] [INFO ] [MAIN ] Update Progress: 1129/2369 symbols completed. IRT took 36.5s. Estimated remaining: 2h 11m 21s. Rate: 9.4 symbols/min [2025-09-16T18:59:38.775665] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-02. [2025-09-16T18:59:38.778063] [INFO ] [MAIN ] ITUB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:38.814668] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T18:59:38.862653] [DEBUG] [LOCAL ] ITUB: Loaded 39382 existing records from primary storage. [2025-09-16T18:59:38.862839] [INFO ] [MAIN ] ITUB: Loaded 39382 existing records. [2025-09-16T18:59:39.036543] [WARN ] [MAIN ] ITUB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:39.036792] [INFO ] [MAIN ] ITUB: Wiping existing primary data before full refresh. [2025-09-16T18:59:39.036839] [INFO ] [LOCAL ] ITUB: Starting deletion of primary data. [2025-09-16T18:59:39.037726] [DEBUG] [LOCAL ] ITUB: Deleted primary data file. [2025-09-16T18:59:39.037774] [INFO ] [MAIN ] ITUB: Starting full primary refresh. [2025-09-16T18:59:39.396181] [INFO ] [REMOTE ] ITUB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:39.428881] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T18:59:39.574873] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-02. [2025-09-16T18:59:40.131336] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-09-16T18:59:40.374923] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-03. [2025-09-16T18:59:41.174917] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-05. [2025-09-16T18:59:41.267261] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T18:59:41.863613] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-09-16T18:59:41.974901] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-08. [2025-09-16T18:59:42.503773] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T18:59:42.774904] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-03. [2025-09-16T18:59:43.574886] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-06. [2025-09-16T18:59:43.847356] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-09-16T18:59:44.383865] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-04. [2025-09-16T18:59:44.393373] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-09-16T18:59:44.540593] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T18:59:45.174924] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-09. [2025-09-16T18:59:45.931468] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-09-16T18:59:45.974852] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-05. [2025-09-16T18:59:46.774955] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-07. [2025-09-16T18:59:46.872358] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T18:59:46.879055] [INFO ] [REMOTE ] IT: Full fetch complete (43835 records). [2025-09-16T18:59:46.879154] [DEBUG] [VALIDATOR ] IT: Validating primary data. [2025-09-16T18:59:47.262588] [DEBUG] [VALIDATOR ] IT: All expected trading days are present. [2025-09-16T18:59:47.262683] [DEBUG] [VALIDATOR ] IT: Primary data validation passed. [2025-09-16T18:59:47.263356] [INFO ] [MAIN ] IT: Fetched primary data is valid. [2025-09-16T18:59:47.263399] [DEBUG] [LOCAL ] IT: Saving primary data. [2025-09-16T18:59:47.264408] [DEBUG] [LOCAL ] IT: Trimmed 43835 records to 31073 (keeping 150 days of data) [2025-09-16T18:59:47.465573] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T18:59:47.488738] [DEBUG] [LOCAL ] IT: Primary data saved (31073 records). [2025-09-16T18:59:47.488883] [INFO ] [MAIN ] IT: Full history saved. [2025-09-16T18:59:47.488970] [INFO ] [MAIN ] Update Progress: 1130/2369 symbols completed. IT took 24.9s. Estimated remaining: 2h 11m 17s. Rate: 9.4 symbols/min [2025-09-16T18:59:47.581503] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-04. [2025-09-16T18:59:47.664341] [INFO ] [MAIN ] ITW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:47.700560] [DEBUG] [LOCAL ] ITW: Loaded 33742 existing records from primary storage. [2025-09-16T18:59:47.700716] [INFO ] [MAIN ] ITW: Loaded 33742 existing records. [2025-09-16T18:59:47.881175] [WARN ] [MAIN ] ITW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:47.881453] [INFO ] [MAIN ] ITW: Wiping existing primary data before full refresh. [2025-09-16T18:59:47.881500] [INFO ] [LOCAL ] ITW: Starting deletion of primary data. [2025-09-16T18:59:47.882366] [DEBUG] [LOCAL ] ITW: Deleted primary data file. [2025-09-16T18:59:47.882416] [INFO ] [MAIN ] ITW: Starting full primary refresh. [2025-09-16T18:59:48.253991] [INFO ] [REMOTE ] ITW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:48.274399] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T18:59:48.374898] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-08. [2025-09-16T18:59:49.174882] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-06. [2025-09-16T18:59:49.236008] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T18:59:49.532909] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T18:59:49.974902] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-05. [2025-09-16T18:59:50.369688] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-09-16T18:59:50.774908] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-09. [2025-09-16T18:59:51.574902] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-02. [2025-09-16T18:59:51.786961] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T18:59:51.811473] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-09-16T18:59:51.815360] [INFO ] [REMOTE ] ITGR: Full fetch complete (33997 records). [2025-09-16T18:59:51.815481] [DEBUG] [VALIDATOR ] ITGR: Validating primary data. [2025-09-16T18:59:52.187359] [DEBUG] [VALIDATOR ] ITGR: All expected trading days are present. [2025-09-16T18:59:52.187452] [DEBUG] [VALIDATOR ] ITGR: Primary data validation passed. [2025-09-16T18:59:52.188134] [INFO ] [MAIN ] ITGR: Fetched primary data is valid. [2025-09-16T18:59:52.188199] [DEBUG] [LOCAL ] ITGR: Saving primary data. [2025-09-16T18:59:52.189231] [DEBUG] [LOCAL ] ITGR: Trimmed 33997 records to 22250 (keeping 150 days of data) [2025-09-16T18:59:52.322208] [DEBUG] [LOCAL ] ITGR: Primary data saved (22250 records). [2025-09-16T18:59:52.322341] [INFO ] [MAIN ] ITGR: Full history saved. [2025-09-16T18:59:52.322425] [INFO ] [MAIN ] Update Progress: 1131/2369 symbols completed. ITGR took 20.7s. Estimated remaining: 2h 11m 9s. Rate: 9.4 symbols/min [2025-09-16T18:59:52.386388] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-07. [2025-09-16T18:59:52.499268] [INFO ] [MAIN ] IVR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:52.533017] [DEBUG] [LOCAL ] IVR: Loaded 34880 existing records from primary storage. [2025-09-16T18:59:52.533156] [INFO ] [MAIN ] IVR: Loaded 34880 existing records. [2025-09-16T18:59:52.710087] [WARN ] [MAIN ] IVR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:52.710324] [INFO ] [MAIN ] IVR: Wiping existing primary data before full refresh. [2025-09-16T18:59:52.710377] [INFO ] [LOCAL ] IVR: Starting deletion of primary data. [2025-09-16T18:59:52.711264] [DEBUG] [LOCAL ] IVR: Deleted primary data file. [2025-09-16T18:59:52.711314] [INFO ] [MAIN ] IVR: Starting full primary refresh. [2025-09-16T18:59:53.070904] [INFO ] [REMOTE ] IVR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:53.174881] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-02. [2025-09-16T18:59:53.247921] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T18:59:53.819246] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-09-16T18:59:53.974904] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-03. [2025-09-16T18:59:54.774899] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-08. [2025-09-16T18:59:54.829319] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-09-16T18:59:55.574927] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-03. [2025-09-16T18:59:55.799131] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-09-16T18:59:55.838008] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-09-16T18:59:56.374913] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-04. [2025-09-16T18:59:57.174903] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-09. [2025-09-16T18:59:57.290783] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T18:59:57.558927] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T18:59:57.974922] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-05. [2025-09-16T18:59:58.417765] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T18:59:58.421362] [INFO ] [REMOTE ] ITT: Full fetch complete (36469 records). [2025-09-16T18:59:58.421472] [DEBUG] [VALIDATOR ] ITT: Validating primary data. [2025-09-16T18:59:58.779541] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-04. [2025-09-16T18:59:58.802081] [DEBUG] [VALIDATOR ] ITT: All expected trading days are present. [2025-09-16T18:59:58.802182] [DEBUG] [VALIDATOR ] ITT: Primary data validation passed. [2025-09-16T18:59:58.802830] [INFO ] [MAIN ] ITT: Fetched primary data is valid. [2025-09-16T18:59:58.802886] [DEBUG] [LOCAL ] ITT: Saving primary data. [2025-09-16T18:59:58.803800] [DEBUG] [LOCAL ] ITT: Trimmed 36469 records to 23458 (keeping 150 days of data) [2025-09-16T18:59:58.953809] [DEBUG] [LOCAL ] ITT: Primary data saved (23458 records). [2025-09-16T18:59:58.953938] [INFO ] [MAIN ] ITT: Full history saved. [2025-09-16T18:59:58.954069] [INFO ] [MAIN ] Update Progress: 1132/2369 symbols completed. ITT took 26.0s. Estimated remaining: 2h 11m 3s. Rate: 9.4 symbols/min [2025-09-16T18:59:59.153438] [INFO ] [MAIN ] IVT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T18:59:59.175682] [DEBUG] [LOCAL ] IVT: Loaded 22298 existing records from primary storage. [2025-09-16T18:59:59.175815] [INFO ] [MAIN ] IVT: Loaded 22298 existing records. [2025-09-16T18:59:59.359700] [WARN ] [MAIN ] IVT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T18:59:59.359968] [INFO ] [MAIN ] IVT: Wiping existing primary data before full refresh. [2025-09-16T18:59:59.360024] [INFO ] [LOCAL ] IVT: Starting deletion of primary data. [2025-09-16T18:59:59.360555] [DEBUG] [LOCAL ] IVT: Deleted primary data file. [2025-09-16T18:59:59.360602] [INFO ] [MAIN ] IVT: Starting full primary refresh. [2025-09-16T18:59:59.580973] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-06. [2025-09-16T18:59:59.721677] [INFO ] [REMOTE ] IVT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T18:59:59.750980] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-09-16T19:00:00.374895] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-02. [2025-09-16T19:00:00.766842] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-09-16T19:00:01.174958] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-05. [2025-09-16T19:00:01.379896] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-09-16T19:00:01.974940] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-07. [2025-09-16T19:00:02.330685] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-09-16T19:00:02.774893] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-03. [2025-09-16T19:00:03.367945] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-05 (2.19 seconds). [2025-09-16T19:00:03.574877] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-06. [2025-09-16T19:00:03.936084] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-09-16T19:00:04.272605] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-09-16T19:00:04.374851] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-04. [2025-09-16T19:00:05.174946] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-08. [2025-09-16T19:00:05.230273] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T19:00:05.642059] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-09-16T19:00:05.974966] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-05. [2025-09-16T19:00:06.774931] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-06. [2025-09-16T19:00:07.238334] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-09-16T19:00:07.290985] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T19:00:07.574901] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-06. [2025-09-16T19:00:08.086966] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T19:00:08.374906] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-07. [2025-09-16T19:00:08.789052] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-09-16T19:00:09.174943] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-07. [2025-09-16T19:00:09.974905] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-09. [2025-09-16T19:00:10.186724] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T19:00:10.774959] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-07. [2025-09-16T19:00:11.015781] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-09-16T19:00:11.557852] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-09-16T19:00:11.564402] [INFO ] [REMOTE ] ITUB: Full fetch complete (57962 records). [2025-09-16T19:00:11.564554] [DEBUG] [VALIDATOR ] ITUB: Validating primary data. [2025-09-16T19:00:11.580081] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-08. [2025-09-16T19:00:11.971264] [DEBUG] [VALIDATOR ] ITUB: All expected trading days are present. [2025-09-16T19:00:11.971358] [DEBUG] [VALIDATOR ] ITUB: Primary data validation passed. [2025-09-16T19:00:11.972163] [INFO ] [MAIN ] ITUB: Fetched primary data is valid. [2025-09-16T19:00:11.972211] [DEBUG] [LOCAL ] ITUB: Saving primary data. [2025-09-16T19:00:11.973500] [DEBUG] [LOCAL ] ITUB: Trimmed 57962 records to 39769 (keeping 150 days of data) [2025-09-16T19:00:12.099567] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-09-16T19:00:12.245538] [DEBUG] [LOCAL ] ITUB: Primary data saved (39769 records). [2025-09-16T19:00:12.245748] [INFO ] [MAIN ] ITUB: Full history saved. [2025-09-16T19:00:12.246427] [INFO ] [MAIN ] Update Progress: 1133/2369 symbols completed. ITUB took 33.6s. Estimated remaining: 2h 11m 5s. Rate: 9.4 symbols/min [2025-09-16T19:00:12.389386] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-08. [2025-09-16T19:00:12.422159] [INFO ] [MAIN ] IVZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:00:12.463976] [DEBUG] [LOCAL ] IVZ: Loaded 39170 existing records from primary storage. [2025-09-16T19:00:12.464135] [INFO ] [MAIN ] IVZ: Loaded 39170 existing records. [2025-09-16T19:00:12.648776] [WARN ] [MAIN ] IVZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:00:12.649041] [INFO ] [MAIN ] IVZ: Wiping existing primary data before full refresh. [2025-09-16T19:00:12.649089] [INFO ] [LOCAL ] IVZ: Starting deletion of primary data. [2025-09-16T19:00:12.650050] [DEBUG] [LOCAL ] IVZ: Deleted primary data file. [2025-09-16T19:00:12.650101] [INFO ] [MAIN ] IVZ: Starting full primary refresh. [2025-09-16T19:00:13.009618] [INFO ] [REMOTE ] IVZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:00:13.180604] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-02. [2025-09-16T19:00:13.182930] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-09-16T19:00:13.694991] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T19:00:13.974927] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-09. [2025-09-16T19:00:14.774929] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-08. [2025-09-16T19:00:15.012231] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T19:00:15.067135] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T19:00:15.071468] [INFO ] [REMOTE ] IVT: Full fetch complete (33845 records). [2025-09-16T19:00:15.071699] [DEBUG] [VALIDATOR ] IVT: Validating primary data. [2025-09-16T19:00:15.465662] [DEBUG] [VALIDATOR ] IVT: All expected trading days are present. [2025-09-16T19:00:15.465783] [DEBUG] [VALIDATOR ] IVT: Primary data validation passed. [2025-09-16T19:00:15.466967] [INFO ] [MAIN ] IVT: Fetched primary data is valid. [2025-09-16T19:00:15.467025] [DEBUG] [LOCAL ] IVT: Saving primary data. [2025-09-16T19:00:15.468418] [DEBUG] [LOCAL ] IVT: Trimmed 33845 records to 22503 (keeping 150 days of data) [2025-09-16T19:00:15.582498] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-09. [2025-09-16T19:00:15.609909] [DEBUG] [LOCAL ] IVT: Primary data saved (22503 records). [2025-09-16T19:00:15.610050] [INFO ] [MAIN ] IVT: Full history saved. [2025-09-16T19:00:15.610155] [INFO ] [MAIN ] Update Progress: 1134/2369 symbols completed. IVT took 16.7s. Estimated remaining: 2h 10m 55s. Rate: 9.4 symbols/min [2025-09-16T19:00:15.793894] [INFO ] [MAIN ] IX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:00:15.812024] [DEBUG] [LOCAL ] IX: Loaded 16193 existing records from primary storage. [2025-09-16T19:00:15.812170] [INFO ] [MAIN ] IX: Loaded 16193 existing records. [2025-09-16T19:00:15.994638] [WARN ] [MAIN ] IX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:00:15.994955] [INFO ] [MAIN ] IX: Wiping existing primary data before full refresh. [2025-09-16T19:00:15.995014] [INFO ] [LOCAL ] IX: Starting deletion of primary data. [2025-09-16T19:00:15.995504] [DEBUG] [LOCAL ] IX: Deleted primary data file. [2025-09-16T19:00:15.995557] [INFO ] [MAIN ] IX: Starting full primary refresh. [2025-09-16T19:00:16.369436] [INFO ] [REMOTE ] IX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:00:16.380470] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-02. [2025-09-16T19:00:16.401276] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-09-16T19:00:16.904538] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T19:00:16.910146] [INFO ] [REMOTE ] ITW: Full fetch complete (50587 records). [2025-09-16T19:00:16.910271] [DEBUG] [VALIDATOR ] ITW: Validating primary data. [2025-09-16T19:00:17.177914] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-09. [2025-09-16T19:00:17.289382] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-09-16T19:00:17.304846] [DEBUG] [VALIDATOR ] ITW: All expected trading days are present. [2025-09-16T19:00:17.304900] [DEBUG] [VALIDATOR ] ITW: Primary data validation passed. [2025-09-16T19:00:17.305614] [INFO ] [MAIN ] ITW: Fetched primary data is valid. [2025-09-16T19:00:17.305680] [DEBUG] [LOCAL ] ITW: Saving primary data. [2025-09-16T19:00:17.306635] [DEBUG] [LOCAL ] ITW: Trimmed 50587 records to 34070 (keeping 150 days of data) [2025-09-16T19:00:17.530310] [DEBUG] [LOCAL ] ITW: Primary data saved (34070 records). [2025-09-16T19:00:17.530462] [INFO ] [MAIN ] ITW: Full history saved. [2025-09-16T19:00:17.530549] [INFO ] [MAIN ] Update Progress: 1135/2369 symbols completed. ITW took 30.0s. Estimated remaining: 2h 10m 44s. Rate: 9.4 symbols/min [2025-09-16T19:00:17.707356] [INFO ] [MAIN ] J: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:00:17.736214] [DEBUG] [LOCAL ] J: Loaded 29311 existing records from primary storage. [2025-09-16T19:00:17.736351] [INFO ] [MAIN ] J: Loaded 29311 existing records. [2025-09-16T19:00:17.914442] [WARN ] [MAIN ] J: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:00:17.914692] [INFO ] [MAIN ] J: Wiping existing primary data before full refresh. [2025-09-16T19:00:17.914748] [INFO ] [LOCAL ] J: Starting deletion of primary data. [2025-09-16T19:00:17.915447] [DEBUG] [LOCAL ] J: Deleted primary data file. [2025-09-16T19:00:17.915494] [INFO ] [MAIN ] J: Starting full primary refresh. [2025-09-16T19:00:17.979990] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-03. [2025-09-16T19:00:18.295907] [INFO ] [REMOTE ] J: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:00:18.650289] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T19:00:18.655919] [INFO ] [REMOTE ] IVR: Full fetch complete (51803 records). [2025-09-16T19:00:18.656018] [DEBUG] [VALIDATOR ] IVR: Validating primary data. [2025-09-16T19:00:18.780066] [INFO ] [REMOTE ] J: Fetching primary data for 2025-02. [2025-09-16T19:00:19.039104] [DEBUG] [VALIDATOR ] IVR: All expected trading days are present. [2025-09-16T19:00:19.039204] [DEBUG] [VALIDATOR ] IVR: Primary data validation passed. [2025-09-16T19:00:19.040129] [INFO ] [MAIN ] IVR: Fetched primary data is valid. [2025-09-16T19:00:19.040181] [DEBUG] [LOCAL ] IVR: Saving primary data. [2025-09-16T19:00:19.041483] [DEBUG] [LOCAL ] IVR: Trimmed 51803 records to 35217 (keeping 150 days of data) [2025-09-16T19:00:19.174230] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-09-16T19:00:19.282462] [DEBUG] [LOCAL ] IVR: Primary data saved (35217 records). [2025-09-16T19:00:19.282668] [INFO ] [MAIN ] IVR: Full history saved. [2025-09-16T19:00:19.282802] [INFO ] [MAIN ] Update Progress: 1136/2369 symbols completed. IVR took 27.0s. Estimated remaining: 2h 10m 32s. Rate: 9.4 symbols/min [2025-09-16T19:00:19.458516] [INFO ] [MAIN ] JBGS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:00:19.497782] [DEBUG] [LOCAL ] JBGS: Loaded 31240 existing records from primary storage. [2025-09-16T19:00:19.497985] [INFO ] [MAIN ] JBGS: Loaded 31240 existing records. [2025-09-16T19:00:19.580011] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-03. [2025-09-16T19:00:19.688392] [WARN ] [MAIN ] JBGS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:00:19.688695] [INFO ] [MAIN ] JBGS: Wiping existing primary data before full refresh. [2025-09-16T19:00:19.688760] [INFO ] [LOCAL ] JBGS: Starting deletion of primary data. [2025-09-16T19:00:19.689571] [DEBUG] [LOCAL ] JBGS: Deleted primary data file. [2025-09-16T19:00:19.689633] [INFO ] [MAIN ] JBGS: Starting full primary refresh. [2025-09-16T19:00:20.074810] [INFO ] [REMOTE ] JBGS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:00:20.290767] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-09-16T19:00:20.374893] [INFO ] [REMOTE ] J: Fetching primary data for 2025-03. [2025-09-16T19:00:21.174876] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-04. [2025-09-16T19:00:21.216124] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T19:00:21.852035] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T19:00:21.974936] [INFO ] [REMOTE ] J: Fetching primary data for 2025-04. [2025-09-16T19:00:22.341815] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-09-16T19:00:22.774951] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-05. [2025-09-16T19:00:23.574891] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-02. [2025-09-16T19:00:23.646932] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T19:00:23.983988] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-09-16T19:00:24.374905] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-04. [2025-09-16T19:00:24.556390] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-09-16T19:00:25.174928] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-06. [2025-09-16T19:00:25.974902] [INFO ] [REMOTE ] J: Fetching primary data for 2025-05. [2025-09-16T19:00:26.308150] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-09-16T19:00:26.678471] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-04 (2.30 seconds). [2025-09-16T19:00:26.774896] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-05. [2025-09-16T19:00:27.550219] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T19:00:27.574847] [INFO ] [REMOTE ] J: Fetching primary data for 2025-06. [2025-09-16T19:00:28.374911] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-03. [2025-09-16T19:00:28.609777] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-09-16T19:00:28.950117] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T19:00:29.174909] [INFO ] [REMOTE ] J: Fetching primary data for 2025-07. [2025-09-16T19:00:29.889938] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T19:00:29.974917] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-04. [2025-09-16T19:00:30.622305] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T19:00:30.774865] [INFO ] [REMOTE ] J: Fetching primary data for 2025-08. [2025-09-16T19:00:31.522948] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-09-16T19:00:31.574858] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-05. [2025-09-16T19:00:32.062448] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T19:00:32.374887] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-06. [2025-09-16T19:00:33.064599] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T19:00:33.174872] [INFO ] [REMOTE ] J: Fetching primary data for 2025-09. [2025-09-16T19:00:33.980026] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-07. [2025-09-16T19:00:33.993689] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T19:00:34.618435] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T19:00:34.623500] [INFO ] [REMOTE ] J: Full fetch complete (44406 records). [2025-09-16T19:00:34.623672] [DEBUG] [VALIDATOR ] J: Validating primary data. [2025-09-16T19:00:34.776389] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-06. [2025-09-16T19:00:35.019910] [DEBUG] [VALIDATOR ] J: All expected trading days are present. [2025-09-16T19:00:35.019997] [DEBUG] [VALIDATOR ] J: Primary data validation passed. [2025-09-16T19:00:35.020771] [INFO ] [MAIN ] J: Fetched primary data is valid. [2025-09-16T19:00:35.020844] [DEBUG] [LOCAL ] J: Saving primary data. [2025-09-16T19:00:35.021946] [DEBUG] [LOCAL ] J: Trimmed 44406 records to 29574 (keeping 150 days of data) [2025-09-16T19:00:35.147002] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T19:00:35.230828] [DEBUG] [LOCAL ] J: Primary data saved (29574 records). [2025-09-16T19:00:35.230963] [INFO ] [MAIN ] J: Full history saved. [2025-09-16T19:00:35.231076] [INFO ] [MAIN ] Update Progress: 1137/2369 symbols completed. J took 17.7s. Estimated remaining: 2h 10m 36s. Rate: 9.4 symbols/min [2025-09-16T19:00:35.406713] [INFO ] [MAIN ] JBI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:00:35.440941] [DEBUG] [LOCAL ] JBI: Loaded 31610 existing records from primary storage. [2025-09-16T19:00:35.441098] [INFO ] [MAIN ] JBI: Loaded 31610 existing records. [2025-09-16T19:00:35.585819] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-07. [2025-09-16T19:00:35.625482] [WARN ] [MAIN ] JBI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:00:35.625780] [INFO ] [MAIN ] JBI: Wiping existing primary data before full refresh. [2025-09-16T19:00:35.625843] [INFO ] [LOCAL ] JBI: Starting deletion of primary data. [2025-09-16T19:00:35.626480] [DEBUG] [LOCAL ] JBI: Deleted primary data file. [2025-09-16T19:00:35.626549] [INFO ] [MAIN ] JBI: Starting full primary refresh. [2025-09-16T19:00:35.994319] [INFO ] [REMOTE ] JBI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:00:36.374872] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-02. [2025-09-16T19:00:36.421139] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T19:00:37.174979] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-08. [2025-09-16T19:00:37.659149] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-09-16T19:00:37.707795] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-09-16T19:00:37.858707] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-09-16T19:00:37.974950] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-08. [2025-09-16T19:00:38.774902] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-03. [2025-09-16T19:00:39.574921] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-09. [2025-09-16T19:00:39.721090] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T19:00:40.151948] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-09-16T19:00:40.225392] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-09-16T19:00:40.227785] [INFO ] [REMOTE ] IX: Full fetch complete (23812 records). [2025-09-16T19:00:40.227865] [DEBUG] [VALIDATOR ] IX: Validating primary data. [2025-09-16T19:00:40.380181] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-04. [2025-09-16T19:00:40.604633] [DEBUG] [VALIDATOR ] IX: All expected trading days are present. [2025-09-16T19:00:40.604743] [DEBUG] [VALIDATOR ] IX: Primary data validation passed. [2025-09-16T19:00:40.605242] [INFO ] [MAIN ] IX: Fetched primary data is valid. [2025-09-16T19:00:40.605283] [DEBUG] [LOCAL ] IX: Saving primary data. [2025-09-16T19:00:40.606009] [DEBUG] [LOCAL ] IX: Trimmed 23812 records to 16330 (keeping 150 days of data) [2025-09-16T19:00:40.709805] [DEBUG] [LOCAL ] IX: Primary data saved (16330 records). [2025-09-16T19:00:40.709963] [INFO ] [MAIN ] IX: Full history saved. [2025-09-16T19:00:40.710074] [INFO ] [MAIN ] Update Progress: 1138/2369 symbols completed. IX took 25.1s. Estimated remaining: 2h 10m 29s. Rate: 9.4 symbols/min [2025-09-16T19:00:40.886043] [INFO ] [MAIN ] JBL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:00:40.924729] [DEBUG] [LOCAL ] JBL: Loaded 35106 existing records from primary storage. [2025-09-16T19:00:40.924879] [INFO ] [MAIN ] JBL: Loaded 35106 existing records. [2025-09-16T19:00:41.129005] [WARN ] [MAIN ] JBL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:00:41.129298] [INFO ] [MAIN ] JBL: Wiping existing primary data before full refresh. [2025-09-16T19:00:41.129355] [INFO ] [LOCAL ] JBL: Starting deletion of primary data. [2025-09-16T19:00:41.130240] [DEBUG] [LOCAL ] JBL: Deleted primary data file. [2025-09-16T19:00:41.130299] [INFO ] [MAIN ] JBL: Starting full primary refresh. [2025-09-16T19:00:41.180022] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-07. [2025-09-16T19:00:41.490112] [INFO ] [REMOTE ] JBL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:00:41.974897] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-02. [2025-09-16T19:00:42.030578] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T19:00:42.649082] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T19:00:42.774896] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-09. [2025-09-16T19:00:43.562760] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T19:00:43.574839] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-03. [2025-09-16T19:00:44.085091] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T19:00:44.093332] [INFO ] [REMOTE ] IVZ: Full fetch complete (57733 records). [2025-09-16T19:00:44.093454] [DEBUG] [VALIDATOR ] IVZ: Validating primary data. [2025-09-16T19:00:44.380048] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-05. [2025-09-16T19:00:44.491263] [DEBUG] [VALIDATOR ] IVZ: All expected trading days are present. [2025-09-16T19:00:44.491367] [DEBUG] [VALIDATOR ] IVZ: Primary data validation passed. [2025-09-16T19:00:44.492465] [INFO ] [MAIN ] IVZ: Fetched primary data is valid. [2025-09-16T19:00:44.492539] [DEBUG] [LOCAL ] IVZ: Saving primary data. [2025-09-16T19:00:44.493850] [DEBUG] [LOCAL ] IVZ: Trimmed 57733 records to 39555 (keeping 150 days of data) [2025-09-16T19:00:44.753713] [DEBUG] [LOCAL ] IVZ: Primary data saved (39555 records). [2025-09-16T19:00:44.753868] [INFO ] [MAIN ] IVZ: Full history saved. [2025-09-16T19:00:44.754596] [INFO ] [MAIN ] Update Progress: 1139/2369 symbols completed. IVZ took 32.5s. Estimated remaining: 2h 10m 20s. Rate: 9.4 symbols/min [2025-09-16T19:00:44.936077] [INFO ] [MAIN ] JBND: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:00:44.958009] [DEBUG] [LOCAL ] JBND: Loaded 18819 existing records from primary storage. [2025-09-16T19:00:44.958156] [INFO ] [MAIN ] JBND: Loaded 18819 existing records. [2025-09-16T19:00:45.144332] [WARN ] [MAIN ] JBND: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:00:45.144585] [INFO ] [MAIN ] JBND: Wiping existing primary data before full refresh. [2025-09-16T19:00:45.144637] [INFO ] [LOCAL ] JBND: Starting deletion of primary data. [2025-09-16T19:00:45.145248] [DEBUG] [LOCAL ] JBND: Deleted primary data file. [2025-09-16T19:00:45.145298] [INFO ] [MAIN ] JBND: Starting full primary refresh. [2025-09-16T19:00:45.188944] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-08. [2025-09-16T19:00:45.518315] [INFO ] [REMOTE ] JBND: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:00:45.555046] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-09-16T19:00:45.597352] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T19:00:45.974896] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-04. [2025-09-16T19:00:46.675917] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T19:00:46.774917] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-09. [2025-09-16T19:00:47.574903] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-06. [2025-09-16T19:00:47.763000] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T19:00:48.112158] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T19:00:48.117934] [INFO ] [REMOTE ] JBGS: Full fetch complete (45969 records). [2025-09-16T19:00:48.118074] [DEBUG] [VALIDATOR ] JBGS: Validating primary data. [2025-09-16T19:00:48.379985] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-02. [2025-09-16T19:00:48.521615] [DEBUG] [VALIDATOR ] JBGS: All expected trading days are present. [2025-09-16T19:00:48.521727] [DEBUG] [VALIDATOR ] JBGS: Primary data validation passed. [2025-09-16T19:00:48.522507] [INFO ] [MAIN ] JBGS: Fetched primary data is valid. [2025-09-16T19:00:48.522552] [DEBUG] [LOCAL ] JBGS: Saving primary data. [2025-09-16T19:00:48.523777] [DEBUG] [LOCAL ] JBGS: Trimmed 45969 records to 31570 (keeping 150 days of data) [2025-09-16T19:00:48.726873] [DEBUG] [LOCAL ] JBGS: Primary data saved (31570 records). [2025-09-16T19:00:48.727033] [INFO ] [MAIN ] JBGS: Full history saved. [2025-09-16T19:00:48.727149] [INFO ] [MAIN ] Update Progress: 1140/2369 symbols completed. JBGS took 29.4s. Estimated remaining: 2h 10m 11s. Rate: 9.4 symbols/min [2025-09-16T19:00:48.896542] [INFO ] [MAIN ] JBS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:00:48.896756] [DEBUG] [LOCAL ] JBS: No existing data file found in primary storage. [2025-09-16T19:00:48.896797] [WARN ] [MAIN ] JBS: No existing data found, performing full refresh instead. [2025-09-16T19:00:48.896919] [INFO ] [MAIN ] JBS: Wiping existing primary data before full refresh. [2025-09-16T19:00:48.896959] [INFO ] [LOCAL ] JBS: Starting deletion of primary data. [2025-09-16T19:00:48.897001] [DEBUG] [LOCAL ] JBS: No primary data file found to delete. [2025-09-16T19:00:48.897034] [INFO ] [MAIN ] JBS: Starting full primary refresh. [2025-09-16T19:00:49.184354] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-05. [2025-09-16T19:00:49.262825] [INFO ] [REMOTE ] JBS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:00:49.286525] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T19:00:49.500039] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-09-16T19:00:49.974952] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-03. [2025-09-16T19:00:50.774952] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-07. [2025-09-16T19:00:50.939803] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T19:00:51.117489] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T19:00:51.574919] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-06. [2025-09-16T19:00:52.302107] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-09-16T19:00:52.374884] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-08. [2025-09-16T19:00:53.076533] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-09-16T19:00:53.174920] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-04. [2025-09-16T19:00:53.509934] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T19:00:53.975006] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-07. [2025-09-16T19:00:54.363349] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-09-16T19:00:54.774918] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-09. [2025-09-16T19:00:55.574895] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-05. [2025-09-16T19:00:55.849910] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T19:00:55.876582] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-09-16T19:00:55.883026] [INFO ] [REMOTE ] JBI: Full fetch complete (48081 records). [2025-09-16T19:00:55.883161] [DEBUG] [VALIDATOR ] JBI: Validating primary data. [2025-09-16T19:00:56.263831] [DEBUG] [VALIDATOR ] JBI: All expected trading days are present. [2025-09-16T19:00:56.263924] [DEBUG] [VALIDATOR ] JBI: Primary data validation passed. [2025-09-16T19:00:56.264869] [INFO ] [MAIN ] JBI: Fetched primary data is valid. [2025-09-16T19:00:56.264941] [DEBUG] [LOCAL ] JBI: Saving primary data. [2025-09-16T19:00:56.266089] [DEBUG] [LOCAL ] JBI: Trimmed 48081 records to 31900 (keeping 150 days of data) [2025-09-16T19:00:56.375412] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-02. [2025-09-16T19:00:56.470726] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-09-16T19:00:56.473708] [DEBUG] [LOCAL ] JBI: Primary data saved (31900 records). [2025-09-16T19:00:56.479359] [INFO ] [MAIN ] JBI: Full history saved. [2025-09-16T19:00:56.479813] [INFO ] [MAIN ] Update Progress: 1141/2369 symbols completed. JBI took 21.2s. Estimated remaining: 2h 10m 6s. Rate: 9.4 symbols/min [2025-09-16T19:00:56.651186] [INFO ] [MAIN ] JBTM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:00:56.655418] [DEBUG] [REMOTE ] JBS: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:00:56.655806] [WARN ] [REMOTE ] JBS: No data returned for month 2025-02 [2025-09-16T19:00:56.675989] [DEBUG] [LOCAL ] JBTM: Loaded 23791 existing records from primary storage. [2025-09-16T19:00:56.676145] [INFO ] [MAIN ] JBTM: Loaded 23791 existing records. [2025-09-16T19:00:56.854531] [WARN ] [MAIN ] JBTM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:00:56.854807] [INFO ] [MAIN ] JBTM: Wiping existing primary data before full refresh. [2025-09-16T19:00:56.854854] [INFO ] [LOCAL ] JBTM: Starting deletion of primary data. [2025-09-16T19:00:56.855504] [DEBUG] [LOCAL ] JBTM: Deleted primary data file. [2025-09-16T19:00:56.855550] [INFO ] [MAIN ] JBTM: Starting full primary refresh. [2025-09-16T19:00:57.179002] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-03. [2025-09-16T19:00:57.213551] [INFO ] [REMOTE ] JBTM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:00:57.400061] [DEBUG] [REMOTE ] JBS: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:00:57.401070] [WARN ] [REMOTE ] JBS: No data returned for month 2025-03 [2025-09-16T19:00:57.974966] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-06. [2025-09-16T19:00:58.774934] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-08. [2025-09-16T19:00:59.027301] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T19:00:59.574880] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-07. [2025-09-16T19:01:00.374912] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-02. [2025-09-16T19:01:00.514542] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-09-16T19:01:00.910081] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-09-16T19:01:01.174913] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-09. [2025-09-16T19:01:01.461143] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T19:01:01.974913] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-04. [2025-09-16T19:01:02.172261] [DEBUG] [REMOTE ] JBS: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:01:02.172608] [WARN ] [REMOTE ] JBS: No data returned for month 2025-04 [2025-09-16T19:01:02.774921] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-05. [2025-09-16T19:01:02.978713] [DEBUG] [REMOTE ] JBS: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:01:02.979043] [WARN ] [REMOTE ] JBS: No data returned for month 2025-05 [2025-09-16T19:01:03.129125] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-09-16T19:01:03.135202] [INFO ] [REMOTE ] JBL: Full fetch complete (52453 records). [2025-09-16T19:01:03.135349] [DEBUG] [VALIDATOR ] JBL: Validating primary data. [2025-09-16T19:01:03.525947] [DEBUG] [VALIDATOR ] JBL: All expected trading days are present. [2025-09-16T19:01:03.526037] [DEBUG] [VALIDATOR ] JBL: Primary data validation passed. [2025-09-16T19:01:03.527008] [INFO ] [MAIN ] JBL: Fetched primary data is valid. [2025-09-16T19:01:03.527056] [DEBUG] [LOCAL ] JBL: Saving primary data. [2025-09-16T19:01:03.528224] [DEBUG] [LOCAL ] JBL: Trimmed 52453 records to 35457 (keeping 150 days of data) [2025-09-16T19:01:03.590428] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-06. [2025-09-16T19:01:03.763047] [DEBUG] [LOCAL ] JBL: Primary data saved (35457 records). [2025-09-16T19:01:03.763207] [INFO ] [MAIN ] JBL: Full history saved. [2025-09-16T19:01:03.763304] [INFO ] [MAIN ] Update Progress: 1142/2369 symbols completed. JBL took 23.1s. Estimated remaining: 2h 10m 1s. Rate: 9.4 symbols/min [2025-09-16T19:01:03.933200] [INFO ] [MAIN ] JCE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:01:03.939842] [DEBUG] [LOCAL ] JCE: Loaded 4686 existing records from primary storage. [2025-09-16T19:01:03.939953] [INFO ] [MAIN ] JCE: Loaded 4686 existing records. [2025-09-16T19:01:04.114636] [WARN ] [MAIN ] JCE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:01:04.114884] [INFO ] [MAIN ] JCE: Wiping existing primary data before full refresh. [2025-09-16T19:01:04.114930] [INFO ] [LOCAL ] JCE: Starting deletion of primary data. [2025-09-16T19:01:04.115208] [DEBUG] [LOCAL ] JCE: Deleted primary data file. [2025-09-16T19:01:04.115256] [INFO ] [MAIN ] JCE: Starting full primary refresh. [2025-09-16T19:01:04.380372] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-08. [2025-09-16T19:01:04.468779] [INFO ] [REMOTE ] JCE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:01:04.820229] [INFO ] [REMOTE ] JBS: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T19:01:05.174950] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-07. [2025-09-16T19:01:05.689133] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T19:01:05.974949] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-02. [2025-09-16T19:01:06.595103] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-09-16T19:01:06.743514] [INFO ] [REMOTE ] JBS: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T19:01:06.774847] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-08. [2025-09-16T19:01:07.574919] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-03. [2025-09-16T19:01:08.374944] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-03. [2025-09-16T19:01:08.592924] [INFO ] [REMOTE ] JBS: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T19:01:08.695224] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-09-16T19:01:09.174901] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-04. [2025-09-16T19:01:09.701598] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-09-16T19:01:09.974868] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-04. [2025-09-16T19:01:09.976431] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-09-16T19:01:10.774900] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-09. [2025-09-16T19:01:11.575634] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-05. [2025-09-16T19:01:11.580089] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-09-16T19:01:12.207107] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-09-16T19:01:12.374913] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-06. [2025-09-16T19:01:12.461739] [INFO ] [REMOTE ] JBS: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-09-16T19:01:12.468018] [INFO ] [REMOTE ] JBS: Full fetch complete (24448 records). [2025-09-16T19:01:12.468106] [DEBUG] [VALIDATOR ] JBS: Validating primary data. [2025-09-16T19:01:12.653820] [WARN ] [VALIDATOR ] JBS: Validation failed - insufficient historical data. Have 64 unique days, need 150 days [2025-09-16T19:01:12.654270] [FATAL] [MAIN ] JBS: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:01:12.654378] [WARN ] [MAIN ] JBS: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:01:12.654450] [INFO ] [LOCAL ] JBS: Starting deletion of primary data. [2025-09-16T19:01:12.654527] [DEBUG] [LOCAL ] JBS: No primary data file found to delete. [2025-09-16T19:01:12.654590] [INFO ] [LOCAL ] JBS: Starting deletion of derivative data. [2025-09-16T19:01:12.654681] [DEBUG] [LOCAL ] JBS: No derivative data file found to delete. [2025-09-16T19:01:12.654783] [INFO ] [MAIN ] Update Progress: 1143/2369 symbols completed. JBS took 23.9s. Estimated remaining: 2h 9m 57s. Rate: 9.4 symbols/min [2025-09-16T19:01:12.824741] [INFO ] [MAIN ] JCI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:01:12.864597] [DEBUG] [LOCAL ] JCI: Loaded 39530 existing records from primary storage. [2025-09-16T19:01:12.864785] [INFO ] [MAIN ] JCI: Loaded 39530 existing records. [2025-09-16T19:01:13.042519] [WARN ] [MAIN ] JCI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:01:13.042783] [INFO ] [MAIN ] JCI: Wiping existing primary data before full refresh. [2025-09-16T19:01:13.042830] [INFO ] [LOCAL ] JCI: Starting deletion of primary data. [2025-09-16T19:01:13.043788] [DEBUG] [LOCAL ] JCI: Deleted primary data file. [2025-09-16T19:01:13.043853] [INFO ] [MAIN ] JCI: Starting full primary refresh. [2025-09-16T19:01:13.184155] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-09. [2025-09-16T19:01:13.249229] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-09-16T19:01:13.407307] [INFO ] [REMOTE ] JCI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:01:13.974908] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-02. [2025-09-16T19:01:14.774951] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-05. [2025-09-16T19:01:14.816852] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-09-16T19:01:14.821476] [INFO ] [REMOTE ] JBND: Full fetch complete (26445 records). [2025-09-16T19:01:14.821680] [DEBUG] [VALIDATOR ] JBND: Validating primary data. [2025-09-16T19:01:15.223503] [DEBUG] [VALIDATOR ] JBND: All expected trading days are present. [2025-09-16T19:01:15.223619] [DEBUG] [VALIDATOR ] JBND: Primary data validation passed. [2025-09-16T19:01:15.224334] [INFO ] [MAIN ] JBND: Fetched primary data is valid. [2025-09-16T19:01:15.224413] [DEBUG] [LOCAL ] JBND: Saving primary data. [2025-09-16T19:01:15.225407] [DEBUG] [LOCAL ] JBND: Trimmed 26445 records to 19070 (keeping 150 days of data) [2025-09-16T19:01:15.345525] [DEBUG] [LOCAL ] JBND: Primary data saved (19070 records). [2025-09-16T19:01:15.345705] [INFO ] [MAIN ] JBND: Full history saved. [2025-09-16T19:01:15.345825] [INFO ] [MAIN ] Update Progress: 1144/2369 symbols completed. JBND took 30.6s. Estimated remaining: 2h 9m 47s. Rate: 9.4 symbols/min [2025-09-16T19:01:15.520916] [INFO ] [MAIN ] JEF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:01:15.561076] [DEBUG] [LOCAL ] JEF: Loaded 36411 existing records from primary storage. [2025-09-16T19:01:15.561228] [INFO ] [MAIN ] JEF: Loaded 36411 existing records. [2025-09-16T19:01:15.593533] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-07. [2025-09-16T19:01:15.742352] [WARN ] [MAIN ] JEF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:01:15.742792] [INFO ] [MAIN ] JEF: Wiping existing primary data before full refresh. [2025-09-16T19:01:15.742870] [INFO ] [LOCAL ] JEF: Starting deletion of primary data. [2025-09-16T19:01:15.743918] [DEBUG] [LOCAL ] JEF: Deleted primary data file. [2025-09-16T19:01:15.743997] [INFO ] [MAIN ] JEF: Starting full primary refresh. [2025-09-16T19:01:16.150708] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-02 (2.18 seconds). [2025-09-16T19:01:16.182724] [INFO ] [REMOTE ] JEF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:01:16.195672] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-09-16T19:01:16.216725] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T19:01:16.374875] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-02. [2025-09-16T19:01:17.174899] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-03. [2025-09-16T19:01:17.950560] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T19:01:17.974842] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-03. [2025-09-16T19:01:18.588323] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-09-16T19:01:18.774892] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-08. [2025-09-16T19:01:19.387517] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-09-16T19:01:19.574898] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-04. [2025-09-16T19:01:19.895669] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-09-16T19:01:20.374908] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-04. [2025-09-16T19:01:21.174920] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-09. [2025-09-16T19:01:21.487969] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-09-16T19:01:21.766062] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-09-16T19:01:21.767816] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T19:01:21.779746] [INFO ] [REMOTE ] JCE: Full fetch complete (6662 records). [2025-09-16T19:01:21.780040] [DEBUG] [VALIDATOR ] JCE: Validating primary data. [2025-09-16T19:01:21.984610] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-06. [2025-09-16T19:01:22.151552] [DEBUG] [VALIDATOR ] JCE: All expected trading days are present. [2025-09-16T19:01:22.151629] [DEBUG] [VALIDATOR ] JCE: Primary data validation passed. [2025-09-16T19:01:22.151840] [INFO ] [MAIN ] JCE: Fetched primary data is valid. [2025-09-16T19:01:22.151882] [DEBUG] [LOCAL ] JCE: Saving primary data. [2025-09-16T19:01:22.152274] [DEBUG] [LOCAL ] JCE: Trimmed 6662 records to 4761 (keeping 150 days of data) [2025-09-16T19:01:22.184308] [DEBUG] [LOCAL ] JCE: Primary data saved (4761 records). [2025-09-16T19:01:22.184410] [INFO ] [MAIN ] JCE: Full history saved. [2025-09-16T19:01:22.184494] [INFO ] [MAIN ] Update Progress: 1145/2369 symbols completed. JCE took 18.4s. Estimated remaining: 2h 9m 41s. Rate: 9.4 symbols/min [2025-09-16T19:01:22.353873] [INFO ] [MAIN ] JELD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:01:22.385415] [DEBUG] [LOCAL ] JELD: Loaded 34097 existing records from primary storage. [2025-09-16T19:01:22.385549] [INFO ] [MAIN ] JELD: Loaded 34097 existing records. [2025-09-16T19:01:22.561692] [WARN ] [MAIN ] JELD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:01:22.561915] [INFO ] [MAIN ] JELD: Wiping existing primary data before full refresh. [2025-09-16T19:01:22.561962] [INFO ] [LOCAL ] JELD: Starting deletion of primary data. [2025-09-16T19:01:22.562667] [DEBUG] [LOCAL ] JELD: Deleted primary data file. [2025-09-16T19:01:22.562720] [INFO ] [MAIN ] JELD: Starting full primary refresh. [2025-09-16T19:01:22.779967] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-05. [2025-09-16T19:01:22.924465] [INFO ] [REMOTE ] JELD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:01:23.470586] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T19:01:23.574870] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-07. [2025-09-16T19:01:24.374935] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-02. [2025-09-16T19:01:24.460239] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T19:01:25.049355] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T19:01:25.174886] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-08. [2025-09-16T19:01:25.461031] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T19:01:25.974900] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-06. [2025-09-16T19:01:26.459376] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-09-16T19:01:26.774903] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-09. [2025-09-16T19:01:27.574906] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-05. [2025-09-16T19:01:27.731209] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T19:01:28.045154] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T19:01:28.049376] [INFO ] [REMOTE ] JBTM: Full fetch complete (35139 records). [2025-09-16T19:01:28.049491] [DEBUG] [VALIDATOR ] JBTM: Validating primary data. [2025-09-16T19:01:28.385794] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-07. [2025-09-16T19:01:28.432177] [DEBUG] [VALIDATOR ] JBTM: All expected trading days are present. [2025-09-16T19:01:28.432261] [DEBUG] [VALIDATOR ] JBTM: Primary data validation passed. [2025-09-16T19:01:28.432891] [INFO ] [MAIN ] JBTM: Fetched primary data is valid. [2025-09-16T19:01:28.432936] [DEBUG] [LOCAL ] JBTM: Saving primary data. [2025-09-16T19:01:28.433854] [DEBUG] [LOCAL ] JBTM: Trimmed 35139 records to 24015 (keeping 150 days of data) [2025-09-16T19:01:28.590214] [DEBUG] [LOCAL ] JBTM: Primary data saved (24015 records). [2025-09-16T19:01:28.590477] [INFO ] [MAIN ] JBTM: Full history saved. [2025-09-16T19:01:28.590563] [INFO ] [MAIN ] Update Progress: 1146/2369 symbols completed. JBTM took 32.1s. Estimated remaining: 2h 9m 35s. Rate: 9.4 symbols/min [2025-09-16T19:01:28.759842] [INFO ] [MAIN ] JEQ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:01:28.767373] [DEBUG] [LOCAL ] JEQ: Loaded 5801 existing records from primary storage. [2025-09-16T19:01:28.767483] [INFO ] [MAIN ] JEQ: Loaded 5801 existing records. [2025-09-16T19:01:28.943681] [WARN ] [MAIN ] JEQ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:01:28.943983] [INFO ] [MAIN ] JEQ: Wiping existing primary data before full refresh. [2025-09-16T19:01:28.944045] [INFO ] [LOCAL ] JEQ: Starting deletion of primary data. [2025-09-16T19:01:28.944661] [DEBUG] [LOCAL ] JEQ: Deleted primary data file. [2025-09-16T19:01:28.944719] [INFO ] [MAIN ] JEQ: Starting full primary refresh. [2025-09-16T19:01:29.193265] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-03. [2025-09-16T19:01:29.317586] [INFO ] [REMOTE ] JEQ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:01:29.379498] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-09-16T19:01:29.974896] [INFO ] [REMOTE ] JEQ: Fetching primary data for 2025-02. [2025-09-16T19:01:30.308201] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T19:01:30.572600] [INFO ] [REMOTE ] JEQ: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-09-16T19:01:30.774911] [INFO ] [REMOTE ] JEQ: Fetching primary data for 2025-03. [2025-09-16T19:01:30.849550] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-09-16T19:01:31.574978] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-08. [2025-09-16T19:01:31.634405] [INFO ] [REMOTE ] JEQ: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-09-16T19:01:32.374894] [INFO ] [REMOTE ] JEQ: Fetching primary data for 2025-04. [2025-09-16T19:01:33.164482] [INFO ] [REMOTE ] JEQ: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-09-16T19:01:33.174840] [INFO ] [REMOTE ] JEQ: Fetching primary data for 2025-05. [2025-09-16T19:01:33.403826] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-09-16T19:01:33.900834] [INFO ] [REMOTE ] JEQ: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T19:01:33.974871] [INFO ] [REMOTE ] JEQ: Fetching primary data for 2025-06. [2025-09-16T19:01:34.544240] [INFO ] [REMOTE ] JEQ: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-09-16T19:01:34.774899] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-09. [2025-09-16T19:01:35.574918] [INFO ] [REMOTE ] JEQ: Fetching primary data for 2025-07. [2025-09-16T19:01:36.238535] [INFO ] [REMOTE ] JEQ: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-09-16T19:01:36.374884] [INFO ] [REMOTE ] JEQ: Fetching primary data for 2025-08. [2025-09-16T19:01:36.484342] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-09-16T19:01:36.492932] [INFO ] [REMOTE ] JCI: Full fetch complete (58225 records). [2025-09-16T19:01:36.493078] [DEBUG] [VALIDATOR ] JCI: Validating primary data. [2025-09-16T19:01:36.869087] [DEBUG] [VALIDATOR ] JCI: All expected trading days are present. [2025-09-16T19:01:36.869167] [DEBUG] [VALIDATOR ] JCI: Primary data validation passed. [2025-09-16T19:01:36.869963] [INFO ] [MAIN ] JCI: Fetched primary data is valid. [2025-09-16T19:01:36.870008] [DEBUG] [LOCAL ] JCI: Saving primary data. [2025-09-16T19:01:36.871189] [DEBUG] [LOCAL ] JCI: Trimmed 58225 records to 39917 (keeping 150 days of data) [2025-09-16T19:01:37.125313] [DEBUG] [LOCAL ] JCI: Primary data saved (39917 records). [2025-09-16T19:01:37.125476] [INFO ] [MAIN ] JCI: Full history saved. [2025-09-16T19:01:37.125586] [INFO ] [MAIN ] Update Progress: 1147/2369 symbols completed. JCI took 24.5s. Estimated remaining: 2h 9m 31s. Rate: 9.4 symbols/min [2025-09-16T19:01:37.180133] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-04. [2025-09-16T19:01:37.297900] [INFO ] [MAIN ] JFR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:01:37.318204] [DEBUG] [LOCAL ] JFR: Loaded 21424 existing records from primary storage. [2025-09-16T19:01:37.318364] [INFO ] [MAIN ] JFR: Loaded 21424 existing records. [2025-09-16T19:01:37.502026] [WARN ] [MAIN ] JFR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:01:37.502531] [INFO ] [REMOTE ] JEQ: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T19:01:37.506240] [INFO ] [MAIN ] JFR: Wiping existing primary data before full refresh. [2025-09-16T19:01:37.506521] [INFO ] [LOCAL ] JFR: Starting deletion of primary data. [2025-09-16T19:01:37.507023] [DEBUG] [LOCAL ] JFR: Deleted primary data file. [2025-09-16T19:01:37.507070] [INFO ] [MAIN ] JFR: Starting full primary refresh. [2025-09-16T19:01:37.876584] [INFO ] [REMOTE ] JFR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:01:37.974895] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-02. [2025-09-16T19:01:38.774894] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-06. [2025-09-16T19:01:38.960408] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-09-16T19:01:39.574899] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-05. [2025-09-16T19:01:39.612345] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T19:01:40.125089] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T19:01:40.374894] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-07. [2025-09-16T19:01:41.174937] [INFO ] [REMOTE ] JEQ: Fetching primary data for 2025-09. [2025-09-16T19:01:41.278603] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T19:01:41.974894] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-06. [2025-09-16T19:01:42.182732] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-09-16T19:01:42.197420] [INFO ] [REMOTE ] JEQ: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T19:01:42.199205] [INFO ] [REMOTE ] JEQ: Full fetch complete (9435 records). [2025-09-16T19:01:42.199278] [DEBUG] [VALIDATOR ] JEQ: Validating primary data. [2025-09-16T19:01:42.574684] [DEBUG] [VALIDATOR ] JEQ: All expected trading days are present. [2025-09-16T19:01:42.574763] [DEBUG] [VALIDATOR ] JEQ: Primary data validation passed. [2025-09-16T19:01:42.575018] [INFO ] [MAIN ] JEQ: Fetched primary data is valid. [2025-09-16T19:01:42.575068] [DEBUG] [LOCAL ] JEQ: Saving primary data. [2025-09-16T19:01:42.575497] [DEBUG] [LOCAL ] JEQ: Trimmed 9435 records to 5934 (keeping 150 days of data) [2025-09-16T19:01:42.610111] [DEBUG] [LOCAL ] JEQ: Primary data saved (5934 records). [2025-09-16T19:01:42.610289] [INFO ] [MAIN ] JEQ: Full history saved. [2025-09-16T19:01:42.610379] [INFO ] [MAIN ] Update Progress: 1148/2369 symbols completed. JEQ took 14.0s. Estimated remaining: 2h 9m 24s. Rate: 9.4 symbols/min [2025-09-16T19:01:42.776300] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-08. [2025-09-16T19:01:42.795171] [INFO ] [MAIN ] JGH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:01:42.807458] [DEBUG] [LOCAL ] JGH: Loaded 8029 existing records from primary storage. [2025-09-16T19:01:42.807720] [INFO ] [MAIN ] JGH: Loaded 8029 existing records. [2025-09-16T19:01:42.996429] [WARN ] [MAIN ] JGH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:01:42.996745] [INFO ] [MAIN ] JGH: Wiping existing primary data before full refresh. [2025-09-16T19:01:42.996811] [INFO ] [LOCAL ] JGH: Starting deletion of primary data. [2025-09-16T19:01:42.997130] [DEBUG] [LOCAL ] JGH: Deleted primary data file. [2025-09-16T19:01:42.997196] [INFO ] [MAIN ] JGH: Starting full primary refresh. [2025-09-16T19:01:43.371526] [INFO ] [REMOTE ] JGH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:01:43.574912] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-02. [2025-09-16T19:01:43.621855] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T19:01:43.972374] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-09-16T19:01:44.371898] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-09-16T19:01:44.381935] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-03. [2025-09-16T19:01:45.146587] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-09-16T19:01:45.174847] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-04. [2025-09-16T19:01:45.974885] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-09. [2025-09-16T19:01:46.038085] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-09-16T19:01:46.774887] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-05. [2025-09-16T19:01:47.514349] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T19:01:47.520215] [INFO ] [REMOTE ] JEF: Full fetch complete (54480 records). [2025-09-16T19:01:47.520336] [DEBUG] [VALIDATOR ] JEF: Validating primary data. [2025-09-16T19:01:47.584544] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-07. [2025-09-16T19:01:47.634808] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T19:01:47.933184] [DEBUG] [VALIDATOR ] JEF: All expected trading days are present. [2025-09-16T19:01:47.933279] [DEBUG] [VALIDATOR ] JEF: Primary data validation passed. [2025-09-16T19:01:47.934600] [INFO ] [MAIN ] JEF: Fetched primary data is valid. [2025-09-16T19:01:47.934679] [DEBUG] [LOCAL ] JEF: Saving primary data. [2025-09-16T19:01:47.936208] [DEBUG] [LOCAL ] JEF: Trimmed 54480 records to 36773 (keeping 150 days of data) [2025-09-16T19:01:48.170714] [DEBUG] [LOCAL ] JEF: Primary data saved (36773 records). [2025-09-16T19:01:48.170863] [INFO ] [MAIN ] JEF: Full history saved. [2025-09-16T19:01:48.171509] [INFO ] [MAIN ] Update Progress: 1149/2369 symbols completed. JEF took 32.8s. Estimated remaining: 2h 9m 16s. Rate: 9.4 symbols/min [2025-09-16T19:01:48.348270] [INFO ] [MAIN ] JHG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:01:48.374900] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-06. [2025-09-16T19:01:48.388805] [DEBUG] [LOCAL ] JHG: Loaded 35163 existing records from primary storage. [2025-09-16T19:01:48.388932] [INFO ] [MAIN ] JHG: Loaded 35163 existing records. [2025-09-16T19:01:48.577772] [WARN ] [MAIN ] JHG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:01:48.578050] [INFO ] [MAIN ] JHG: Wiping existing primary data before full refresh. [2025-09-16T19:01:48.578099] [INFO ] [LOCAL ] JHG: Starting deletion of primary data. [2025-09-16T19:01:48.579159] [DEBUG] [LOCAL ] JHG: Deleted primary data file. [2025-09-16T19:01:48.579214] [INFO ] [MAIN ] JHG: Starting full primary refresh. [2025-09-16T19:01:49.074806] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-09-16T19:01:49.074057] [INFO ] [REMOTE ] JHG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:01:49.191042] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-07. [2025-09-16T19:01:49.222347] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T19:01:49.974958] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-03. [2025-09-16T19:01:50.111623] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-09-16T19:01:50.774944] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-08. [2025-09-16T19:01:51.122956] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T19:01:51.557276] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-09-16T19:01:51.574863] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-09. [2025-09-16T19:01:52.374946] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-02. [2025-09-16T19:01:52.581823] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T19:01:52.584014] [INFO ] [REMOTE ] JGH: Full fetch complete (11439 records). [2025-09-16T19:01:52.584084] [DEBUG] [VALIDATOR ] JGH: Validating primary data. [2025-09-16T19:01:52.968224] [DEBUG] [VALIDATOR ] JGH: All expected trading days are present. [2025-09-16T19:01:52.968319] [DEBUG] [VALIDATOR ] JGH: Primary data validation passed. [2025-09-16T19:01:52.968802] [INFO ] [MAIN ] JGH: Fetched primary data is valid. [2025-09-16T19:01:52.968847] [DEBUG] [LOCAL ] JGH: Saving primary data. [2025-09-16T19:01:52.969446] [DEBUG] [LOCAL ] JGH: Trimmed 11439 records to 8121 (keeping 150 days of data) [2025-09-16T19:01:53.023176] [DEBUG] [LOCAL ] JGH: Primary data saved (8121 records). [2025-09-16T19:01:53.023339] [INFO ] [MAIN ] JGH: Full history saved. [2025-09-16T19:01:53.023459] [INFO ] [MAIN ] Update Progress: 1150/2369 symbols completed. JGH took 10.4s. Estimated remaining: 2h 9m 9s. Rate: 9.4 symbols/min [2025-09-16T19:01:53.182126] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-04. [2025-09-16T19:01:53.208361] [INFO ] [MAIN ] JHI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:01:53.213952] [DEBUG] [LOCAL ] JHI: Loaded 2473 existing records from primary storage. [2025-09-16T19:01:53.214237] [INFO ] [MAIN ] JHI: Loaded 2473 existing records. [2025-09-16T19:01:53.397221] [WARN ] [MAIN ] JHI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:01:53.397444] [INFO ] [MAIN ] JHI: Wiping existing primary data before full refresh. [2025-09-16T19:01:53.397490] [INFO ] [LOCAL ] JHI: Starting deletion of primary data. [2025-09-16T19:01:53.397731] [DEBUG] [LOCAL ] JHI: Deleted primary data file. [2025-09-16T19:01:53.397780] [INFO ] [MAIN ] JHI: Starting full primary refresh. [2025-09-16T19:01:53.776818] [INFO ] [REMOTE ] JHI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:01:53.923801] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T19:01:53.974892] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-03. [2025-09-16T19:01:54.526710] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-09-16T19:01:54.774956] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-05. [2025-09-16T19:01:55.574920] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-02. [2025-09-16T19:01:55.643549] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T19:01:56.039058] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-09-16T19:01:56.083343] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T19:01:56.374910] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-06. [2025-09-16T19:01:57.174881] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-04. [2025-09-16T19:01:57.668112] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T19:01:57.974936] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-07. [2025-09-16T19:01:58.774902] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-08. [2025-09-16T19:01:58.934495] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T19:01:58.988843] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-09-16T19:01:59.574916] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-08. [2025-09-16T19:02:00.374917] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-03. [2025-09-16T19:02:00.410117] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-09-16T19:02:01.049613] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-09-16T19:02:01.071242] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T19:02:01.174914] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-04. [2025-09-16T19:02:01.873579] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-09-16T19:02:01.974910] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-05. [2025-09-16T19:02:02.774938] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-09. [2025-09-16T19:02:03.574902] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-09. [2025-09-16T19:02:03.672715] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T19:02:04.105363] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-09-16T19:02:04.110984] [INFO ] [REMOTE ] JFR: Full fetch complete (34558 records). [2025-09-16T19:02:04.111126] [DEBUG] [VALIDATOR ] JFR: Validating primary data. [2025-09-16T19:02:04.380061] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-06. [2025-09-16T19:02:04.502636] [DEBUG] [VALIDATOR ] JFR: All expected trading days are present. [2025-09-16T19:02:04.502726] [DEBUG] [VALIDATOR ] JFR: Primary data validation passed. [2025-09-16T19:02:04.503319] [INFO ] [MAIN ] JFR: Fetched primary data is valid. [2025-09-16T19:02:04.503360] [DEBUG] [LOCAL ] JFR: Saving primary data. [2025-09-16T19:02:04.504133] [DEBUG] [LOCAL ] JFR: Trimmed 34558 records to 21662 (keeping 150 days of data) [2025-09-16T19:02:04.632931] [DEBUG] [LOCAL ] JFR: Primary data saved (21662 records). [2025-09-16T19:02:04.633061] [INFO ] [MAIN ] JFR: Full history saved. [2025-09-16T19:02:04.633155] [INFO ] [MAIN ] Update Progress: 1151/2369 symbols completed. JFR took 27.5s. Estimated remaining: 2h 9m 8s. Rate: 9.4 symbols/min [2025-09-16T19:02:04.812859] [INFO ] [MAIN ] JHS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:04.816726] [DEBUG] [LOCAL ] JHS: Loaded 2257 existing records from primary storage. [2025-09-16T19:02:04.816804] [INFO ] [MAIN ] JHS: Loaded 2257 existing records. [2025-09-16T19:02:05.004793] [WARN ] [MAIN ] JHS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:02:05.005019] [INFO ] [MAIN ] JHS: Wiping existing primary data before full refresh. [2025-09-16T19:02:05.005097] [INFO ] [LOCAL ] JHS: Starting deletion of primary data. [2025-09-16T19:02:05.005325] [DEBUG] [LOCAL ] JHS: Deleted primary data file. [2025-09-16T19:02:05.005368] [INFO ] [MAIN ] JHS: Starting full primary refresh. [2025-09-16T19:02:05.179304] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-05. [2025-09-16T19:02:05.324371] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-09-16T19:02:05.375899] [INFO ] [REMOTE ] JELD: Full fetch complete (50033 records). [2025-09-16T19:02:05.399975] [DEBUG] [VALIDATOR ] JELD: Validating primary data. [2025-09-16T19:02:05.403207] [INFO ] [REMOTE ] JHS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:05.802225] [DEBUG] [VALIDATOR ] JELD: All expected trading days are present. [2025-09-16T19:02:05.802309] [DEBUG] [VALIDATOR ] JELD: Primary data validation passed. [2025-09-16T19:02:05.803101] [INFO ] [MAIN ] JELD: Fetched primary data is valid. [2025-09-16T19:02:05.803148] [DEBUG] [LOCAL ] JELD: Saving primary data. [2025-09-16T19:02:05.804229] [DEBUG] [LOCAL ] JELD: Trimmed 50033 records to 34406 (keeping 150 days of data) [2025-09-16T19:02:05.984899] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-02. [2025-09-16T19:02:05.990402] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-09-16T19:02:06.068904] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-09-16T19:02:06.095135] [DEBUG] [LOCAL ] JELD: Primary data saved (34406 records). [2025-09-16T19:02:06.095292] [INFO ] [MAIN ] JELD: Full history saved. [2025-09-16T19:02:06.095386] [INFO ] [MAIN ] Update Progress: 1152/2369 symbols completed. JELD took 43.9s. Estimated remaining: 2h 8m 56s. Rate: 9.4 symbols/min [2025-09-16T19:02:06.266236] [INFO ] [MAIN ] JHX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:06.304761] [DEBUG] [LOCAL ] JHX: Loaded 39218 existing records from primary storage. [2025-09-16T19:02:06.304916] [INFO ] [MAIN ] JHX: Loaded 39218 existing records. [2025-09-16T19:02:06.486597] [WARN ] [MAIN ] JHX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:02:06.486864] [INFO ] [MAIN ] JHX: Wiping existing primary data before full refresh. [2025-09-16T19:02:06.486911] [INFO ] [LOCAL ] JHX: Starting deletion of primary data. [2025-09-16T19:02:06.487699] [DEBUG] [LOCAL ] JHX: Deleted primary data file. [2025-09-16T19:02:06.487749] [INFO ] [MAIN ] JHX: Starting full primary refresh. [2025-09-16T19:02:06.746914] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T19:02:06.779026] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-06. [2025-09-16T19:02:06.859025] [INFO ] [REMOTE ] JHX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:07.371383] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T19:02:07.574923] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-07. [2025-09-16T19:02:08.374937] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-03. [2025-09-16T19:02:08.828171] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-09-16T19:02:09.174908] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-02. [2025-09-16T19:02:09.278233] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T19:02:09.974918] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-08. [2025-09-16T19:02:10.001159] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-09-16T19:02:10.774943] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-03. [2025-09-16T19:02:11.580514] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-04. [2025-09-16T19:02:11.598959] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-09-16T19:02:12.052272] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-09-16T19:02:12.066403] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-09-16T19:02:12.374966] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-05. [2025-09-16T19:02:12.821750] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-09-16T19:02:13.174920] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-06. [2025-09-16T19:02:13.754209] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-09-16T19:02:13.974922] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-04. [2025-09-16T19:02:14.774955] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-09. [2025-09-16T19:02:15.583615] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-07. [2025-09-16T19:02:15.611732] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T19:02:16.156994] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-09-16T19:02:16.317872] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T19:02:16.323606] [INFO ] [REMOTE ] JHG: Full fetch complete (52110 records). [2025-09-16T19:02:16.323782] [DEBUG] [VALIDATOR ] JHG: Validating primary data. [2025-09-16T19:02:16.385061] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-05. [2025-09-16T19:02:16.737725] [DEBUG] [VALIDATOR ] JHG: All expected trading days are present. [2025-09-16T19:02:16.737823] [DEBUG] [VALIDATOR ] JHG: Primary data validation passed. [2025-09-16T19:02:16.738708] [INFO ] [MAIN ] JHG: Fetched primary data is valid. [2025-09-16T19:02:16.738786] [DEBUG] [LOCAL ] JHG: Saving primary data. [2025-09-16T19:02:16.741223] [DEBUG] [LOCAL ] JHG: Trimmed 52110 records to 35502 (keeping 150 days of data) [2025-09-16T19:02:16.963611] [DEBUG] [LOCAL ] JHG: Primary data saved (35502 records). [2025-09-16T19:02:16.963788] [INFO ] [MAIN ] JHG: Full history saved. [2025-09-16T19:02:16.964480] [INFO ] [MAIN ] Update Progress: 1153/2369 symbols completed. JHG took 28.8s. Estimated remaining: 2h 8m 55s. Rate: 9.4 symbols/min [2025-09-16T19:02:17.144351] [INFO ] [MAIN ] JILL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:17.164981] [DEBUG] [LOCAL ] JILL: Loaded 11480 existing records from primary storage. [2025-09-16T19:02:17.165129] [INFO ] [MAIN ] JILL: Loaded 11480 existing records. [2025-09-16T19:02:17.180054] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-07. [2025-09-16T19:02:17.353452] [WARN ] [MAIN ] JILL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:02:17.353758] [INFO ] [MAIN ] JILL: Wiping existing primary data before full refresh. [2025-09-16T19:02:17.353818] [INFO ] [LOCAL ] JILL: Starting deletion of primary data. [2025-09-16T19:02:17.354228] [DEBUG] [LOCAL ] JILL: Deleted primary data file. [2025-09-16T19:02:17.354292] [INFO ] [MAIN ] JILL: Starting full primary refresh. [2025-09-16T19:02:17.716415] [INFO ] [REMOTE ] JILL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:17.720176] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-09-16T19:02:17.974908] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-08. [2025-09-16T19:02:18.412255] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-09-16T19:02:18.420002] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-09-16T19:02:18.774945] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-02. [2025-09-16T19:02:19.537762] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T19:02:19.574953] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-03. [2025-09-16T19:02:20.374917] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-09. [2025-09-16T19:02:20.580705] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-09-16T19:02:21.023808] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-09-16T19:02:21.025383] [INFO ] [REMOTE ] JHS: Full fetch complete (3241 records). [2025-09-16T19:02:21.025477] [DEBUG] [VALIDATOR ] JHS: Validating primary data. [2025-09-16T19:02:21.179118] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-04. [2025-09-16T19:02:21.406796] [DEBUG] [VALIDATOR ] JHS: All expected trading days are present. [2025-09-16T19:02:21.406901] [DEBUG] [VALIDATOR ] JHS: Primary data validation passed. [2025-09-16T19:02:21.407053] [INFO ] [MAIN ] JHS: Fetched primary data is valid. [2025-09-16T19:02:21.407100] [DEBUG] [LOCAL ] JHS: Saving primary data. [2025-09-16T19:02:21.407572] [DEBUG] [LOCAL ] JHS: Trimmed 3241 records to 2298 (keeping 150 days of data) [2025-09-16T19:02:21.423399] [DEBUG] [LOCAL ] JHS: Primary data saved (2298 records). [2025-09-16T19:02:21.423510] [INFO ] [MAIN ] JHS: Full history saved. [2025-09-16T19:02:21.423665] [INFO ] [MAIN ] Update Progress: 1154/2369 symbols completed. JHS took 16.8s. Estimated remaining: 2h 8m 46s. Rate: 9.4 symbols/min [2025-09-16T19:02:21.618017] [INFO ] [MAIN ] JKS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:21.653800] [DEBUG] [LOCAL ] JKS: Loaded 28310 existing records from primary storage. [2025-09-16T19:02:21.653978] [INFO ] [MAIN ] JKS: Loaded 28310 existing records. [2025-09-16T19:02:21.877366] [WARN ] [MAIN ] JKS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:02:21.877615] [INFO ] [MAIN ] JKS: Wiping existing primary data before full refresh. [2025-09-16T19:02:21.877683] [INFO ] [LOCAL ] JKS: Starting deletion of primary data. [2025-09-16T19:02:21.878379] [DEBUG] [LOCAL ] JKS: Deleted primary data file. [2025-09-16T19:02:21.878448] [INFO ] [MAIN ] JKS: Starting full primary refresh. [2025-09-16T19:02:21.981427] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-06. [2025-09-16T19:02:22.258143] [INFO ] [REMOTE ] JKS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:22.272035] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T19:02:22.774923] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-08. [2025-09-16T19:02:23.299083] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-09-16T19:02:23.580047] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-05. [2025-09-16T19:02:23.604193] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T19:02:24.374947] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-07. [2025-09-16T19:02:24.525603] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-09-16T19:02:25.174920] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-09. [2025-09-16T19:02:25.974918] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-02. [2025-09-16T19:02:26.020910] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-09-16T19:02:26.022539] [INFO ] [REMOTE ] JHI: Full fetch complete (4014 records). [2025-09-16T19:02:26.022740] [DEBUG] [VALIDATOR ] JHI: Validating primary data. [2025-09-16T19:02:26.068974] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T19:02:26.394167] [DEBUG] [VALIDATOR ] JHI: All expected trading days are present. [2025-09-16T19:02:26.394271] [DEBUG] [VALIDATOR ] JHI: Primary data validation passed. [2025-09-16T19:02:26.394449] [INFO ] [MAIN ] JHI: Fetched primary data is valid. [2025-09-16T19:02:26.394493] [DEBUG] [LOCAL ] JHI: Saving primary data. [2025-09-16T19:02:26.394880] [DEBUG] [LOCAL ] JHI: Trimmed 4014 records to 2505 (keeping 150 days of data) [2025-09-16T19:02:26.410636] [DEBUG] [LOCAL ] JHI: Primary data saved (2505 records). [2025-09-16T19:02:26.410725] [INFO ] [MAIN ] JHI: Full history saved. [2025-09-16T19:02:26.410824] [INFO ] [MAIN ] Update Progress: 1155/2369 symbols completed. JHI took 33.4s. Estimated remaining: 2h 8m 38s. Rate: 9.4 symbols/min [2025-09-16T19:02:26.580026] [INFO ] [MAIN ] JLL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:26.604453] [DEBUG] [LOCAL ] JLL: Loaded 23872 existing records from primary storage. [2025-09-16T19:02:26.604608] [INFO ] [MAIN ] JLL: Loaded 23872 existing records. [2025-09-16T19:02:26.779934] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-06. [2025-09-16T19:02:26.783240] [WARN ] [MAIN ] JLL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:02:26.784163] [INFO ] [MAIN ] JLL: Wiping existing primary data before full refresh. [2025-09-16T19:02:26.784211] [INFO ] [LOCAL ] JLL: Starting deletion of primary data. [2025-09-16T19:02:26.784761] [DEBUG] [LOCAL ] JLL: Deleted primary data file. [2025-09-16T19:02:26.784881] [INFO ] [MAIN ] JLL: Starting full primary refresh. [2025-09-16T19:02:27.153000] [INFO ] [REMOTE ] JLL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:27.574856] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-02. [2025-09-16T19:02:27.614460] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-09-16T19:02:27.914992] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T19:02:28.374904] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-08. [2025-09-16T19:02:28.940755] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T19:02:29.174920] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-03. [2025-09-16T19:02:29.974901] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-07. [2025-09-16T19:02:30.293766] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-09-16T19:02:30.377900] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T19:02:30.774956] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-03. [2025-09-16T19:02:30.906615] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-09-16T19:02:31.574918] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-09. [2025-09-16T19:02:32.126489] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T19:02:32.374962] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-08. [2025-09-16T19:02:33.174885] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-04. [2025-09-16T19:02:33.301152] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T19:02:33.577709] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-09-16T19:02:33.584327] [INFO ] [REMOTE ] JHX: Full fetch complete (49729 records). [2025-09-16T19:02:33.584462] [DEBUG] [VALIDATOR ] JHX: Validating primary data. [2025-09-16T19:02:33.980806] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-04. [2025-09-16T19:02:34.002564] [DEBUG] [VALIDATOR ] JHX: All expected trading days are present. [2025-09-16T19:02:34.002696] [DEBUG] [VALIDATOR ] JHX: Primary data validation passed. [2025-09-16T19:02:34.003622] [INFO ] [MAIN ] JHX: Fetched primary data is valid. [2025-09-16T19:02:34.003809] [DEBUG] [LOCAL ] JHX: Saving primary data. [2025-09-16T19:02:34.007362] [DEBUG] [LOCAL ] JHX: Trimmed 49729 records to 39604 (keeping 150 days of data) [2025-09-16T19:02:34.275787] [DEBUG] [LOCAL ] JHX: Primary data saved (39604 records). [2025-09-16T19:02:34.275957] [INFO ] [MAIN ] JHX: Full history saved. [2025-09-16T19:02:34.276059] [INFO ] [MAIN ] Update Progress: 1156/2369 symbols completed. JHX took 28.2s. Estimated remaining: 2h 8m 34s. Rate: 9.4 symbols/min [2025-09-16T19:02:34.456188] [INFO ] [MAIN ] JLS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:34.460951] [DEBUG] [LOCAL ] JLS: Loaded 2854 existing records from primary storage. [2025-09-16T19:02:34.461032] [INFO ] [MAIN ] JLS: Loaded 2854 existing records. [2025-09-16T19:02:34.647290] [WARN ] [MAIN ] JLS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:02:34.647569] [INFO ] [MAIN ] JLS: Wiping existing primary data before full refresh. [2025-09-16T19:02:34.647708] [INFO ] [LOCAL ] JLS: Starting deletion of primary data. [2025-09-16T19:02:34.648239] [DEBUG] [LOCAL ] JLS: Deleted primary data file. [2025-09-16T19:02:34.648304] [INFO ] [MAIN ] JLS: Starting full primary refresh. [2025-09-16T19:02:34.776174] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-09. [2025-09-16T19:02:34.943426] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T19:02:35.052707] [INFO ] [REMOTE ] JLS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:35.574901] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-02. [2025-09-16T19:02:35.852671] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-09-16T19:02:36.042948] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T19:02:36.045571] [INFO ] [REMOTE ] JILL: Full fetch complete (17792 records). [2025-09-16T19:02:36.045687] [DEBUG] [VALIDATOR ] JILL: Validating primary data. [2025-09-16T19:02:36.174629] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-09-16T19:02:36.380864] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-05. [2025-09-16T19:02:36.424053] [DEBUG] [VALIDATOR ] JILL: All expected trading days are present. [2025-09-16T19:02:36.424138] [DEBUG] [VALIDATOR ] JILL: Primary data validation passed. [2025-09-16T19:02:36.424568] [INFO ] [MAIN ] JILL: Fetched primary data is valid. [2025-09-16T19:02:36.424613] [DEBUG] [LOCAL ] JILL: Saving primary data. [2025-09-16T19:02:36.425462] [DEBUG] [LOCAL ] JILL: Trimmed 17792 records to 11627 (keeping 150 days of data) [2025-09-16T19:02:36.498079] [DEBUG] [LOCAL ] JILL: Primary data saved (11627 records). [2025-09-16T19:02:36.498237] [INFO ] [MAIN ] JILL: Full history saved. [2025-09-16T19:02:36.498327] [INFO ] [MAIN ] Update Progress: 1157/2369 symbols completed. JILL took 19.5s. Estimated remaining: 2h 8m 23s. Rate: 9.4 symbols/min [2025-09-16T19:02:37.174900] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-05. [2025-09-16T19:02:37.969895] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T19:02:37.977518] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-03. [2025-09-16T19:02:38.566156] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T19:02:38.583781] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-09-16T19:02:38.774914] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-06. [2025-09-16T19:02:39.574911] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-04. [2025-09-16T19:02:40.171873] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-09-16T19:02:40.212387] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T19:02:40.374903] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-05. [2025-09-16T19:02:41.025740] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-09-16T19:02:41.174921] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-07. [2025-09-16T19:02:41.974923] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-06. [2025-09-16T19:02:42.542633] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T19:02:42.774915] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-06. [2025-09-16T19:02:43.283513] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T19:02:43.352727] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-09-16T19:02:43.574949] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-07. [2025-09-16T19:02:44.145337] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-09-16T19:02:44.374930] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-08. [2025-09-16T19:02:44.938287] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-09-16T19:02:45.174907] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-07. [2025-09-16T19:02:45.974903] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-09. [2025-09-16T19:02:46.258097] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-09-16T19:02:46.610365] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-09-16T19:02:46.611820] [INFO ] [REMOTE ] JLS: Full fetch complete (4419 records). [2025-09-16T19:02:46.611906] [DEBUG] [VALIDATOR ] JLS: Validating primary data. [2025-09-16T19:02:46.785859] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-08. [2025-09-16T19:02:47.034208] [DEBUG] [VALIDATOR ] JLS: All expected trading days are present. [2025-09-16T19:02:47.034347] [DEBUG] [VALIDATOR ] JLS: Primary data validation passed. [2025-09-16T19:02:47.034575] [INFO ] [MAIN ] JLS: Fetched primary data is valid. [2025-09-16T19:02:47.034640] [DEBUG] [LOCAL ] JLS: Saving primary data. [2025-09-16T19:02:47.035392] [DEBUG] [LOCAL ] JLS: Trimmed 4419 records to 2896 (keeping 150 days of data) [2025-09-16T19:02:47.055736] [DEBUG] [LOCAL ] JLS: Primary data saved (2896 records). [2025-09-16T19:02:47.055896] [INFO ] [MAIN ] JLS: Full history saved. [2025-09-16T19:02:47.056013] [INFO ] [MAIN ] Update Progress: 1158/2369 symbols completed. JLS took 12.8s. Estimated remaining: 2h 8m 21s. Rate: 9.4 symbols/min [2025-09-16T19:02:47.575020] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-08. [2025-09-16T19:02:48.551384] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T19:02:48.564225] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-09. [2025-09-16T19:02:48.913161] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T19:02:49.175110] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-09. [2025-09-16T19:02:49.712017] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T19:02:49.720210] [INFO ] [REMOTE ] JKS: Full fetch complete (43926 records). [2025-09-16T19:02:49.720405] [DEBUG] [VALIDATOR ] JKS: Validating primary data. [2025-09-16T19:02:50.118040] [DEBUG] [VALIDATOR ] JKS: All expected trading days are present. [2025-09-16T19:02:50.118128] [DEBUG] [VALIDATOR ] JKS: Primary data validation passed. [2025-09-16T19:02:50.119038] [INFO ] [MAIN ] JKS: Fetched primary data is valid. [2025-09-16T19:02:50.119085] [DEBUG] [LOCAL ] JKS: Saving primary data. [2025-09-16T19:02:50.120163] [DEBUG] [LOCAL ] JKS: Trimmed 43926 records to 28572 (keeping 150 days of data) [2025-09-16T19:02:50.312814] [DEBUG] [LOCAL ] JKS: Primary data saved (28572 records). [2025-09-16T19:02:50.312970] [INFO ] [MAIN ] JKS: Full history saved. [2025-09-16T19:02:50.313063] [INFO ] [MAIN ] Update Progress: 1159/2369 symbols completed. JKS took 28.9s. Estimated remaining: 2h 8m 11s. Rate: 9.4 symbols/min [2025-09-16T19:02:50.535904] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T19:02:50.540169] [INFO ] [REMOTE ] JLL: Full fetch complete (35290 records). [2025-09-16T19:02:50.540333] [DEBUG] [VALIDATOR ] JLL: Validating primary data. [2025-09-16T19:02:50.923414] [DEBUG] [VALIDATOR ] JLL: All expected trading days are present. [2025-09-16T19:02:50.923701] [DEBUG] [VALIDATOR ] JLL: Primary data validation passed. [2025-09-16T19:02:50.924554] [INFO ] [MAIN ] JLL: Fetched primary data is valid. [2025-09-16T19:02:50.924596] [DEBUG] [LOCAL ] JLL: Saving primary data. [2025-09-16T19:02:50.925565] [DEBUG] [LOCAL ] JLL: Trimmed 35290 records to 24106 (keeping 150 days of data) [2025-09-16T19:02:51.079375] [DEBUG] [LOCAL ] JLL: Primary data saved (24106 records). [2025-09-16T19:02:51.079529] [INFO ] [MAIN ] JLL: Full history saved. [2025-09-16T19:02:51.079625] [INFO ] [MAIN ] Update Progress: 1160/2369 symbols completed. JLL took 24.7s. Estimated remaining: 2h 7m 59s. Rate: 9.4 symbols/min [2025-09-16T19:02:51.681876] [INFO ] [MAIN ] JMIA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:51.680316] [INFO ] [MAIN ] JOBY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:51.766910] [INFO ] [MAIN ] JMM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:51.787819] [DEBUG] [LOCAL ] JMM: No existing data file found in primary storage. [2025-09-16T19:02:51.794408] [WARN ] [MAIN ] JMM: No existing data found, performing full refresh instead. [2025-09-16T19:02:51.801994] [INFO ] [MAIN ] JMM: Wiping existing primary data before full refresh. [2025-09-16T19:02:51.817095] [INFO ] [LOCAL ] JMM: Starting deletion of primary data. [2025-09-16T19:02:51.820236] [INFO ] [MAIN ] JNJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:02:51.820527] [DEBUG] [LOCAL ] JMM: No primary data file found to delete. [2025-09-16T19:02:51.820672] [INFO ] [MAIN ] JMM: Starting full primary refresh. [2025-09-16T19:02:51.987672] [DEBUG] [LOCAL ] JMIA: Loaded 37586 existing records from primary storage. [2025-09-16T19:02:52.018882] [INFO ] [MAIN ] JMIA: Loaded 37586 existing records. [2025-09-16T19:02:52.054966] [DEBUG] [LOCAL ] JOBY: Loaded 39771 existing records from primary storage. [2025-09-16T19:02:52.161840] [INFO ] [MAIN ] JOBY: Loaded 39771 existing records. [2025-09-16T19:02:52.372158] [DEBUG] [LOCAL ] JNJ: Loaded 39713 existing records from primary storage. [2025-09-16T19:02:52.439188] [INFO ] [MAIN ] JNJ: Loaded 39713 existing records. [2025-09-16T19:02:52.559789] [WARN ] [MAIN ] JMIA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:02:52.657156] [INFO ] [REMOTE ] JMM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:52.686731] [INFO ] [MAIN ] JMIA: Wiping existing primary data before full refresh. [2025-09-16T19:02:52.692154] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-02. [2025-09-16T19:02:52.692310] [INFO ] [LOCAL ] JMIA: Starting deletion of primary data. [2025-09-16T19:02:52.774601] [DEBUG] [LOCAL ] JMIA: Deleted primary data file. [2025-09-16T19:02:52.774780] [INFO ] [MAIN ] JMIA: Starting full primary refresh. [2025-09-16T19:02:52.809491] [WARN ] [MAIN ] JOBY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:02:52.894150] [INFO ] [MAIN ] JOBY: Wiping existing primary data before full refresh. [2025-09-16T19:02:52.906199] [INFO ] [LOCAL ] JOBY: Starting deletion of primary data. [2025-09-16T19:02:52.940617] [WARN ] [MAIN ] JNJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:02:52.969729] [DEBUG] [LOCAL ] JOBY: Deleted primary data file. [2025-09-16T19:02:52.970142] [INFO ] [MAIN ] JNJ: Wiping existing primary data before full refresh. [2025-09-16T19:02:52.970234] [INFO ] [MAIN ] JOBY: Starting full primary refresh. [2025-09-16T19:02:52.970334] [INFO ] [LOCAL ] JNJ: Starting deletion of primary data. [2025-09-16T19:02:53.027130] [DEBUG] [LOCAL ] JNJ: Deleted primary data file. [2025-09-16T19:02:53.089451] [INFO ] [MAIN ] JNJ: Starting full primary refresh. [2025-09-16T19:02:53.349299] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-09-16T19:02:53.380106] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-03. [2025-09-16T19:02:53.839924] [INFO ] [REMOTE ] JOBY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:53.884142] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-09-16T19:02:53.898884] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-04. [2025-09-16T19:02:53.884044] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-02. [2025-09-16T19:02:53.936195] [INFO ] [REMOTE ] JMIA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:53.942330] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-02. [2025-09-16T19:02:54.028667] [INFO ] [REMOTE ] JNJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:02:54.028988] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-02. [2025-09-16T19:02:54.393144] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-09-16T19:02:54.774962] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-05. [2025-09-16T19:02:55.239584] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-09-16T19:02:55.574979] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-06. [2025-09-16T19:02:55.849613] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-09-16T19:02:55.919691] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-09-16T19:02:56.012996] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-09-16T19:02:56.368567] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-02 (2.48 seconds). [2025-09-16T19:02:56.378384] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-07. [2025-09-16T19:02:56.897279] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-09-16T19:02:57.174956] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-08. [2025-09-16T19:02:57.610756] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-09-16T19:02:57.974965] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-03. [2025-09-16T19:02:58.774936] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-03. [2025-09-16T19:02:59.574960] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-03. [2025-09-16T19:03:00.161463] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-03 (2.19 seconds). [2025-09-16T19:03:00.335059] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T19:03:00.374880] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-04. [2025-09-16T19:03:01.174897] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-04. [2025-09-16T19:03:01.553756] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-09-16T19:03:01.974935] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-04. [2025-09-16T19:03:02.405469] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-09-16T19:03:02.774927] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-05. [2025-09-16T19:03:03.013720] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T19:03:03.574929] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-09. [2025-09-16T19:03:04.084152] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-04 (2.11 seconds). [2025-09-16T19:03:04.106354] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-09-16T19:03:04.107334] [INFO ] [REMOTE ] JMM: Full fetch complete (1968 records). [2025-09-16T19:03:04.107416] [DEBUG] [VALIDATOR ] JMM: Validating primary data. [2025-09-16T19:03:04.307883] [WARN ] [VALIDATOR ] JMM: Validation failed - insufficient historical data. Have 146 unique days, need 150 days [2025-09-16T19:03:04.308259] [FATAL] [MAIN ] JMM: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:03:04.318131] [WARN ] [MAIN ] JMM: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:03:04.331405] [INFO ] [LOCAL ] JMM: Starting deletion of primary data. [2025-09-16T19:03:04.334233] [DEBUG] [LOCAL ] JMM: No primary data file found to delete. [2025-09-16T19:03:04.334289] [INFO ] [LOCAL ] JMM: Starting deletion of derivative data. [2025-09-16T19:03:04.334887] [DEBUG] [LOCAL ] JMM: No derivative data file found to delete. [2025-09-16T19:03:04.335002] [INFO ] [MAIN ] Update Progress: 1161/2369 symbols completed. JMM took 13.3s. Estimated remaining: 2h 8m 0s. Rate: 9.4 symbols/min [2025-09-16T19:03:04.335901] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T19:03:04.384999] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-05. [2025-09-16T19:03:04.542388] [INFO ] [MAIN ] JOE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:03:04.559940] [DEBUG] [LOCAL ] JOE: Loaded 17669 existing records from primary storage. [2025-09-16T19:03:04.560067] [INFO ] [MAIN ] JOE: Loaded 17669 existing records. [2025-09-16T19:03:04.738932] [WARN ] [MAIN ] JOE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:03:04.739163] [INFO ] [MAIN ] JOE: Wiping existing primary data before full refresh. [2025-09-16T19:03:04.739209] [INFO ] [LOCAL ] JOE: Starting deletion of primary data. [2025-09-16T19:03:04.739690] [DEBUG] [LOCAL ] JOE: Deleted primary data file. [2025-09-16T19:03:04.739742] [INFO ] [MAIN ] JOE: Starting full primary refresh. [2025-09-16T19:03:05.097324] [INFO ] [REMOTE ] JOE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:03:05.174856] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-02. [2025-09-16T19:03:05.974916] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-05. [2025-09-16T19:03:06.190561] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T19:03:06.571836] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-05 (2.19 seconds). [2025-09-16T19:03:06.774864] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-06. [2025-09-16T19:03:07.574906] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-06. [2025-09-16T19:03:08.250944] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-05 (2.28 seconds). [2025-09-16T19:03:08.374938] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-06. [2025-09-16T19:03:08.570147] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-09-16T19:03:09.174958] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-07. [2025-09-16T19:03:09.324958] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-09-16T19:03:09.974917] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-07. [2025-09-16T19:03:10.774924] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-03. [2025-09-16T19:03:11.003938] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-06 (2.63 seconds). [2025-09-16T19:03:11.312994] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-09-16T19:03:11.574883] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-08. [2025-09-16T19:03:11.720586] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-09-16T19:03:11.881168] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T19:03:12.375366] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-04. [2025-09-16T19:03:13.174926] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-07. [2025-09-16T19:03:13.604074] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T19:03:13.760574] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-08 (2.19 seconds). [2025-09-16T19:03:13.974945] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-09. [2025-09-16T19:03:14.774949] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-08. [2025-09-16T19:03:15.532718] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-07 (2.36 seconds). [2025-09-16T19:03:15.574856] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-08. [2025-09-16T19:03:16.149909] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-09-16T19:03:16.158019] [INFO ] [REMOTE ] JNJ: Full fetch complete (58433 records). [2025-09-16T19:03:16.158204] [DEBUG] [VALIDATOR ] JNJ: Validating primary data. [2025-09-16T19:03:16.380808] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-05. [2025-09-16T19:03:16.556498] [DEBUG] [VALIDATOR ] JNJ: All expected trading days are present. [2025-09-16T19:03:16.556625] [DEBUG] [VALIDATOR ] JNJ: Primary data validation passed. [2025-09-16T19:03:16.557464] [INFO ] [MAIN ] JNJ: Fetched primary data is valid. [2025-09-16T19:03:16.557509] [DEBUG] [LOCAL ] JNJ: Saving primary data. [2025-09-16T19:03:16.558908] [DEBUG] [LOCAL ] JNJ: Trimmed 58433 records to 40103 (keeping 150 days of data) [2025-09-16T19:03:16.686949] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-09-16T19:03:16.860544] [DEBUG] [LOCAL ] JNJ: Primary data saved (40103 records). [2025-09-16T19:03:16.860696] [INFO ] [MAIN ] JNJ: Full history saved. [2025-09-16T19:03:16.860777] [INFO ] [MAIN ] Update Progress: 1162/2369 symbols completed. JNJ took 25.8s. Estimated remaining: 2h 8m 0s. Rate: 9.4 symbols/min [2025-09-16T19:03:17.042843] [INFO ] [MAIN ] JOF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:03:17.049488] [DEBUG] [LOCAL ] JOF: Loaded 5046 existing records from primary storage. [2025-09-16T19:03:17.049589] [INFO ] [MAIN ] JOF: Loaded 5046 existing records. [2025-09-16T19:03:17.180030] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-09. [2025-09-16T19:03:17.236874] [WARN ] [MAIN ] JOF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:03:17.237160] [INFO ] [MAIN ] JOF: Wiping existing primary data before full refresh. [2025-09-16T19:03:17.237219] [INFO ] [LOCAL ] JOF: Starting deletion of primary data. [2025-09-16T19:03:17.237533] [DEBUG] [LOCAL ] JOF: Deleted primary data file. [2025-09-16T19:03:17.237577] [INFO ] [MAIN ] JOF: Starting full primary refresh. [2025-09-16T19:03:17.591086] [INFO ] [REMOTE ] JOF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:03:17.607852] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-09-16T19:03:17.734778] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-08 (2.16 seconds). [2025-09-16T19:03:17.974935] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-09. [2025-09-16T19:03:18.774868] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-06. [2025-09-16T19:03:18.807215] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-09-16T19:03:18.813478] [INFO ] [REMOTE ] JMIA: Full fetch complete (54839 records). [2025-09-16T19:03:18.813570] [DEBUG] [VALIDATOR ] JMIA: Validating primary data. [2025-09-16T19:03:19.195438] [DEBUG] [VALIDATOR ] JMIA: All expected trading days are present. [2025-09-16T19:03:19.195671] [DEBUG] [VALIDATOR ] JMIA: Primary data validation passed. [2025-09-16T19:03:19.196467] [INFO ] [MAIN ] JMIA: Fetched primary data is valid. [2025-09-16T19:03:19.196512] [DEBUG] [LOCAL ] JMIA: Saving primary data. [2025-09-16T19:03:19.197736] [DEBUG] [LOCAL ] JMIA: Trimmed 54839 records to 37976 (keeping 150 days of data) [2025-09-16T19:03:19.418149] [DEBUG] [LOCAL ] JMIA: Primary data saved (37976 records). [2025-09-16T19:03:19.418290] [INFO ] [MAIN ] JMIA: Full history saved. [2025-09-16T19:03:19.418377] [INFO ] [MAIN ] Update Progress: 1163/2369 symbols completed. JMIA took 28.3s. Estimated remaining: 2h 7m 50s. Rate: 9.4 symbols/min [2025-09-16T19:03:19.587102] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-02. [2025-09-16T19:03:19.588377] [INFO ] [MAIN ] JPC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:03:19.614629] [DEBUG] [LOCAL ] JPC: Loaded 26736 existing records from primary storage. [2025-09-16T19:03:19.614775] [INFO ] [MAIN ] JPC: Loaded 26736 existing records. [2025-09-16T19:03:19.802756] [WARN ] [MAIN ] JPC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:03:19.803057] [INFO ] [MAIN ] JPC: Wiping existing primary data before full refresh. [2025-09-16T19:03:19.803104] [INFO ] [LOCAL ] JPC: Starting deletion of primary data. [2025-09-16T19:03:19.803960] [DEBUG] [LOCAL ] JPC: Deleted primary data file. [2025-09-16T19:03:19.804035] [INFO ] [MAIN ] JPC: Starting full primary refresh. [2025-09-16T19:03:19.983844] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-09-16T19:03:20.166421] [INFO ] [REMOTE ] JPC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:03:20.182056] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T19:03:20.204594] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-09 (2.23 seconds). [2025-09-16T19:03:20.211190] [INFO ] [REMOTE ] JOBY: Full fetch complete (58475 records). [2025-09-16T19:03:20.211344] [DEBUG] [VALIDATOR ] JOBY: Validating primary data. [2025-09-16T19:03:20.376738] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-03. [2025-09-16T19:03:20.606256] [DEBUG] [VALIDATOR ] JOBY: All expected trading days are present. [2025-09-16T19:03:20.606338] [DEBUG] [VALIDATOR ] JOBY: Primary data validation passed. [2025-09-16T19:03:20.607274] [INFO ] [MAIN ] JOBY: Fetched primary data is valid. [2025-09-16T19:03:20.607326] [DEBUG] [LOCAL ] JOBY: Saving primary data. [2025-09-16T19:03:20.608522] [DEBUG] [LOCAL ] JOBY: Trimmed 58475 records to 40161 (keeping 150 days of data) [2025-09-16T19:03:20.846410] [DEBUG] [LOCAL ] JOBY: Primary data saved (40161 records). [2025-09-16T19:03:20.846535] [INFO ] [MAIN ] JOBY: Full history saved. [2025-09-16T19:03:20.846615] [INFO ] [MAIN ] Update Progress: 1164/2369 symbols completed. JOBY took 29.8s. Estimated remaining: 2h 7m 38s. Rate: 9.4 symbols/min [2025-09-16T19:03:21.015942] [INFO ] [MAIN ] JPI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:03:21.021452] [DEBUG] [LOCAL ] JPI: Loaded 3826 existing records from primary storage. [2025-09-16T19:03:21.021553] [INFO ] [MAIN ] JPI: Loaded 3826 existing records. [2025-09-16T19:03:21.198745] [WARN ] [MAIN ] JPI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:03:21.198885] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-07. [2025-09-16T19:03:21.199516] [INFO ] [MAIN ] JPI: Wiping existing primary data before full refresh. [2025-09-16T19:03:21.200400] [INFO ] [LOCAL ] JPI: Starting deletion of primary data. [2025-09-16T19:03:21.209260] [DEBUG] [LOCAL ] JPI: Deleted primary data file. [2025-09-16T19:03:21.209726] [INFO ] [MAIN ] JPI: Starting full primary refresh. [2025-09-16T19:03:21.209621] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T19:03:21.568513] [INFO ] [REMOTE ] JPI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:03:21.974880] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-02. [2025-09-16T19:03:22.495070] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-09-16T19:03:22.774903] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-04. [2025-09-16T19:03:23.327696] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T19:03:23.449364] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T19:03:23.574850] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-03. [2025-09-16T19:03:24.374918] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-08. [2025-09-16T19:03:25.104092] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-09-16T19:03:25.174862] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-04. [2025-09-16T19:03:25.425140] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T19:03:25.974915] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-05. [2025-09-16T19:03:26.660447] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-09-16T19:03:26.689465] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T19:03:26.774945] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-09. [2025-09-16T19:03:27.574897] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-06. [2025-09-16T19:03:27.781243] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T19:03:27.784340] [INFO ] [REMOTE ] JOE: Full fetch complete (26932 records). [2025-09-16T19:03:27.784415] [DEBUG] [VALIDATOR ] JOE: Validating primary data. [2025-09-16T19:03:28.170071] [DEBUG] [VALIDATOR ] JOE: All expected trading days are present. [2025-09-16T19:03:28.170160] [DEBUG] [VALIDATOR ] JOE: Primary data validation passed. [2025-09-16T19:03:28.170806] [INFO ] [MAIN ] JOE: Fetched primary data is valid. [2025-09-16T19:03:28.170870] [DEBUG] [LOCAL ] JOE: Saving primary data. [2025-09-16T19:03:28.171625] [DEBUG] [LOCAL ] JOE: Trimmed 26932 records to 17782 (keeping 150 days of data) [2025-09-16T19:03:28.255716] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-09-16T19:03:28.288139] [DEBUG] [LOCAL ] JOE: Primary data saved (17782 records). [2025-09-16T19:03:28.288273] [INFO ] [MAIN ] JOE: Full history saved. [2025-09-16T19:03:28.288370] [INFO ] [MAIN ] Update Progress: 1165/2369 symbols completed. JOE took 24.0s. Estimated remaining: 2h 7m 33s. Rate: 9.4 symbols/min [2025-09-16T19:03:28.379988] [INFO ] [REMOTE ] JPI: Fetching primary data for 2025-02. [2025-09-16T19:03:28.462092] [INFO ] [MAIN ] JPM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:03:28.499666] [DEBUG] [LOCAL ] JPM: Loaded 39754 existing records from primary storage. [2025-09-16T19:03:28.499835] [INFO ] [MAIN ] JPM: Loaded 39754 existing records. [2025-09-16T19:03:28.673639] [WARN ] [MAIN ] JPM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:03:28.673912] [INFO ] [MAIN ] JPM: Wiping existing primary data before full refresh. [2025-09-16T19:03:28.673957] [INFO ] [LOCAL ] JPM: Starting deletion of primary data. [2025-09-16T19:03:28.674933] [DEBUG] [LOCAL ] JPM: Deleted primary data file. [2025-09-16T19:03:28.674981] [INFO ] [MAIN ] JPM: Starting full primary refresh. [2025-09-16T19:03:29.028731] [INFO ] [REMOTE ] JPM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:03:29.070280] [INFO ] [REMOTE ] JPI: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T19:03:29.174922] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-07. [2025-09-16T19:03:29.723333] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-09-16T19:03:29.974927] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-05. [2025-09-16T19:03:30.774893] [INFO ] [REMOTE ] JPI: Fetching primary data for 2025-03. [2025-09-16T19:03:31.353383] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-09-16T19:03:31.404201] [INFO ] [REMOTE ] JPI: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-09-16T19:03:31.574910] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-08. [2025-09-16T19:03:32.341931] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-09-16T19:03:32.374850] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-09. [2025-09-16T19:03:33.168666] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T19:03:33.170320] [INFO ] [REMOTE ] JOF: Full fetch complete (7542 records). [2025-09-16T19:03:33.170390] [DEBUG] [VALIDATOR ] JOF: Validating primary data. [2025-09-16T19:03:33.181538] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-06. [2025-09-16T19:03:33.545030] [DEBUG] [VALIDATOR ] JOF: All expected trading days are present. [2025-09-16T19:03:33.545161] [DEBUG] [VALIDATOR ] JOF: Primary data validation passed. [2025-09-16T19:03:33.545588] [INFO ] [MAIN ] JOF: Fetched primary data is valid. [2025-09-16T19:03:33.545703] [DEBUG] [LOCAL ] JOF: Saving primary data. [2025-09-16T19:03:33.546540] [DEBUG] [LOCAL ] JOF: Trimmed 7542 records to 5071 (keeping 150 days of data) [2025-09-16T19:03:33.587462] [DEBUG] [LOCAL ] JOF: Primary data saved (5071 records). [2025-09-16T19:03:33.587697] [INFO ] [MAIN ] JOF: Full history saved. [2025-09-16T19:03:33.587855] [INFO ] [MAIN ] Update Progress: 1166/2369 symbols completed. JOF took 16.7s. Estimated remaining: 2h 7m 26s. Rate: 9.4 symbols/min [2025-09-16T19:03:33.832510] [INFO ] [MAIN ] JQC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:03:33.865296] [DEBUG] [LOCAL ] JQC: Loaded 24051 existing records from primary storage. [2025-09-16T19:03:33.865468] [INFO ] [MAIN ] JQC: Loaded 24051 existing records. [2025-09-16T19:03:33.981025] [INFO ] [REMOTE ] JPI: Fetching primary data for 2025-04. [2025-09-16T19:03:34.163770] [WARN ] [MAIN ] JQC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:03:34.164942] [INFO ] [MAIN ] JQC: Wiping existing primary data before full refresh. [2025-09-16T19:03:34.165026] [INFO ] [LOCAL ] JQC: Starting deletion of primary data. [2025-09-16T19:03:34.165675] [DEBUG] [LOCAL ] JQC: Deleted primary data file. [2025-09-16T19:03:34.165735] [INFO ] [MAIN ] JQC: Starting full primary refresh. [2025-09-16T19:03:34.634472] [INFO ] [REMOTE ] JQC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:03:34.656562] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T19:03:34.727379] [INFO ] [REMOTE ] JPI: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-09-16T19:03:34.774882] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-07. [2025-09-16T19:03:35.574896] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-02. [2025-09-16T19:03:36.259715] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T19:03:36.374915] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-08. [2025-09-16T19:03:37.174912] [INFO ] [REMOTE ] JPI: Fetching primary data for 2025-05. [2025-09-16T19:03:37.714148] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-02 (2.14 seconds). [2025-09-16T19:03:37.777920] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-09-16T19:03:37.878201] [INFO ] [REMOTE ] JPI: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-09-16T19:03:37.974897] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-09. [2025-09-16T19:03:38.774899] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-02. [2025-09-16T19:03:39.377318] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T19:03:39.381956] [INFO ] [REMOTE ] JPC: Full fetch complete (40697 records). [2025-09-16T19:03:39.382048] [DEBUG] [VALIDATOR ] JPC: Validating primary data. [2025-09-16T19:03:39.580695] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-03. [2025-09-16T19:03:39.756432] [DEBUG] [VALIDATOR ] JPC: All expected trading days are present. [2025-09-16T19:03:39.756511] [DEBUG] [VALIDATOR ] JPC: Primary data validation passed. [2025-09-16T19:03:39.757182] [INFO ] [MAIN ] JPC: Fetched primary data is valid. [2025-09-16T19:03:39.757228] [DEBUG] [LOCAL ] JPC: Saving primary data. [2025-09-16T19:03:39.758111] [DEBUG] [LOCAL ] JPC: Trimmed 40697 records to 27000 (keeping 150 days of data) [2025-09-16T19:03:39.924098] [DEBUG] [LOCAL ] JPC: Primary data saved (27000 records). [2025-09-16T19:03:39.924277] [INFO ] [MAIN ] JPC: Full history saved. [2025-09-16T19:03:39.924377] [INFO ] [MAIN ] Update Progress: 1167/2369 symbols completed. JPC took 20.5s. Estimated remaining: 2h 7m 19s. Rate: 9.4 symbols/min [2025-09-16T19:03:40.103045] [INFO ] [MAIN ] JRI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:03:40.121040] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-09-16T19:03:40.135893] [DEBUG] [LOCAL ] JRI: Loaded 11077 existing records from primary storage. [2025-09-16T19:03:40.136028] [INFO ] [MAIN ] JRI: Loaded 11077 existing records. [2025-09-16T19:03:40.309316] [WARN ] [MAIN ] JRI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:03:40.309543] [INFO ] [MAIN ] JRI: Wiping existing primary data before full refresh. [2025-09-16T19:03:40.309588] [INFO ] [LOCAL ] JRI: Starting deletion of primary data. [2025-09-16T19:03:40.310031] [DEBUG] [LOCAL ] JRI: Deleted primary data file. [2025-09-16T19:03:40.310079] [INFO ] [MAIN ] JRI: Starting full primary refresh. [2025-09-16T19:03:40.383039] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-03. [2025-09-16T19:03:40.661327] [INFO ] [REMOTE ] JRI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:03:41.174898] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-02. [2025-09-16T19:03:41.818331] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T19:03:41.974909] [INFO ] [REMOTE ] JPI: Fetching primary data for 2025-06. [2025-09-16T19:03:42.119848] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-03 (2.54 seconds). [2025-09-16T19:03:42.147069] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-09-16T19:03:42.354800] [INFO ] [REMOTE ] JPI: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-09-16T19:03:42.774939] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-03. [2025-09-16T19:03:43.574911] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-04. [2025-09-16T19:03:43.773066] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-09-16T19:03:44.374906] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-04. [2025-09-16T19:03:44.978218] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-09-16T19:03:45.174911] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-05. [2025-09-16T19:03:45.284299] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-09-16T19:03:45.974903] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-05. [2025-09-16T19:03:46.555777] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-09-16T19:03:46.774936] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-04. [2025-09-16T19:03:46.976616] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T19:03:47.574917] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-06. [2025-09-16T19:03:48.374882] [INFO ] [REMOTE ] JPI: Fetching primary data for 2025-07. [2025-09-16T19:03:48.761161] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-09-16T19:03:48.786962] [INFO ] [REMOTE ] JPI: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-09-16T19:03:48.962463] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-09-16T19:03:49.174868] [INFO ] [REMOTE ] JPI: Fetching primary data for 2025-08. [2025-09-16T19:03:49.540836] [INFO ] [REMOTE ] JPI: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-09-16T19:03:49.974877] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-06. [2025-09-16T19:03:50.774934] [INFO ] [REMOTE ] JPI: Fetching primary data for 2025-09. [2025-09-16T19:03:50.875726] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-09-16T19:03:51.363280] [INFO ] [REMOTE ] JPI: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-09-16T19:03:51.364568] [INFO ] [REMOTE ] JPI: Full fetch complete (5872 records). [2025-09-16T19:03:51.364636] [DEBUG] [VALIDATOR ] JPI: Validating primary data. [2025-09-16T19:03:51.580083] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-07. [2025-09-16T19:03:51.717529] [DEBUG] [VALIDATOR ] JPI: All expected trading days are present. [2025-09-16T19:03:51.717600] [DEBUG] [VALIDATOR ] JPI: Primary data validation passed. [2025-09-16T19:03:51.717769] [INFO ] [MAIN ] JPI: Fetched primary data is valid. [2025-09-16T19:03:51.717810] [DEBUG] [LOCAL ] JPI: Saving primary data. [2025-09-16T19:03:51.718173] [DEBUG] [LOCAL ] JPI: Trimmed 5872 records to 3856 (keeping 150 days of data) [2025-09-16T19:03:51.741676] [DEBUG] [LOCAL ] JPI: Primary data saved (3856 records). [2025-09-16T19:03:51.741737] [INFO ] [MAIN ] JPI: Full history saved. [2025-09-16T19:03:51.741820] [INFO ] [MAIN ] Update Progress: 1168/2369 symbols completed. JPI took 30.9s. Estimated remaining: 2h 7m 19s. Rate: 9.4 symbols/min [2025-09-16T19:03:51.908059] [INFO ] [MAIN ] JRS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:03:51.917167] [DEBUG] [LOCAL ] JRS: Loaded 8279 existing records from primary storage. [2025-09-16T19:03:51.917268] [INFO ] [MAIN ] JRS: Loaded 8279 existing records. [2025-09-16T19:03:52.094330] [WARN ] [MAIN ] JRS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:03:52.094589] [INFO ] [MAIN ] JRS: Wiping existing primary data before full refresh. [2025-09-16T19:03:52.094664] [INFO ] [LOCAL ] JRS: Starting deletion of primary data. [2025-09-16T19:03:52.094993] [DEBUG] [LOCAL ] JRS: Deleted primary data file. [2025-09-16T19:03:52.095037] [INFO ] [MAIN ] JRS: Starting full primary refresh. [2025-09-16T19:03:52.380012] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-05. [2025-09-16T19:03:52.444549] [INFO ] [REMOTE ] JRS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:03:52.640893] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-09-16T19:03:53.174933] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-02. [2025-09-16T19:03:53.928126] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T19:03:53.974896] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-03. [2025-09-16T19:03:54.780467] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-07. [2025-09-16T19:03:54.786419] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-05 (2.41 seconds). [2025-09-16T19:03:54.805289] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T19:03:55.574920] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-08. [2025-09-16T19:03:56.125163] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-09-16T19:03:56.363398] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-09-16T19:03:56.374840] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-08. [2025-09-16T19:03:57.174966] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-09. [2025-09-16T19:03:57.753043] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-09-16T19:03:57.974879] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-04. [2025-09-16T19:03:58.171233] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T19:03:58.174395] [INFO ] [REMOTE ] JRI: Full fetch complete (16788 records). [2025-09-16T19:03:58.174466] [DEBUG] [VALIDATOR ] JRI: Validating primary data. [2025-09-16T19:03:58.531898] [DEBUG] [VALIDATOR ] JRI: All expected trading days are present. [2025-09-16T19:03:58.531977] [DEBUG] [VALIDATOR ] JRI: Primary data validation passed. [2025-09-16T19:03:58.532320] [INFO ] [MAIN ] JRI: Fetched primary data is valid. [2025-09-16T19:03:58.532360] [DEBUG] [LOCAL ] JRI: Saving primary data. [2025-09-16T19:03:58.532925] [DEBUG] [LOCAL ] JRI: Trimmed 16788 records to 11215 (keeping 150 days of data) [2025-09-16T19:03:58.597513] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-09-16T19:03:58.613962] [DEBUG] [LOCAL ] JRI: Primary data saved (11215 records). [2025-09-16T19:03:58.614097] [INFO ] [MAIN ] JRI: Full history saved. [2025-09-16T19:03:58.614183] [INFO ] [MAIN ] Update Progress: 1169/2369 symbols completed. JRI took 18.7s. Estimated remaining: 2h 7m 13s. Rate: 9.4 symbols/min [2025-09-16T19:03:58.782994] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-05. [2025-09-16T19:03:58.785906] [INFO ] [MAIN ] JXN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:03:58.821381] [DEBUG] [LOCAL ] JXN: Loaded 32954 existing records from primary storage. [2025-09-16T19:03:58.821520] [INFO ] [MAIN ] JXN: Loaded 32954 existing records. [2025-09-16T19:03:59.032557] [WARN ] [MAIN ] JXN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:03:59.032851] [INFO ] [MAIN ] JXN: Wiping existing primary data before full refresh. [2025-09-16T19:03:59.032913] [INFO ] [LOCAL ] JXN: Starting deletion of primary data. [2025-09-16T19:03:59.033766] [DEBUG] [LOCAL ] JXN: Deleted primary data file. [2025-09-16T19:03:59.033826] [INFO ] [MAIN ] JXN: Starting full primary refresh. [2025-09-16T19:03:59.383510] [INFO ] [REMOTE ] JXN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:03:59.574909] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-09. [2025-09-16T19:03:59.620273] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T19:04:00.374903] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-02. [2025-09-16T19:04:00.911897] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T19:04:00.921806] [INFO ] [REMOTE ] JQC: Full fetch complete (36613 records). [2025-09-16T19:04:00.922016] [DEBUG] [VALIDATOR ] JQC: Validating primary data. [2025-09-16T19:04:01.182707] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-06. [2025-09-16T19:04:01.295969] [DEBUG] [VALIDATOR ] JQC: All expected trading days are present. [2025-09-16T19:04:01.296049] [DEBUG] [VALIDATOR ] JQC: Primary data validation passed. [2025-09-16T19:04:01.296727] [INFO ] [MAIN ] JQC: Fetched primary data is valid. [2025-09-16T19:04:01.296782] [DEBUG] [LOCAL ] JQC: Saving primary data. [2025-09-16T19:04:01.297842] [DEBUG] [LOCAL ] JQC: Trimmed 36613 records to 24293 (keeping 150 days of data) [2025-09-16T19:04:01.446107] [DEBUG] [LOCAL ] JQC: Primary data saved (24293 records). [2025-09-16T19:04:01.446248] [INFO ] [MAIN ] JQC: Full history saved. [2025-09-16T19:04:01.446347] [INFO ] [MAIN ] Update Progress: 1170/2369 symbols completed. JQC took 27.9s. Estimated remaining: 2h 7m 3s. Rate: 9.4 symbols/min [2025-09-16T19:04:01.618546] [INFO ] [MAIN ] K: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:01.656738] [DEBUG] [LOCAL ] K: Loaded 37003 existing records from primary storage. [2025-09-16T19:04:01.656900] [INFO ] [MAIN ] K: Loaded 37003 existing records. [2025-09-16T19:04:01.851451] [WARN ] [MAIN ] K: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:01.851829] [INFO ] [MAIN ] K: Wiping existing primary data before full refresh. [2025-09-16T19:04:01.851992] [INFO ] [LOCAL ] K: Starting deletion of primary data. [2025-09-16T19:04:01.853003] [DEBUG] [LOCAL ] K: Deleted primary data file. [2025-09-16T19:04:01.853069] [INFO ] [MAIN ] K: Starting full primary refresh. [2025-09-16T19:04:01.978782] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-06. [2025-09-16T19:04:02.134100] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-09-16T19:04:02.262970] [INFO ] [REMOTE ] K: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:02.266265] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T19:04:02.774995] [INFO ] [REMOTE ] K: Fetching primary data for 2025-02. [2025-09-16T19:04:03.575016] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-03. [2025-09-16T19:04:04.325258] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-06 (2.35 seconds). [2025-09-16T19:04:04.375243] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-07. [2025-09-16T19:04:04.375160] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T19:04:05.174918] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-07. [2025-09-16T19:04:05.384151] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T19:04:05.429912] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-09-16T19:04:05.974913] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-04. [2025-09-16T19:04:06.774947] [INFO ] [REMOTE ] K: Fetching primary data for 2025-03. [2025-09-16T19:04:07.567568] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-09-16T19:04:07.577480] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-08. [2025-09-16T19:04:07.672266] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-07 (2.50 seconds). [2025-09-16T19:04:08.381617] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-08. [2025-09-16T19:04:08.403849] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T19:04:08.423384] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-09-16T19:04:09.174943] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-09. [2025-09-16T19:04:09.965921] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T19:04:09.967724] [INFO ] [REMOTE ] JRS: Full fetch complete (11955 records). [2025-09-16T19:04:09.967839] [DEBUG] [VALIDATOR ] JRS: Validating primary data. [2025-09-16T19:04:09.980791] [INFO ] [REMOTE ] K: Fetching primary data for 2025-04. [2025-09-16T19:04:10.334376] [DEBUG] [VALIDATOR ] JRS: All expected trading days are present. [2025-09-16T19:04:10.334471] [DEBUG] [VALIDATOR ] JRS: Primary data validation passed. [2025-09-16T19:04:10.334780] [INFO ] [MAIN ] JRS: Fetched primary data is valid. [2025-09-16T19:04:10.334838] [DEBUG] [LOCAL ] JRS: Saving primary data. [2025-09-16T19:04:10.335315] [DEBUG] [LOCAL ] JRS: Trimmed 11955 records to 8343 (keeping 150 days of data) [2025-09-16T19:04:10.385651] [DEBUG] [LOCAL ] JRS: Primary data saved (8343 records). [2025-09-16T19:04:10.385796] [INFO ] [MAIN ] JRS: Full history saved. [2025-09-16T19:04:10.385944] [INFO ] [MAIN ] Update Progress: 1171/2369 symbols completed. JRS took 18.6s. Estimated remaining: 2h 6m 59s. Rate: 9.4 symbols/min [2025-09-16T19:04:10.557862] [INFO ] [MAIN ] KAI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:10.569635] [DEBUG] [LOCAL ] KAI: Loaded 9742 existing records from primary storage. [2025-09-16T19:04:10.569779] [INFO ] [MAIN ] KAI: Loaded 9742 existing records. [2025-09-16T19:04:10.752857] [WARN ] [MAIN ] KAI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:10.753162] [INFO ] [MAIN ] KAI: Wiping existing primary data before full refresh. [2025-09-16T19:04:10.753215] [INFO ] [LOCAL ] KAI: Starting deletion of primary data. [2025-09-16T19:04:10.753630] [DEBUG] [LOCAL ] KAI: Deleted primary data file. [2025-09-16T19:04:10.753732] [INFO ] [MAIN ] KAI: Starting full primary refresh. [2025-09-16T19:04:10.776620] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-05. [2025-09-16T19:04:10.962872] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-08 (2.58 seconds). [2025-09-16T19:04:11.177960] [INFO ] [REMOTE ] KAI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:11.574933] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-02. [2025-09-16T19:04:11.794542] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T19:04:12.374914] [INFO ] [REMOTE ] K: Fetching primary data for 2025-05. [2025-09-16T19:04:12.449256] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-09-16T19:04:12.519762] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-09-16T19:04:13.174914] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-09. [2025-09-16T19:04:13.974880] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-03. [2025-09-16T19:04:14.053452] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T19:04:14.774913] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-06. [2025-09-16T19:04:14.989620] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-09-16T19:04:15.317790] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-09-16T19:04:15.324426] [INFO ] [REMOTE ] JPM: Full fetch complete (58474 records). [2025-09-16T19:04:15.324570] [DEBUG] [VALIDATOR ] JPM: Validating primary data. [2025-09-16T19:04:15.580008] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-04. [2025-09-16T19:04:15.705746] [DEBUG] [VALIDATOR ] JPM: All expected trading days are present. [2025-09-16T19:04:15.705827] [DEBUG] [VALIDATOR ] JPM: Primary data validation passed. [2025-09-16T19:04:15.706605] [INFO ] [MAIN ] JPM: Fetched primary data is valid. [2025-09-16T19:04:15.706665] [DEBUG] [LOCAL ] JPM: Saving primary data. [2025-09-16T19:04:15.708023] [DEBUG] [LOCAL ] JPM: Trimmed 58474 records to 40144 (keeping 150 days of data) [2025-09-16T19:04:15.972556] [DEBUG] [LOCAL ] JPM: Primary data saved (40144 records). [2025-09-16T19:04:15.972728] [INFO ] [MAIN ] JPM: Full history saved. [2025-09-16T19:04:15.972863] [INFO ] [MAIN ] Update Progress: 1172/2369 symbols completed. JPM took 47.7s. Estimated remaining: 2h 6m 52s. Rate: 9.4 symbols/min [2025-09-16T19:04:16.156052] [INFO ] [MAIN ] KAR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:16.188122] [DEBUG] [LOCAL ] KAR: Loaded 29474 existing records from primary storage. [2025-09-16T19:04:16.188268] [INFO ] [MAIN ] KAR: Loaded 29474 existing records. [2025-09-16T19:04:16.372749] [WARN ] [MAIN ] KAR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:16.373066] [INFO ] [MAIN ] KAR: Wiping existing primary data before full refresh. [2025-09-16T19:04:16.373127] [INFO ] [LOCAL ] KAR: Starting deletion of primary data. [2025-09-16T19:04:16.373743] [DEBUG] [LOCAL ] KAR: Deleted primary data file. [2025-09-16T19:04:16.373792] [INFO ] [MAIN ] KAR: Starting full primary refresh. [2025-09-16T19:04:16.379395] [INFO ] [REMOTE ] K: Fetching primary data for 2025-06. [2025-09-16T19:04:16.735161] [INFO ] [REMOTE ] KAR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:16.747636] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-09-16T19:04:16.779569] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-09-16T19:04:17.174908] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-07. [2025-09-16T19:04:17.974901] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-05. [2025-09-16T19:04:18.147500] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-09-16T19:04:18.623734] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T19:04:18.774912] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-08. [2025-09-16T19:04:18.916943] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-09-16T19:04:19.574913] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-02. [2025-09-16T19:04:20.381920] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-06. [2025-09-16T19:04:20.387319] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-09-16T19:04:21.138067] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T19:04:21.174871] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-03. [2025-09-16T19:04:21.327033] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-09-16T19:04:21.974926] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-09. [2025-09-16T19:04:22.492940] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-09-16T19:04:22.774901] [INFO ] [REMOTE ] K: Fetching primary data for 2025-07. [2025-09-16T19:04:23.245505] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-09-16T19:04:23.250897] [INFO ] [REMOTE ] JXN: Full fetch complete (50175 records). [2025-09-16T19:04:23.251042] [DEBUG] [VALIDATOR ] JXN: Validating primary data. [2025-09-16T19:04:23.580023] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-07. [2025-09-16T19:04:23.649482] [DEBUG] [VALIDATOR ] JXN: All expected trading days are present. [2025-09-16T19:04:23.649569] [DEBUG] [VALIDATOR ] JXN: Primary data validation passed. [2025-09-16T19:04:23.650386] [INFO ] [MAIN ] JXN: Fetched primary data is valid. [2025-09-16T19:04:23.650435] [DEBUG] [LOCAL ] JXN: Saving primary data. [2025-09-16T19:04:23.652515] [DEBUG] [LOCAL ] JXN: Trimmed 50175 records to 33286 (keeping 150 days of data) [2025-09-16T19:04:23.868144] [DEBUG] [LOCAL ] JXN: Primary data saved (33286 records). [2025-09-16T19:04:23.868303] [INFO ] [MAIN ] JXN: Full history saved. [2025-09-16T19:04:23.869511] [INFO ] [MAIN ] Update Progress: 1173/2369 symbols completed. JXN took 25.3s. Estimated remaining: 2h 6m 47s. Rate: 9.4 symbols/min [2025-09-16T19:04:24.040990] [INFO ] [MAIN ] KB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:24.064804] [DEBUG] [LOCAL ] KB: Loaded 17840 existing records from primary storage. [2025-09-16T19:04:24.064956] [INFO ] [MAIN ] KB: Loaded 17840 existing records. [2025-09-16T19:04:24.244191] [WARN ] [MAIN ] KB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:24.244493] [INFO ] [MAIN ] KB: Wiping existing primary data before full refresh. [2025-09-16T19:04:24.244541] [INFO ] [LOCAL ] KB: Starting deletion of primary data. [2025-09-16T19:04:24.245169] [DEBUG] [LOCAL ] KB: Deleted primary data file. [2025-09-16T19:04:24.245219] [INFO ] [MAIN ] KB: Starting full primary refresh. [2025-09-16T19:04:24.391690] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-04. [2025-09-16T19:04:24.458049] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-09-16T19:04:24.641115] [INFO ] [REMOTE ] KB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:24.681502] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T19:04:25.174953] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-02. [2025-09-16T19:04:25.963086] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-09-16T19:04:25.975109] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-05. [2025-09-16T19:04:26.013058] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T19:04:26.774987] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-08. [2025-09-16T19:04:27.528294] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-09-16T19:04:27.574853] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-06. [2025-09-16T19:04:27.637464] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T19:04:28.374906] [INFO ] [REMOTE ] K: Fetching primary data for 2025-08. [2025-09-16T19:04:28.956236] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T19:04:29.174946] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-09. [2025-09-16T19:04:29.974964] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-03. [2025-09-16T19:04:30.069011] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T19:04:30.070831] [INFO ] [REMOTE ] KAI: Full fetch complete (15399 records). [2025-09-16T19:04:30.070909] [DEBUG] [VALIDATOR ] KAI: Validating primary data. [2025-09-16T19:04:30.453432] [DEBUG] [VALIDATOR ] KAI: All expected trading days are present. [2025-09-16T19:04:30.453543] [DEBUG] [VALIDATOR ] KAI: Primary data validation passed. [2025-09-16T19:04:30.453988] [INFO ] [MAIN ] KAI: Fetched primary data is valid. [2025-09-16T19:04:30.454116] [DEBUG] [LOCAL ] KAI: Saving primary data. [2025-09-16T19:04:30.454597] [DEBUG] [LOCAL ] KAI: Trimmed 15399 records to 9846 (keeping 150 days of data) [2025-09-16T19:04:30.530268] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-08 (2.16 seconds). [2025-09-16T19:04:30.553903] [DEBUG] [LOCAL ] KAI: Primary data saved (9846 records). [2025-09-16T19:04:30.554033] [INFO ] [MAIN ] KAI: Full history saved. [2025-09-16T19:04:30.554113] [INFO ] [MAIN ] Update Progress: 1174/2369 symbols completed. KAI took 20.2s. Estimated remaining: 2h 6m 41s. Rate: 9.4 symbols/min [2025-09-16T19:04:30.722740] [INFO ] [MAIN ] KBDC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:30.738111] [DEBUG] [LOCAL ] KBDC: Loaded 14554 existing records from primary storage. [2025-09-16T19:04:30.738273] [INFO ] [MAIN ] KBDC: Loaded 14554 existing records. [2025-09-16T19:04:30.781406] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-07. [2025-09-16T19:04:30.922134] [WARN ] [MAIN ] KBDC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:30.922385] [INFO ] [MAIN ] KBDC: Wiping existing primary data before full refresh. [2025-09-16T19:04:30.922438] [INFO ] [LOCAL ] KBDC: Starting deletion of primary data. [2025-09-16T19:04:30.922882] [DEBUG] [LOCAL ] KBDC: Deleted primary data file. [2025-09-16T19:04:30.922930] [INFO ] [MAIN ] KBDC: Starting full primary refresh. [2025-09-16T19:04:31.277277] [INFO ] [REMOTE ] KBDC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:31.295041] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-09-16T19:04:31.574929] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-04. [2025-09-16T19:04:32.386044] [INFO ] [REMOTE ] K: Fetching primary data for 2025-09. [2025-09-16T19:04:32.392372] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-09-16T19:04:32.746879] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-09-16T19:04:33.174905] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-05. [2025-09-16T19:04:33.979873] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-02. [2025-09-16T19:04:33.991228] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T19:04:33.998489] [INFO ] [REMOTE ] K: Full fetch complete (53997 records). [2025-09-16T19:04:33.998659] [DEBUG] [VALIDATOR ] K: Validating primary data. [2025-09-16T19:04:34.397127] [DEBUG] [VALIDATOR ] K: All expected trading days are present. [2025-09-16T19:04:34.397325] [DEBUG] [VALIDATOR ] K: Primary data validation passed. [2025-09-16T19:04:34.398186] [INFO ] [MAIN ] K: Fetched primary data is valid. [2025-09-16T19:04:34.398254] [DEBUG] [LOCAL ] K: Saving primary data. [2025-09-16T19:04:34.399832] [DEBUG] [LOCAL ] K: Trimmed 53997 records to 37359 (keeping 150 days of data) [2025-09-16T19:04:34.515082] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T19:04:34.672783] [DEBUG] [LOCAL ] K: Primary data saved (37359 records). [2025-09-16T19:04:34.672972] [INFO ] [MAIN ] K: Full history saved. [2025-09-16T19:04:34.673091] [INFO ] [MAIN ] Update Progress: 1175/2369 symbols completed. K took 33.2s. Estimated remaining: 2h 6m 32s. Rate: 9.4 symbols/min [2025-09-16T19:04:34.780100] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-08. [2025-09-16T19:04:34.850390] [INFO ] [MAIN ] KBH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:34.887829] [DEBUG] [LOCAL ] KBH: Loaded 35669 existing records from primary storage. [2025-09-16T19:04:34.888022] [INFO ] [MAIN ] KBH: Loaded 35669 existing records. [2025-09-16T19:04:35.088245] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-09-16T19:04:35.088423] [WARN ] [MAIN ] KBH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:35.092965] [INFO ] [MAIN ] KBH: Wiping existing primary data before full refresh. [2025-09-16T19:04:35.093032] [INFO ] [LOCAL ] KBH: Starting deletion of primary data. [2025-09-16T19:04:35.093944] [DEBUG] [LOCAL ] KBH: Deleted primary data file. [2025-09-16T19:04:35.094007] [INFO ] [MAIN ] KBH: Starting full primary refresh. [2025-09-16T19:04:35.442961] [INFO ] [REMOTE ] KBH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:35.574898] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-02. [2025-09-16T19:04:36.281299] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T19:04:36.374920] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-03. [2025-09-16T19:04:37.174926] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-09. [2025-09-16T19:04:37.223072] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-09-16T19:04:37.451877] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T19:04:37.974941] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-04. [2025-09-16T19:04:38.330748] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-09-16T19:04:38.336592] [INFO ] [REMOTE ] KAR: Full fetch complete (43743 records). [2025-09-16T19:04:38.336802] [DEBUG] [VALIDATOR ] KAR: Validating primary data. [2025-09-16T19:04:38.742139] [DEBUG] [VALIDATOR ] KAR: All expected trading days are present. [2025-09-16T19:04:38.742237] [DEBUG] [VALIDATOR ] KAR: Primary data validation passed. [2025-09-16T19:04:38.743006] [INFO ] [MAIN ] KAR: Fetched primary data is valid. [2025-09-16T19:04:38.743069] [DEBUG] [LOCAL ] KAR: Saving primary data. [2025-09-16T19:04:38.744551] [DEBUG] [LOCAL ] KAR: Trimmed 43743 records to 29727 (keeping 150 days of data) [2025-09-16T19:04:38.800138] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-03. [2025-09-16T19:04:38.948037] [DEBUG] [LOCAL ] KAR: Primary data saved (29727 records). [2025-09-16T19:04:38.948199] [INFO ] [MAIN ] KAR: Full history saved. [2025-09-16T19:04:38.948295] [INFO ] [MAIN ] Update Progress: 1176/2369 symbols completed. KAR took 23.0s. Estimated remaining: 2h 6m 24s. Rate: 9.4 symbols/min [2025-09-16T19:04:39.121280] [INFO ] [MAIN ] KBR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:39.136219] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-09-16T19:04:39.172097] [DEBUG] [LOCAL ] KBR: Loaded 35623 existing records from primary storage. [2025-09-16T19:04:39.172244] [INFO ] [MAIN ] KBR: Loaded 35623 existing records. [2025-09-16T19:04:39.348607] [WARN ] [MAIN ] KBR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:39.348862] [INFO ] [MAIN ] KBR: Wiping existing primary data before full refresh. [2025-09-16T19:04:39.348909] [INFO ] [LOCAL ] KBR: Starting deletion of primary data. [2025-09-16T19:04:39.349676] [DEBUG] [LOCAL ] KBR: Deleted primary data file. [2025-09-16T19:04:39.349725] [INFO ] [MAIN ] KBR: Starting full primary refresh. [2025-09-16T19:04:39.580040] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-06. [2025-09-16T19:04:39.706308] [INFO ] [REMOTE ] KBR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:40.374938] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-02. [2025-09-16T19:04:40.536757] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T19:04:40.894369] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T19:04:41.174901] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-04. [2025-09-16T19:04:41.800081] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-09-16T19:04:41.974936] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-03. [2025-09-16T19:04:42.774898] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-05. [2025-09-16T19:04:42.940724] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-09-16T19:04:43.574880] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-05. [2025-09-16T19:04:43.693925] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T19:04:43.882322] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-09-16T19:04:44.374975] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-06. [2025-09-16T19:04:44.964734] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T19:04:44.985529] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-09-16T19:04:45.174859] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-06. [2025-09-16T19:04:45.974917] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-07. [2025-09-16T19:04:46.774902] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-07. [2025-09-16T19:04:46.876536] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T19:04:47.059070] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-09-16T19:04:47.574967] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-07. [2025-09-16T19:04:47.711780] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-09-16T19:04:48.374927] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-08. [2025-09-16T19:04:49.174935] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-04. [2025-09-16T19:04:49.347520] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-09-16T19:04:49.396960] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T19:04:49.974922] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-08. [2025-09-16T19:04:50.774924] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-09. [2025-09-16T19:04:50.962892] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T19:04:51.574905] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-08. [2025-09-16T19:04:51.633223] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T19:04:51.692346] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T19:04:51.696309] [INFO ] [REMOTE ] KBDC: Full fetch complete (22097 records). [2025-09-16T19:04:51.696389] [DEBUG] [VALIDATOR ] KBDC: Validating primary data. [2025-09-16T19:04:52.072700] [DEBUG] [VALIDATOR ] KBDC: All expected trading days are present. [2025-09-16T19:04:52.072814] [DEBUG] [VALIDATOR ] KBDC: Primary data validation passed. [2025-09-16T19:04:52.073266] [INFO ] [MAIN ] KBDC: Fetched primary data is valid. [2025-09-16T19:04:52.073306] [DEBUG] [LOCAL ] KBDC: Saving primary data. [2025-09-16T19:04:52.073901] [DEBUG] [LOCAL ] KBDC: Trimmed 22097 records to 14804 (keeping 150 days of data) [2025-09-16T19:04:52.166275] [DEBUG] [LOCAL ] KBDC: Primary data saved (14804 records). [2025-09-16T19:04:52.166435] [INFO ] [MAIN ] KBDC: Full history saved. [2025-09-16T19:04:52.166593] [INFO ] [MAIN ] Update Progress: 1177/2369 symbols completed. KBDC took 21.6s. Estimated remaining: 2h 6m 25s. Rate: 9.4 symbols/min [2025-09-16T19:04:52.340333] [INFO ] [MAIN ] KD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:52.378829] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-09. [2025-09-16T19:04:52.380564] [DEBUG] [LOCAL ] KD: Loaded 38505 existing records from primary storage. [2025-09-16T19:04:52.380733] [INFO ] [MAIN ] KD: Loaded 38505 existing records. [2025-09-16T19:04:52.561386] [WARN ] [MAIN ] KD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:52.561660] [INFO ] [MAIN ] KD: Wiping existing primary data before full refresh. [2025-09-16T19:04:52.561710] [INFO ] [LOCAL ] KD: Starting deletion of primary data. [2025-09-16T19:04:52.562496] [DEBUG] [LOCAL ] KD: Deleted primary data file. [2025-09-16T19:04:52.562542] [INFO ] [MAIN ] KD: Starting full primary refresh. [2025-09-16T19:04:52.942295] [INFO ] [REMOTE ] KD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:52.944442] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-09-16T19:04:53.174954] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-05. [2025-09-16T19:04:53.843632] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-09-16T19:04:53.850026] [INFO ] [REMOTE ] KBH: Full fetch complete (53033 records). [2025-09-16T19:04:53.850159] [DEBUG] [VALIDATOR ] KBH: Validating primary data. [2025-09-16T19:04:53.979973] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-09. [2025-09-16T19:04:54.231035] [DEBUG] [VALIDATOR ] KBH: All expected trading days are present. [2025-09-16T19:04:54.231112] [DEBUG] [VALIDATOR ] KBH: Primary data validation passed. [2025-09-16T19:04:54.231868] [INFO ] [MAIN ] KBH: Fetched primary data is valid. [2025-09-16T19:04:54.231913] [DEBUG] [LOCAL ] KBH: Saving primary data. [2025-09-16T19:04:54.232943] [DEBUG] [LOCAL ] KBH: Trimmed 53033 records to 36047 (keeping 150 days of data) [2025-09-16T19:04:54.450030] [DEBUG] [LOCAL ] KBH: Primary data saved (36047 records). [2025-09-16T19:04:54.450171] [INFO ] [MAIN ] KBH: Full history saved. [2025-09-16T19:04:54.450269] [INFO ] [MAIN ] Update Progress: 1178/2369 symbols completed. KBH took 19.8s. Estimated remaining: 2h 6m 14s. Rate: 9.4 symbols/min [2025-09-16T19:04:54.627069] [INFO ] [MAIN ] KEN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:54.632231] [DEBUG] [LOCAL ] KEN: Loaded 3286 existing records from primary storage. [2025-09-16T19:04:54.632312] [INFO ] [MAIN ] KEN: Loaded 3286 existing records. [2025-09-16T19:04:54.811573] [WARN ] [MAIN ] KEN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:54.783343] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-02. [2025-09-16T19:04:54.813309] [INFO ] [MAIN ] KEN: Wiping existing primary data before full refresh. [2025-09-16T19:04:54.813389] [INFO ] [LOCAL ] KEN: Starting deletion of primary data. [2025-09-16T19:04:54.813670] [DEBUG] [LOCAL ] KEN: Deleted primary data file. [2025-09-16T19:04:54.813721] [INFO ] [MAIN ] KEN: Starting full primary refresh. [2025-09-16T19:04:55.238396] [INFO ] [REMOTE ] KEN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:55.241557] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-09-16T19:04:55.257724] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-09-16T19:04:55.260908] [INFO ] [REMOTE ] KB: Full fetch complete (26755 records). [2025-09-16T19:04:55.260988] [DEBUG] [VALIDATOR ] KB: Validating primary data. [2025-09-16T19:04:55.584560] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-02. [2025-09-16T19:04:55.627370] [DEBUG] [VALIDATOR ] KB: All expected trading days are present. [2025-09-16T19:04:55.627453] [DEBUG] [VALIDATOR ] KB: Primary data validation passed. [2025-09-16T19:04:55.628032] [INFO ] [MAIN ] KB: Fetched primary data is valid. [2025-09-16T19:04:55.628076] [DEBUG] [LOCAL ] KB: Saving primary data. [2025-09-16T19:04:55.628756] [DEBUG] [LOCAL ] KB: Trimmed 26755 records to 17934 (keeping 150 days of data) [2025-09-16T19:04:55.740256] [DEBUG] [LOCAL ] KB: Primary data saved (17934 records). [2025-09-16T19:04:55.740419] [INFO ] [MAIN ] KB: Full history saved. [2025-09-16T19:04:55.740519] [INFO ] [MAIN ] Update Progress: 1179/2369 symbols completed. KB took 31.9s. Estimated remaining: 2h 6m 3s. Rate: 9.4 symbols/min [2025-09-16T19:04:55.910934] [INFO ] [MAIN ] KEP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:04:55.934975] [DEBUG] [LOCAL ] KEP: Loaded 24216 existing records from primary storage. [2025-09-16T19:04:55.935126] [INFO ] [MAIN ] KEP: Loaded 24216 existing records. [2025-09-16T19:04:56.042107] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-09-16T19:04:56.125779] [WARN ] [MAIN ] KEP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:04:56.126556] [INFO ] [MAIN ] KEP: Wiping existing primary data before full refresh. [2025-09-16T19:04:56.126608] [INFO ] [LOCAL ] KEP: Starting deletion of primary data. [2025-09-16T19:04:56.127572] [DEBUG] [LOCAL ] KEP: Deleted primary data file. [2025-09-16T19:04:56.127632] [INFO ] [MAIN ] KEP: Starting full primary refresh. [2025-09-16T19:04:56.380924] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-03. [2025-09-16T19:04:56.527526] [INFO ] [REMOTE ] KEP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:04:56.570425] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-09-16T19:04:57.098022] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-09-16T19:04:57.174976] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-04. [2025-09-16T19:04:57.927335] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-09-16T19:04:57.974884] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-03. [2025-09-16T19:04:58.774924] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-02. [2025-09-16T19:04:59.574873] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-05. [2025-09-16T19:04:59.745216] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T19:04:59.860058] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-09-16T19:05:00.164091] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-09-16T19:05:00.374930] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-03. [2025-09-16T19:05:01.174903] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-04. [2025-09-16T19:05:01.682283] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T19:05:01.974920] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-06. [2025-09-16T19:05:02.757898] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T19:05:02.774841] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-07. [2025-09-16T19:05:03.383243] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-09-16T19:05:03.502657] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-04 (2.33 seconds). [2025-09-16T19:05:03.574904] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-05. [2025-09-16T19:05:04.374905] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-06. [2025-09-16T19:05:05.082185] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-09-16T19:05:05.174946] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-08. [2025-09-16T19:05:05.966586] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T19:05:05.984530] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-09. [2025-09-16T19:05:06.006384] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T19:05:06.585432] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-09-16T19:05:06.586493] [INFO ] [REMOTE ] KEN: Full fetch complete (4877 records). [2025-09-16T19:05:06.586579] [DEBUG] [VALIDATOR ] KEN: Validating primary data. [2025-09-16T19:05:06.779998] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-04. [2025-09-16T19:05:06.967906] [DEBUG] [VALIDATOR ] KEN: All expected trading days are present. [2025-09-16T19:05:06.968004] [DEBUG] [VALIDATOR ] KEN: Primary data validation passed. [2025-09-16T19:05:06.968180] [INFO ] [MAIN ] KEN: Fetched primary data is valid. [2025-09-16T19:05:06.968222] [DEBUG] [LOCAL ] KEN: Saving primary data. [2025-09-16T19:05:06.968629] [DEBUG] [LOCAL ] KEN: Trimmed 4877 records to 3309 (keeping 150 days of data) [2025-09-16T19:05:06.989543] [DEBUG] [LOCAL ] KEN: Primary data saved (3309 records). [2025-09-16T19:05:06.989695] [INFO ] [MAIN ] KEN: Full history saved. [2025-09-16T19:05:06.989807] [INFO ] [MAIN ] Update Progress: 1180/2369 symbols completed. KEN took 12.5s. Estimated remaining: 2h 6m 1s. Rate: 9.4 symbols/min [2025-09-16T19:05:07.167385] [INFO ] [MAIN ] KEX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:05:07.195876] [DEBUG] [LOCAL ] KEX: Loaded 27514 existing records from primary storage. [2025-09-16T19:05:07.196013] [INFO ] [MAIN ] KEX: Loaded 27514 existing records. [2025-09-16T19:05:07.378137] [WARN ] [MAIN ] KEX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:05:07.378431] [INFO ] [MAIN ] KEX: Wiping existing primary data before full refresh. [2025-09-16T19:05:07.378485] [INFO ] [LOCAL ] KEX: Starting deletion of primary data. [2025-09-16T19:05:07.379106] [DEBUG] [LOCAL ] KEX: Deleted primary data file. [2025-09-16T19:05:07.379153] [INFO ] [MAIN ] KEX: Starting full primary refresh. [2025-09-16T19:05:07.580085] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-07. [2025-09-16T19:05:07.742005] [INFO ] [REMOTE ] KEX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:05:08.008378] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T19:05:08.374896] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-05. [2025-09-16T19:05:09.174993] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-06. [2025-09-16T19:05:09.482526] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-09-16T19:05:09.829687] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T19:05:09.974939] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-06. [2025-09-16T19:05:10.774904] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-02. [2025-09-16T19:05:10.999042] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-09-16T19:05:11.454481] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T19:05:11.574874] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-07. [2025-09-16T19:05:12.374898] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-07. [2025-09-16T19:05:13.174906] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-08. [2025-09-16T19:05:13.916531] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-09-16T19:05:13.974863] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-08. [2025-09-16T19:05:15.343385] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-09-16T19:05:15.350666] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-09. [2025-09-16T19:05:16.599045] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-09-16T19:05:16.604387] [INFO ] [REMOTE ] KEP: Full fetch complete (32063 records). [2025-09-16T19:05:16.604543] [DEBUG] [VALIDATOR ] KEP: Validating primary data. [2025-09-16T19:05:17.017342] [DEBUG] [VALIDATOR ] KEP: All expected trading days are present. [2025-09-16T19:05:17.017427] [DEBUG] [VALIDATOR ] KEP: Primary data validation passed. [2025-09-16T19:05:17.018043] [INFO ] [MAIN ] KEP: Fetched primary data is valid. [2025-09-16T19:05:17.018104] [DEBUG] [LOCAL ] KEP: Saving primary data. [2025-09-16T19:05:17.018957] [DEBUG] [LOCAL ] KEP: Trimmed 32063 records to 24401 (keeping 150 days of data) [2025-09-16T19:05:17.164222] [DEBUG] [LOCAL ] KEP: Primary data saved (24401 records). [2025-09-16T19:05:17.164353] [INFO ] [MAIN ] KEP: Full history saved. [2025-09-16T19:05:17.164436] [INFO ] [MAIN ] Update Progress: 1181/2369 symbols completed. KEP took 21.4s. Estimated remaining: 2h 5m 59s. Rate: 9.4 symbols/min [2025-09-16T19:05:17.340295] [INFO ] [MAIN ] KEY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:05:17.340504] [DEBUG] [LOCAL ] KEY: No existing data file found in primary storage. [2025-09-16T19:05:17.340542] [WARN ] [MAIN ] KEY: No existing data found, performing full refresh instead. [2025-09-16T19:05:17.340684] [INFO ] [MAIN ] KEY: Wiping existing primary data before full refresh. [2025-09-16T19:05:17.340725] [INFO ] [LOCAL ] KEY: Starting deletion of primary data. [2025-09-16T19:05:17.340766] [DEBUG] [LOCAL ] KEY: No primary data file found to delete. [2025-09-16T19:05:17.340798] [INFO ] [MAIN ] KEY: Starting full primary refresh. [2025-09-16T19:05:17.697812] [INFO ] [REMOTE ] KEY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:05:17.698142] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-02. [2025-09-16T19:05:19.482396] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T19:05:19.492746] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-03. [2025-09-16T19:05:21.329933] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T19:05:21.341678] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-04. [2025-09-16T19:05:23.365034] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-09-16T19:05:23.377737] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-05. [2025-09-16T19:05:25.247024] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-09-16T19:05:25.258887] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-06. [2025-09-16T19:05:53.274250] [DEBUG] [REMOTE ] KEX: Error fetching primary data for 2025-02: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read)) [2025-09-16T19:05:53.274447] [WARN ] [REMOTE ] KEX: No data returned for month 2025-02 [2025-09-16T19:05:53.275170] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-03. [2025-09-16T19:05:54.748964] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-09-16T19:05:54.757496] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-04. [2025-09-16T19:05:54.802246] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-06 (29.54 seconds). [2025-09-16T19:05:54.814883] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-07. [2025-09-16T19:05:54.874921] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-07 (43.30 seconds). [2025-09-16T19:05:54.887855] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-08. [2025-09-16T19:05:54.978170] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-08 (41.80 seconds). [2025-09-16T19:05:54.989666] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-09. [2025-09-16T19:05:56.031893] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-09-16T19:05:56.040127] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-05. [2025-09-16T19:05:56.301721] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-09-16T19:05:56.307829] [INFO ] [REMOTE ] KBR: Full fetch complete (52832 records). [2025-09-16T19:05:56.307974] [DEBUG] [VALIDATOR ] KBR: Validating primary data. [2025-09-16T19:05:56.677947] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-09-16T19:05:56.755686] [DEBUG] [VALIDATOR ] KBR: All expected trading days are present. [2025-09-16T19:05:56.756048] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-09. [2025-09-16T19:05:56.755784] [DEBUG] [VALIDATOR ] KBR: Primary data validation passed. [2025-09-16T19:05:56.757841] [INFO ] [MAIN ] KBR: Fetched primary data is valid. [2025-09-16T19:05:56.758023] [DEBUG] [LOCAL ] KBR: Saving primary data. [2025-09-16T19:05:56.759513] [DEBUG] [LOCAL ] KBR: Trimmed 52832 records to 35949 (keeping 150 days of data) [2025-09-16T19:05:56.999766] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-07 (2.18 seconds). [2025-09-16T19:05:57.021246] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-08. [2025-09-16T19:05:57.038197] [DEBUG] [LOCAL ] KBR: Primary data saved (35949 records). [2025-09-16T19:05:57.038306] [INFO ] [MAIN ] KBR: Full history saved. [2025-09-16T19:05:57.038411] [INFO ] [MAIN ] Update Progress: 1182/2369 symbols completed. KBR took 1m 18s. Estimated remaining: 2h 6m 26s. Rate: 9.4 symbols/min [2025-09-16T19:05:57.215147] [INFO ] [MAIN ] KEYS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:05:57.249623] [DEBUG] [LOCAL ] KEYS: Loaded 31401 existing records from primary storage. [2025-09-16T19:05:57.249809] [INFO ] [MAIN ] KEYS: Loaded 31401 existing records. [2025-09-16T19:05:57.432553] [WARN ] [MAIN ] KEYS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:05:57.433375] [INFO ] [MAIN ] KEYS: Wiping existing primary data before full refresh. [2025-09-16T19:05:57.433565] [INFO ] [LOCAL ] KEYS: Starting deletion of primary data. [2025-09-16T19:05:57.434603] [DEBUG] [LOCAL ] KEYS: Deleted primary data file. [2025-09-16T19:05:57.434703] [INFO ] [MAIN ] KEYS: Starting full primary refresh. [2025-09-16T19:05:57.719879] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T19:05:57.769320] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-06. [2025-09-16T19:05:57.848834] [INFO ] [REMOTE ] KEYS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:05:57.849208] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-02. [2025-09-16T19:05:58.682944] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T19:05:58.710900] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-09-16T19:05:58.717624] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-09. [2025-09-16T19:05:58.719023] [INFO ] [REMOTE ] KD: Full fetch complete (56903 records). [2025-09-16T19:05:58.719125] [DEBUG] [VALIDATOR ] KD: Validating primary data. [2025-09-16T19:05:59.107589] [DEBUG] [VALIDATOR ] KD: All expected trading days are present. [2025-09-16T19:05:59.107948] [DEBUG] [VALIDATOR ] KD: Primary data validation passed. [2025-09-16T19:05:59.108870] [INFO ] [MAIN ] KD: Fetched primary data is valid. [2025-09-16T19:05:59.108954] [DEBUG] [LOCAL ] KD: Saving primary data. [2025-09-16T19:05:59.110304] [DEBUG] [LOCAL ] KD: Trimmed 56903 records to 38888 (keeping 150 days of data) [2025-09-16T19:05:59.336177] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T19:05:59.362843] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-07. [2025-09-16T19:05:59.372191] [DEBUG] [LOCAL ] KD: Primary data saved (38888 records). [2025-09-16T19:05:59.372348] [INFO ] [MAIN ] KD: Full history saved. [2025-09-16T19:05:59.372440] [INFO ] [MAIN ] Update Progress: 1183/2369 symbols completed. KD took 1m 7s. Estimated remaining: 2h 6m 15s. Rate: 9.4 symbols/min [2025-09-16T19:05:59.542756] [INFO ] [MAIN ] KF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:05:59.546529] [DEBUG] [LOCAL ] KF: Loaded 1744 existing records from primary storage. [2025-09-16T19:05:59.546665] [INFO ] [MAIN ] KF: Loaded 1744 existing records. [2025-09-16T19:05:59.712571] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-09-16T19:05:59.754993] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-03. [2025-09-16T19:05:59.759313] [WARN ] [MAIN ] KF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:05:59.760551] [INFO ] [MAIN ] KF: Wiping existing primary data before full refresh. [2025-09-16T19:05:59.760603] [INFO ] [LOCAL ] KF: Starting deletion of primary data. [2025-09-16T19:05:59.760798] [DEBUG] [LOCAL ] KF: Deleted primary data file. [2025-09-16T19:05:59.760845] [INFO ] [MAIN ] KF: Starting full primary refresh. [2025-09-16T19:06:00.119637] [INFO ] [REMOTE ] KF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:00.120124] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-02. [2025-09-16T19:06:00.166126] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-09-16T19:06:00.172362] [INFO ] [REMOTE ] KEY: Full fetch complete (57927 records). [2025-09-16T19:06:00.172522] [DEBUG] [VALIDATOR ] KEY: Validating primary data. [2025-09-16T19:06:00.372339] [WARN ] [VALIDATOR ] KEY: Validation failed - insufficient historical data. Have 149 unique days, need 150 days [2025-09-16T19:06:00.372957] [FATAL] [MAIN ] KEY: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:06:00.373012] [WARN ] [MAIN ] KEY: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:06:00.373059] [INFO ] [LOCAL ] KEY: Starting deletion of primary data. [2025-09-16T19:06:00.373122] [DEBUG] [LOCAL ] KEY: No primary data file found to delete. [2025-09-16T19:06:00.373181] [INFO ] [LOCAL ] KEY: Starting deletion of derivative data. [2025-09-16T19:06:00.373242] [DEBUG] [LOCAL ] KEY: No derivative data file found to delete. [2025-09-16T19:06:00.373316] [INFO ] [MAIN ] Update Progress: 1184/2369 symbols completed. KEY took 43.2s. Estimated remaining: 2h 6m 4s. Rate: 9.4 symbols/min [2025-09-16T19:06:00.545543] [INFO ] [MAIN ] KFRC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:00.560799] [DEBUG] [LOCAL ] KFRC: Loaded 14809 existing records from primary storage. [2025-09-16T19:06:00.560922] [INFO ] [MAIN ] KFRC: Loaded 14809 existing records. [2025-09-16T19:06:00.745627] [WARN ] [MAIN ] KFRC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:00.746091] [INFO ] [MAIN ] KFRC: Wiping existing primary data before full refresh. [2025-09-16T19:06:00.746187] [INFO ] [LOCAL ] KFRC: Starting deletion of primary data. [2025-09-16T19:06:00.746989] [DEBUG] [LOCAL ] KFRC: Deleted primary data file. [2025-09-16T19:06:00.747091] [INFO ] [MAIN ] KFRC: Starting full primary refresh. [2025-09-16T19:06:00.786282] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-09-16T19:06:00.821311] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-03. [2025-09-16T19:06:01.003750] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T19:06:01.078512] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-08. [2025-09-16T19:06:01.153929] [INFO ] [REMOTE ] KFRC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:01.154333] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-02. [2025-09-16T19:06:01.467226] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-09-16T19:06:01.468756] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-04. [2025-09-16T19:06:01.627124] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-09-16T19:06:01.636555] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-04. [2025-09-16T19:06:01.956464] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-09-16T19:06:01.957868] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-05. [2025-09-16T19:06:02.085114] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-09-16T19:06:02.088504] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-03. [2025-09-16T19:06:02.552712] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-09-16T19:06:02.554205] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-06. [2025-09-16T19:06:02.669471] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T19:06:02.678402] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-09. [2025-09-16T19:06:02.756603] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-09-16T19:06:02.760363] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-04. [2025-09-16T19:06:03.117409] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T19:06:03.127509] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-05. [2025-09-16T19:06:03.193230] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-09-16T19:06:03.194847] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-07. [2025-09-16T19:06:03.720507] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-09-16T19:06:03.721570] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-08. [2025-09-16T19:06:03.739030] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-09-16T19:06:03.743507] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-05. [2025-09-16T19:06:04.019238] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T19:06:04.024909] [INFO ] [REMOTE ] KEX: Full fetch complete (37388 records). [2025-09-16T19:06:04.025026] [DEBUG] [VALIDATOR ] KEX: Validating primary data. [2025-09-16T19:06:04.110932] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-09-16T19:06:04.146747] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-09. [2025-09-16T19:06:04.239406] [WARN ] [VALIDATOR ] KEX: Validation failed - insufficient historical data. Have 137 unique days, need 150 days [2025-09-16T19:06:04.239851] [FATAL] [MAIN ] KEX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:06:04.239909] [WARN ] [MAIN ] KEX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:06:04.239962] [INFO ] [LOCAL ] KEX: Starting deletion of primary data. [2025-09-16T19:06:04.240021] [DEBUG] [LOCAL ] KEX: No primary data file found to delete. [2025-09-16T19:06:04.240057] [INFO ] [LOCAL ] KEX: Starting deletion of derivative data. [2025-09-16T19:06:04.240449] [DEBUG] [LOCAL ] KEX: Deleted derivative data file. [2025-09-16T19:06:04.240561] [INFO ] [MAIN ] Update Progress: 1185/2369 symbols completed. KEX took 57.3s. Estimated remaining: 2h 5m 55s. Rate: 9.4 symbols/min [2025-09-16T19:06:04.417046] [INFO ] [MAIN ] KFS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:04.448253] [DEBUG] [LOCAL ] KFS: Loaded 6290 existing records from primary storage. [2025-09-16T19:06:04.448396] [INFO ] [MAIN ] KFS: Loaded 6290 existing records. [2025-09-16T19:06:04.510470] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-09-16T19:06:04.573624] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-06. [2025-09-16T19:06:04.627410] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-09-16T19:06:04.652476] [INFO ] [REMOTE ] KF: Full fetch complete (2690 records). [2025-09-16T19:06:04.655856] [WARN ] [MAIN ] KFS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:04.655998] [DEBUG] [VALIDATOR ] KF: Validating primary data. [2025-09-16T19:06:04.667927] [INFO ] [MAIN ] KFS: Wiping existing primary data before full refresh. [2025-09-16T19:06:04.701214] [INFO ] [LOCAL ] KFS: Starting deletion of primary data. [2025-09-16T19:06:04.751076] [DEBUG] [LOCAL ] KFS: Deleted primary data file. [2025-09-16T19:06:04.758605] [INFO ] [MAIN ] KFS: Starting full primary refresh. [2025-09-16T19:06:05.124569] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-09-16T19:06:05.255592] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-06. [2025-09-16T19:06:05.288377] [DEBUG] [VALIDATOR ] KF: All expected trading days are present. [2025-09-16T19:06:05.338807] [DEBUG] [VALIDATOR ] KF: Primary data validation passed. [2025-09-16T19:06:05.353092] [INFO ] [MAIN ] KF: Fetched primary data is valid. [2025-09-16T19:06:05.386566] [DEBUG] [LOCAL ] KF: Saving primary data. [2025-09-16T19:06:05.417146] [DEBUG] [LOCAL ] KF: Trimmed 2690 records to 1755 (keeping 150 days of data) [2025-09-16T19:06:05.420447] [INFO ] [REMOTE ] KFS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:05.421130] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-02. [2025-09-16T19:06:05.432430] [DEBUG] [LOCAL ] KF: Primary data saved (1755 records). [2025-09-16T19:06:05.432514] [INFO ] [MAIN ] KF: Full history saved. [2025-09-16T19:06:05.432614] [INFO ] [MAIN ] Update Progress: 1186/2369 symbols completed. KF took 6.1s. Estimated remaining: 2h 5m 43s. Rate: 9.4 symbols/min [2025-09-16T19:06:05.604770] [INFO ] [MAIN ] KFY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:05.626803] [DEBUG] [LOCAL ] KFY: Loaded 21543 existing records from primary storage. [2025-09-16T19:06:05.626976] [INFO ] [MAIN ] KFY: Loaded 21543 existing records. [2025-09-16T19:06:05.810206] [WARN ] [MAIN ] KFY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:05.810465] [INFO ] [MAIN ] KFY: Wiping existing primary data before full refresh. [2025-09-16T19:06:05.810514] [INFO ] [LOCAL ] KFY: Starting deletion of primary data. [2025-09-16T19:06:05.811150] [DEBUG] [LOCAL ] KFY: Deleted primary data file. [2025-09-16T19:06:05.811202] [INFO ] [MAIN ] KFY: Starting full primary refresh. [2025-09-16T19:06:06.111882] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T19:06:06.114091] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-03. [2025-09-16T19:06:06.194843] [INFO ] [REMOTE ] KFY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:06.195516] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-02. [2025-09-16T19:06:06.225810] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T19:06:06.237725] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-07. [2025-09-16T19:06:06.279594] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-09-16T19:06:06.283889] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-07. [2025-09-16T19:06:06.933035] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-09-16T19:06:06.934974] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-04. [2025-09-16T19:06:07.438024] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-09-16T19:06:07.446402] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-03. [2025-09-16T19:06:07.472726] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-09-16T19:06:07.477909] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-08. [2025-09-16T19:06:07.746014] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-09-16T19:06:07.748273] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-05. [2025-09-16T19:06:07.884680] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T19:06:07.894270] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-08. [2025-09-16T19:06:08.565014] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T19:06:08.568765] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-06. [2025-09-16T19:06:08.679878] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-09-16T19:06:08.686935] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-09. [2025-09-16T19:06:08.918472] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-09-16T19:06:08.927502] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-04. [2025-09-16T19:06:09.479710] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-09-16T19:06:09.482999] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-07. [2025-09-16T19:06:09.513441] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-09-16T19:06:09.524179] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-09. [2025-09-16T19:06:09.822314] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T19:06:09.825504] [INFO ] [REMOTE ] KFRC: Full fetch complete (20233 records). [2025-09-16T19:06:09.825599] [DEBUG] [VALIDATOR ] KFRC: Validating primary data. [2025-09-16T19:06:10.238148] [DEBUG] [VALIDATOR ] KFRC: All expected trading days are present. [2025-09-16T19:06:10.238244] [DEBUG] [VALIDATOR ] KFRC: Primary data validation passed. [2025-09-16T19:06:10.238750] [INFO ] [MAIN ] KFRC: Fetched primary data is valid. [2025-09-16T19:06:10.238806] [DEBUG] [LOCAL ] KFRC: Saving primary data. [2025-09-16T19:06:10.240535] [DEBUG] [LOCAL ] KFRC: Trimmed 20233 records to 14991 (keeping 150 days of data) [2025-09-16T19:06:10.318799] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-09-16T19:06:10.323557] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-08. [2025-09-16T19:06:10.352233] [DEBUG] [LOCAL ] KFRC: Primary data saved (14991 records). [2025-09-16T19:06:10.352342] [INFO ] [MAIN ] KFRC: Full history saved. [2025-09-16T19:06:10.352435] [INFO ] [MAIN ] Update Progress: 1187/2369 symbols completed. KFRC took 10.0s. Estimated remaining: 2h 5m 35s. Rate: 9.4 symbols/min [2025-09-16T19:06:10.538663] [INFO ] [MAIN ] KGC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:10.542203] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-09-16T19:06:10.550107] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-05. [2025-09-16T19:06:10.595402] [DEBUG] [LOCAL ] KGC: Loaded 39660 existing records from primary storage. [2025-09-16T19:06:10.595547] [INFO ] [MAIN ] KGC: Loaded 39660 existing records. [2025-09-16T19:06:10.774550] [WARN ] [MAIN ] KGC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:10.774794] [INFO ] [MAIN ] KGC: Wiping existing primary data before full refresh. [2025-09-16T19:06:10.774842] [INFO ] [LOCAL ] KGC: Starting deletion of primary data. [2025-09-16T19:06:10.775616] [DEBUG] [LOCAL ] KGC: Deleted primary data file. [2025-09-16T19:06:10.775685] [INFO ] [MAIN ] KGC: Starting full primary refresh. [2025-09-16T19:06:11.126981] [INFO ] [REMOTE ] KGC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:11.127316] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-02. [2025-09-16T19:06:11.142750] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T19:06:11.154367] [INFO ] [REMOTE ] KEYS: Full fetch complete (47480 records). [2025-09-16T19:06:11.155116] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-09-16T19:06:11.155178] [DEBUG] [VALIDATOR ] KEYS: Validating primary data. [2025-09-16T19:06:11.157899] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-09. [2025-09-16T19:06:11.561054] [DEBUG] [VALIDATOR ] KEYS: All expected trading days are present. [2025-09-16T19:06:11.561164] [DEBUG] [VALIDATOR ] KEYS: Primary data validation passed. [2025-09-16T19:06:11.561888] [INFO ] [MAIN ] KEYS: Fetched primary data is valid. [2025-09-16T19:06:11.561932] [DEBUG] [LOCAL ] KEYS: Saving primary data. [2025-09-16T19:06:11.563033] [DEBUG] [LOCAL ] KEYS: Trimmed 47480 records to 31718 (keeping 150 days of data) [2025-09-16T19:06:11.706054] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-09-16T19:06:11.713958] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-06. [2025-09-16T19:06:11.798346] [DEBUG] [LOCAL ] KEYS: Primary data saved (31718 records). [2025-09-16T19:06:11.798506] [INFO ] [MAIN ] KEYS: Full history saved. [2025-09-16T19:06:11.798603] [INFO ] [MAIN ] Update Progress: 1188/2369 symbols completed. KEYS took 14.8s. Estimated remaining: 2h 5m 24s. Rate: 9.4 symbols/min [2025-09-16T19:06:11.971245] [INFO ] [MAIN ] KGS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:12.006699] [DEBUG] [LOCAL ] KGS: Loaded 33588 existing records from primary storage. [2025-09-16T19:06:12.006843] [INFO ] [MAIN ] KGS: Loaded 33588 existing records. [2025-09-16T19:06:12.123669] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T19:06:12.154484] [INFO ] [REMOTE ] KFS: Full fetch complete (8397 records). [2025-09-16T19:06:12.187751] [DEBUG] [VALIDATOR ] KFS: Validating primary data. [2025-09-16T19:06:12.190987] [WARN ] [MAIN ] KGS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:12.191310] [INFO ] [MAIN ] KGS: Wiping existing primary data before full refresh. [2025-09-16T19:06:12.260409] [INFO ] [LOCAL ] KGS: Starting deletion of primary data. [2025-09-16T19:06:12.291972] [DEBUG] [LOCAL ] KGS: Deleted primary data file. [2025-09-16T19:06:12.292091] [INFO ] [MAIN ] KGS: Starting full primary refresh. [2025-09-16T19:06:12.898991] [DEBUG] [VALIDATOR ] KFS: All expected trading days are present. [2025-09-16T19:06:12.929721] [DEBUG] [VALIDATOR ] KFS: Primary data validation passed. [2025-09-16T19:06:12.937359] [INFO ] [MAIN ] KFS: Fetched primary data is valid. [2025-09-16T19:06:12.942661] [DEBUG] [LOCAL ] KFS: Saving primary data. [2025-09-16T19:06:12.973891] [INFO ] [REMOTE ] KGS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:12.974948] [DEBUG] [LOCAL ] KFS: Trimmed 8397 records to 6350 (keeping 150 days of data) [2025-09-16T19:06:12.975438] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-02. [2025-09-16T19:06:13.037598] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T19:06:13.053197] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-07. [2025-09-16T19:06:13.053394] [DEBUG] [LOCAL ] KFS: Primary data saved (6350 records). [2025-09-16T19:06:13.054750] [INFO ] [MAIN ] KFS: Full history saved. [2025-09-16T19:06:13.054990] [INFO ] [MAIN ] Update Progress: 1189/2369 symbols completed. KFS took 8.8s. Estimated remaining: 2h 5m 13s. Rate: 9.4 symbols/min [2025-09-16T19:06:13.209672] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-02 (2.08 seconds). [2025-09-16T19:06:13.257111] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-03. [2025-09-16T19:06:13.283409] [INFO ] [MAIN ] KIM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:13.329960] [DEBUG] [LOCAL ] KIM: Loaded 39124 existing records from primary storage. [2025-09-16T19:06:13.330133] [INFO ] [MAIN ] KIM: Loaded 39124 existing records. [2025-09-16T19:06:13.505259] [WARN ] [MAIN ] KIM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:13.505538] [INFO ] [MAIN ] KIM: Wiping existing primary data before full refresh. [2025-09-16T19:06:13.505633] [INFO ] [LOCAL ] KIM: Starting deletion of primary data. [2025-09-16T19:06:13.506529] [DEBUG] [LOCAL ] KIM: Deleted primary data file. [2025-09-16T19:06:13.506577] [INFO ] [MAIN ] KIM: Starting full primary refresh. [2025-09-16T19:06:13.859037] [INFO ] [REMOTE ] KIM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:13.859416] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-02. [2025-09-16T19:06:14.402505] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T19:06:14.409779] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-08. [2025-09-16T19:06:14.460083] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T19:06:14.472837] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-03. [2025-09-16T19:06:14.992842] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T19:06:15.004562] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-04. [2025-09-16T19:06:15.423228] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T19:06:15.433491] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-03. [2025-09-16T19:06:15.503065] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-09-16T19:06:15.508799] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-09. [2025-09-16T19:06:15.817174] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-09-16T19:06:15.827075] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-04. [2025-09-16T19:06:16.822536] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T19:06:16.834239] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-05. [2025-09-16T19:06:17.052237] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T19:06:17.061813] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-05. [2025-09-16T19:06:17.174066] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-09-16T19:06:17.178483] [INFO ] [REMOTE ] KFY: Full fetch complete (33285 records). [2025-09-16T19:06:17.178579] [DEBUG] [VALIDATOR ] KFY: Validating primary data. [2025-09-16T19:06:17.557206] [DEBUG] [VALIDATOR ] KFY: All expected trading days are present. [2025-09-16T19:06:17.557357] [DEBUG] [VALIDATOR ] KFY: Primary data validation passed. [2025-09-16T19:06:17.558010] [INFO ] [MAIN ] KFY: Fetched primary data is valid. [2025-09-16T19:06:17.558066] [DEBUG] [LOCAL ] KFY: Saving primary data. [2025-09-16T19:06:17.559018] [DEBUG] [LOCAL ] KFY: Trimmed 33285 records to 21739 (keeping 150 days of data) [2025-09-16T19:06:17.681938] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-03 (2.25 seconds). [2025-09-16T19:06:17.697924] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-04. [2025-09-16T19:06:17.737798] [DEBUG] [LOCAL ] KFY: Primary data saved (21739 records). [2025-09-16T19:06:17.737959] [INFO ] [MAIN ] KFY: Full history saved. [2025-09-16T19:06:17.738072] [INFO ] [MAIN ] Update Progress: 1190/2369 symbols completed. KFY took 12.3s. Estimated remaining: 2h 5m 5s. Rate: 9.4 symbols/min [2025-09-16T19:06:17.910238] [INFO ] [MAIN ] KIO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:17.927578] [DEBUG] [LOCAL ] KIO: Loaded 15394 existing records from primary storage. [2025-09-16T19:06:17.927755] [INFO ] [MAIN ] KIO: Loaded 15394 existing records. [2025-09-16T19:06:18.107422] [WARN ] [MAIN ] KIO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:18.107725] [INFO ] [MAIN ] KIO: Wiping existing primary data before full refresh. [2025-09-16T19:06:18.107772] [INFO ] [LOCAL ] KIO: Starting deletion of primary data. [2025-09-16T19:06:18.108350] [DEBUG] [LOCAL ] KIO: Deleted primary data file. [2025-09-16T19:06:18.108397] [INFO ] [MAIN ] KIO: Starting full primary refresh. [2025-09-16T19:06:18.463245] [INFO ] [REMOTE ] KIO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:18.463578] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-02. [2025-09-16T19:06:18.497922] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T19:06:18.509077] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-06. [2025-09-16T19:06:18.627957] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T19:06:18.637505] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-06. [2025-09-16T19:06:19.130607] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T19:06:19.142176] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-05. [2025-09-16T19:06:19.307758] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T19:06:19.312970] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-03. [2025-09-16T19:06:20.153248] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T19:06:20.162363] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-07. [2025-09-16T19:06:20.375421] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-09-16T19:06:20.380672] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-04. [2025-09-16T19:06:20.446234] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-09-16T19:06:20.457258] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-07. [2025-09-16T19:06:21.000723] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-09-16T19:06:21.011994] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-06. [2025-09-16T19:06:21.302198] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-09-16T19:06:21.307282] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-05. [2025-09-16T19:06:21.820059] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-09-16T19:06:21.830381] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-08. [2025-09-16T19:06:22.230532] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-09-16T19:06:22.243635] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-08. [2025-09-16T19:06:22.342523] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-09-16T19:06:22.347385] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-06. [2025-09-16T19:06:22.524596] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-09-16T19:06:22.535687] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-07. [2025-09-16T19:06:23.083470] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-09-16T19:06:23.096841] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-09. [2025-09-16T19:06:23.315129] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-09-16T19:06:23.319335] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-07. [2025-09-16T19:06:24.019366] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T19:06:24.031336] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-08. [2025-09-16T19:06:24.061940] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T19:06:24.374927] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-09. [2025-09-16T19:06:24.469876] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T19:06:24.567677] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T19:06:24.576144] [INFO ] [REMOTE ] KGS: Full fetch complete (49610 records). [2025-09-16T19:06:24.576404] [DEBUG] [VALIDATOR ] KGS: Validating primary data. [2025-09-16T19:06:24.967012] [DEBUG] [VALIDATOR ] KGS: All expected trading days are present. [2025-09-16T19:06:24.967108] [DEBUG] [VALIDATOR ] KGS: Primary data validation passed. [2025-09-16T19:06:24.967877] [INFO ] [MAIN ] KGS: Fetched primary data is valid. [2025-09-16T19:06:24.967925] [DEBUG] [LOCAL ] KGS: Saving primary data. [2025-09-16T19:06:24.969144] [DEBUG] [LOCAL ] KGS: Trimmed 49610 records to 33944 (keeping 150 days of data) [2025-09-16T19:06:25.174877] [DEBUG] [LOCAL ] KGS: Primary data saved (33944 records). [2025-09-16T19:06:25.174983] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-08. [2025-09-16T19:06:25.175876] [INFO ] [MAIN ] KGS: Full history saved. [2025-09-16T19:06:25.175964] [INFO ] [MAIN ] Update Progress: 1191/2369 symbols completed. KGS took 13.4s. Estimated remaining: 2h 4m 59s. Rate: 9.4 symbols/min [2025-09-16T19:06:25.348273] [INFO ] [MAIN ] KKR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:25.387837] [DEBUG] [LOCAL ] KKR: Loaded 39486 existing records from primary storage. [2025-09-16T19:06:25.387997] [INFO ] [MAIN ] KKR: Loaded 39486 existing records. [2025-09-16T19:06:25.563267] [WARN ] [MAIN ] KKR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:25.563474] [INFO ] [MAIN ] KKR: Wiping existing primary data before full refresh. [2025-09-16T19:06:25.563519] [INFO ] [LOCAL ] KKR: Starting deletion of primary data. [2025-09-16T19:06:25.564559] [DEBUG] [LOCAL ] KKR: Deleted primary data file. [2025-09-16T19:06:25.564610] [INFO ] [MAIN ] KKR: Starting full primary refresh. [2025-09-16T19:06:25.923569] [INFO ] [REMOTE ] KKR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:25.957878] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T19:06:25.974845] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-09. [2025-09-16T19:06:26.242992] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T19:06:26.304143] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-09-16T19:06:26.310996] [INFO ] [REMOTE ] KGC: Full fetch complete (58377 records). [2025-09-16T19:06:26.311135] [DEBUG] [VALIDATOR ] KGC: Validating primary data. [2025-09-16T19:06:26.720051] [DEBUG] [VALIDATOR ] KGC: All expected trading days are present. [2025-09-16T19:06:26.720148] [DEBUG] [VALIDATOR ] KGC: Primary data validation passed. [2025-09-16T19:06:26.721054] [INFO ] [MAIN ] KGC: Fetched primary data is valid. [2025-09-16T19:06:26.721104] [DEBUG] [LOCAL ] KGC: Saving primary data. [2025-09-16T19:06:26.722459] [DEBUG] [LOCAL ] KGC: Trimmed 58377 records to 40050 (keeping 150 days of data) [2025-09-16T19:06:26.799787] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-02. [2025-09-16T19:06:27.005656] [DEBUG] [LOCAL ] KGC: Primary data saved (40050 records). [2025-09-16T19:06:27.005787] [INFO ] [MAIN ] KGC: Full history saved. [2025-09-16T19:06:27.007420] [INFO ] [MAIN ] Update Progress: 1192/2369 symbols completed. KGC took 16.7s. Estimated remaining: 2h 4m 48s. Rate: 9.4 symbols/min [2025-09-16T19:06:27.203925] [INFO ] [MAIN ] KKRS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:27.213293] [DEBUG] [LOCAL ] KKRS: Loaded 4474 existing records from primary storage. [2025-09-16T19:06:27.213451] [INFO ] [MAIN ] KKRS: Loaded 4474 existing records. [2025-09-16T19:06:27.407584] [WARN ] [MAIN ] KKRS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:27.408127] [INFO ] [MAIN ] KKRS: Wiping existing primary data before full refresh. [2025-09-16T19:06:27.408583] [INFO ] [LOCAL ] KKRS: Starting deletion of primary data. [2025-09-16T19:06:27.409028] [DEBUG] [LOCAL ] KKRS: Deleted primary data file. [2025-09-16T19:06:27.409081] [INFO ] [MAIN ] KKRS: Starting full primary refresh. [2025-09-16T19:06:27.579836] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-09. [2025-09-16T19:06:27.816910] [INFO ] [REMOTE ] KKRS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:27.835161] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-09-16T19:06:27.841663] [INFO ] [REMOTE ] KIM: Full fetch complete (57694 records). [2025-09-16T19:06:27.841787] [DEBUG] [VALIDATOR ] KIM: Validating primary data. [2025-09-16T19:06:28.234133] [DEBUG] [VALIDATOR ] KIM: All expected trading days are present. [2025-09-16T19:06:28.234237] [DEBUG] [VALIDATOR ] KIM: Primary data validation passed. [2025-09-16T19:06:28.235106] [INFO ] [MAIN ] KIM: Fetched primary data is valid. [2025-09-16T19:06:28.235170] [DEBUG] [LOCAL ] KIM: Saving primary data. [2025-09-16T19:06:28.236512] [DEBUG] [LOCAL ] KIM: Trimmed 57694 records to 39507 (keeping 150 days of data) [2025-09-16T19:06:28.425789] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-02. [2025-09-16T19:06:28.494128] [DEBUG] [LOCAL ] KIM: Primary data saved (39507 records). [2025-09-16T19:06:28.494391] [INFO ] [MAIN ] KIM: Full history saved. [2025-09-16T19:06:28.494540] [INFO ] [MAIN ] Update Progress: 1193/2369 symbols completed. KIM took 15.4s. Estimated remaining: 2h 4m 37s. Rate: 9.4 symbols/min [2025-09-16T19:06:28.671262] [INFO ] [MAIN ] KKRT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:28.671545] [DEBUG] [LOCAL ] KKRT: No existing data file found in primary storage. [2025-09-16T19:06:28.671597] [WARN ] [MAIN ] KKRT: No existing data found, performing full refresh instead. [2025-09-16T19:06:28.671849] [INFO ] [MAIN ] KKRT: Wiping existing primary data before full refresh. [2025-09-16T19:06:28.671902] [INFO ] [LOCAL ] KKRT: Starting deletion of primary data. [2025-09-16T19:06:28.671951] [DEBUG] [LOCAL ] KKRT: No primary data file found to delete. [2025-09-16T19:06:28.671985] [INFO ] [MAIN ] KKRT: Starting full primary refresh. [2025-09-16T19:06:28.785765] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-09-16T19:06:28.835178] [INFO ] [REMOTE ] KIO: Full fetch complete (23428 records). [2025-09-16T19:06:28.879044] [DEBUG] [VALIDATOR ] KIO: Validating primary data. [2025-09-16T19:06:29.015780] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-02 (2.22 seconds). [2025-09-16T19:06:29.116974] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T19:06:29.175719] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-03. [2025-09-16T19:06:29.314487] [INFO ] [REMOTE ] KKRT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:29.517799] [DEBUG] [VALIDATOR ] KIO: All expected trading days are present. [2025-09-16T19:06:29.517884] [DEBUG] [VALIDATOR ] KIO: Primary data validation passed. [2025-09-16T19:06:29.518367] [INFO ] [MAIN ] KIO: Fetched primary data is valid. [2025-09-16T19:06:29.518408] [DEBUG] [LOCAL ] KIO: Saving primary data. [2025-09-16T19:06:29.519133] [DEBUG] [LOCAL ] KIO: Trimmed 23428 records to 15558 (keeping 150 days of data) [2025-09-16T19:06:29.617294] [DEBUG] [LOCAL ] KIO: Primary data saved (15558 records). [2025-09-16T19:06:29.617445] [INFO ] [MAIN ] KIO: Full history saved. [2025-09-16T19:06:29.617532] [INFO ] [MAIN ] Update Progress: 1194/2369 symbols completed. KIO took 11.9s. Estimated remaining: 2h 4m 26s. Rate: 9.4 symbols/min [2025-09-16T19:06:29.793219] [INFO ] [MAIN ] KLAR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:29.793444] [DEBUG] [LOCAL ] KLAR: No existing data file found in primary storage. [2025-09-16T19:06:29.793485] [WARN ] [MAIN ] KLAR: No existing data found, performing full refresh instead. [2025-09-16T19:06:29.793604] [INFO ] [MAIN ] KLAR: Wiping existing primary data before full refresh. [2025-09-16T19:06:29.793661] [INFO ] [LOCAL ] KLAR: Starting deletion of primary data. [2025-09-16T19:06:29.793706] [DEBUG] [LOCAL ] KLAR: No primary data file found to delete. [2025-09-16T19:06:29.793738] [INFO ] [MAIN ] KLAR: Starting full primary refresh. [2025-09-16T19:06:29.984352] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-03. [2025-09-16T19:06:30.157574] [INFO ] [REMOTE ] KLAR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:30.647208] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-09-16T19:06:30.774876] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-04. [2025-09-16T19:06:31.260036] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-09-16T19:06:31.276423] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-09-16T19:06:31.574902] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-05. [2025-09-16T19:06:32.337471] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-09-16T19:06:32.374973] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-02. [2025-09-16T19:06:32.565828] [DEBUG] [REMOTE ] KKRT: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:32.566422] [WARN ] [REMOTE ] KKRT: No data returned for month 2025-02 [2025-09-16T19:06:33.174962] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-03. [2025-09-16T19:06:33.405434] [DEBUG] [REMOTE ] KKRT: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:33.405887] [WARN ] [REMOTE ] KKRT: No data returned for month 2025-03 [2025-09-16T19:06:33.974957] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-04. [2025-09-16T19:06:34.265501] [DEBUG] [REMOTE ] KKRT: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:34.265879] [WARN ] [REMOTE ] KKRT: No data returned for month 2025-04 [2025-09-16T19:06:34.774942] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-02. [2025-09-16T19:06:34.969743] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:34.970077] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-02 [2025-09-16T19:06:35.574926] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-06. [2025-09-16T19:06:36.255462] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-09-16T19:06:36.374892] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-07. [2025-09-16T19:06:36.985581] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-09-16T19:06:37.174931] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-04. [2025-09-16T19:06:37.974900] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-05. [2025-09-16T19:06:38.409329] [INFO ] [REMOTE ] KKRT: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-09-16T19:06:38.774907] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-06. [2025-09-16T19:06:39.058889] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T19:06:39.574882] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-05. [2025-09-16T19:06:39.900142] [INFO ] [REMOTE ] KKRT: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-09-16T19:06:40.374890] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-03. [2025-09-16T19:06:40.581869] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:40.582175] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-03 [2025-09-16T19:06:41.174926] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-08. [2025-09-16T19:06:41.454658] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-09-16T19:06:41.791445] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-09-16T19:06:41.974920] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-09. [2025-09-16T19:06:42.724859] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-09-16T19:06:42.725986] [INFO ] [REMOTE ] KKRS: Full fetch complete (6284 records). [2025-09-16T19:06:42.726053] [DEBUG] [VALIDATOR ] KKRS: Validating primary data. [2025-09-16T19:06:42.776677] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-07. [2025-09-16T19:06:43.108353] [DEBUG] [VALIDATOR ] KKRS: All expected trading days are present. [2025-09-16T19:06:43.108431] [DEBUG] [VALIDATOR ] KKRS: Primary data validation passed. [2025-09-16T19:06:43.108608] [INFO ] [MAIN ] KKRS: Fetched primary data is valid. [2025-09-16T19:06:43.108666] [DEBUG] [LOCAL ] KKRS: Saving primary data. [2025-09-16T19:06:43.109061] [DEBUG] [LOCAL ] KKRS: Trimmed 6284 records to 4505 (keeping 150 days of data) [2025-09-16T19:06:43.136728] [DEBUG] [LOCAL ] KKRS: Primary data saved (4505 records). [2025-09-16T19:06:43.136826] [INFO ] [MAIN ] KKRS: Full history saved. [2025-09-16T19:06:43.136918] [INFO ] [MAIN ] Update Progress: 1195/2369 symbols completed. KKRS took 16.1s. Estimated remaining: 2h 4m 26s. Rate: 9.4 symbols/min [2025-09-16T19:06:43.341795] [INFO ] [MAIN ] KLC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:43.372396] [DEBUG] [LOCAL ] KLC: Loaded 26104 existing records from primary storage. [2025-09-16T19:06:43.372533] [INFO ] [MAIN ] KLC: Loaded 26104 existing records. [2025-09-16T19:06:43.561839] [WARN ] [MAIN ] KLC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:43.562138] [INFO ] [MAIN ] KLC: Wiping existing primary data before full refresh. [2025-09-16T19:06:43.562194] [INFO ] [LOCAL ] KLC: Starting deletion of primary data. [2025-09-16T19:06:43.562816] [DEBUG] [LOCAL ] KLC: Deleted primary data file. [2025-09-16T19:06:43.562870] [INFO ] [MAIN ] KLC: Starting full primary refresh. [2025-09-16T19:06:43.583717] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-06. [2025-09-16T19:06:43.686849] [INFO ] [REMOTE ] KKRT: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-09-16T19:06:43.931037] [INFO ] [REMOTE ] KLC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:44.374923] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-08. [2025-09-16T19:06:45.151416] [INFO ] [REMOTE ] KKRT: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-09-16T19:06:45.174866] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-09. [2025-09-16T19:06:45.412782] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T19:06:45.916409] [INFO ] [REMOTE ] KKRT: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-09-16T19:06:45.918027] [INFO ] [REMOTE ] KKRT: Full fetch complete (7991 records). [2025-09-16T19:06:45.918098] [DEBUG] [VALIDATOR ] KKRT: Validating primary data. [2025-09-16T19:06:45.980014] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-02. [2025-09-16T19:06:46.119175] [WARN ] [VALIDATOR ] KKRT: Validation failed - insufficient historical data. Have 75 unique days, need 150 days [2025-09-16T19:06:46.119487] [FATAL] [MAIN ] KKRT: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:06:46.119569] [WARN ] [MAIN ] KKRT: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:06:46.119635] [INFO ] [LOCAL ] KKRT: Starting deletion of primary data. [2025-09-16T19:06:46.119739] [DEBUG] [LOCAL ] KKRT: No primary data file found to delete. [2025-09-16T19:06:46.119794] [INFO ] [LOCAL ] KKRT: Starting deletion of derivative data. [2025-09-16T19:06:46.119866] [DEBUG] [LOCAL ] KKRT: No derivative data file found to delete. [2025-09-16T19:06:46.119952] [INFO ] [MAIN ] Update Progress: 1196/2369 symbols completed. KKRT took 17.6s. Estimated remaining: 2h 4m 17s. Rate: 9.4 symbols/min [2025-09-16T19:06:46.294282] [INFO ] [MAIN ] KLG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:06:46.325610] [DEBUG] [LOCAL ] KLG: Loaded 30713 existing records from primary storage. [2025-09-16T19:06:46.325786] [INFO ] [MAIN ] KLG: Loaded 30713 existing records. [2025-09-16T19:06:46.506846] [WARN ] [MAIN ] KLG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:06:46.507189] [INFO ] [MAIN ] KLG: Wiping existing primary data before full refresh. [2025-09-16T19:06:46.507251] [INFO ] [LOCAL ] KLG: Starting deletion of primary data. [2025-09-16T19:06:46.508063] [DEBUG] [LOCAL ] KLG: Deleted primary data file. [2025-09-16T19:06:46.508126] [INFO ] [MAIN ] KLG: Starting full primary refresh. [2025-09-16T19:06:46.780395] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-04. [2025-09-16T19:06:46.870937] [INFO ] [REMOTE ] KLG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:06:47.043517] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:47.043891] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-04 [2025-09-16T19:06:47.189464] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-09-16T19:06:47.574900] [INFO ] [REMOTE ] KLG: Fetching primary data for 2025-02. [2025-09-16T19:06:48.374911] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-05. [2025-09-16T19:06:48.581473] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:48.581987] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-05 [2025-09-16T19:06:48.827400] [INFO ] [REMOTE ] KLG: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T19:06:49.174953] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-07. [2025-09-16T19:06:49.974916] [INFO ] [REMOTE ] KLG: Fetching primary data for 2025-03. [2025-09-16T19:06:50.774900] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-03. [2025-09-16T19:06:51.211751] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-09-16T19:06:51.491709] [INFO ] [REMOTE ] KLG: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-09-16T19:06:51.574912] [INFO ] [REMOTE ] KLG: Fetching primary data for 2025-04. [2025-09-16T19:06:51.872837] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T19:06:52.374936] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-08. [2025-09-16T19:06:53.176463] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-06. [2025-09-16T19:06:53.181713] [INFO ] [REMOTE ] KLG: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-09-16T19:06:53.385894] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:53.386267] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-06 [2025-09-16T19:06:53.942365] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T19:06:53.974865] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-09. [2025-09-16T19:06:54.774945] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-07. [2025-09-16T19:06:54.963591] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-07: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:54.964079] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-07 [2025-09-16T19:06:55.574899] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-08. [2025-09-16T19:06:55.759476] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T19:06:55.767062] [INFO ] [REMOTE ] KKR: Full fetch complete (58194 records). [2025-09-16T19:06:55.767249] [DEBUG] [VALIDATOR ] KKR: Validating primary data. [2025-09-16T19:06:55.808601] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-08: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:06:55.853847] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-08 [2025-09-16T19:06:56.181706] [DEBUG] [VALIDATOR ] KKR: All expected trading days are present. [2025-09-16T19:06:56.181845] [DEBUG] [VALIDATOR ] KKR: Primary data validation passed. [2025-09-16T19:06:56.182942] [INFO ] [MAIN ] KKR: Fetched primary data is valid. [2025-09-16T19:06:56.183006] [DEBUG] [LOCAL ] KKR: Saving primary data. [2025-09-16T19:06:56.185085] [DEBUG] [LOCAL ] KKR: Trimmed 58194 records to 39871 (keeping 150 days of data) [2025-09-16T19:06:56.375704] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-04. [2025-09-16T19:06:56.458978] [DEBUG] [LOCAL ] KKR: Primary data saved (39871 records). [2025-09-16T19:06:56.459212] [INFO ] [MAIN ] KKR: Full history saved. [2025-09-16T19:06:56.459357] [INFO ] [MAIN ] Update Progress: 1197/2369 symbols completed. KKR took 31.3s. Estimated remaining: 2h 4m 14s. Rate: 9.4 symbols/min [2025-09-16T19:06:57.174959] [INFO ] [REMOTE ] KLG: Fetching primary data for 2025-05. [2025-09-16T19:06:57.848236] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T19:06:57.974915] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-05. [2025-09-16T19:06:58.780038] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-09. [2025-09-16T19:06:58.799022] [INFO ] [REMOTE ] KLG: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T19:06:59.361995] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T19:06:59.578607] [INFO ] [REMOTE ] KLG: Fetching primary data for 2025-06. [2025-09-16T19:06:59.908410] [INFO ] [REMOTE ] KLAR: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T19:06:59.911318] [INFO ] [REMOTE ] KLAR: Full fetch complete (1733 records). [2025-09-16T19:06:59.911386] [DEBUG] [VALIDATOR ] KLAR: Validating primary data. [2025-09-16T19:07:00.089886] [WARN ] [VALIDATOR ] KLAR: Validation failed - insufficient historical data. Have 5 unique days, need 150 days [2025-09-16T19:07:00.090078] [FATAL] [MAIN ] KLAR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:07:00.090120] [WARN ] [MAIN ] KLAR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:07:00.090158] [INFO ] [LOCAL ] KLAR: Starting deletion of primary data. [2025-09-16T19:07:00.090214] [DEBUG] [LOCAL ] KLAR: No primary data file found to delete. [2025-09-16T19:07:00.090251] [INFO ] [LOCAL ] KLAR: Starting deletion of derivative data. [2025-09-16T19:07:00.090350] [DEBUG] [LOCAL ] KLAR: No derivative data file found to delete. [2025-09-16T19:07:00.090428] [INFO ] [MAIN ] Update Progress: 1198/2369 symbols completed. KLAR took 30.5s. Estimated remaining: 2h 4m 5s. Rate: 9.4 symbols/min [2025-09-16T19:07:00.374957] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-06. [2025-09-16T19:07:01.207559] [INFO ] [REMOTE ] KLG: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T19:07:01.217425] [INFO ] [REMOTE ] KLG: Fetching primary data for 2025-07. [2025-09-16T19:07:01.446545] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T19:07:01.974947] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-07. [2025-09-16T19:07:03.080119] [INFO ] [REMOTE ] KLG: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-09-16T19:07:03.090752] [INFO ] [REMOTE ] KLG: Fetching primary data for 2025-08. [2025-09-16T19:07:03.151407] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-09-16T19:07:03.574907] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-08. [2025-09-16T19:07:04.626188] [INFO ] [REMOTE ] KLG: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T19:07:04.635236] [INFO ] [REMOTE ] KLG: Fetching primary data for 2025-09. [2025-09-16T19:07:04.744197] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T19:07:05.174917] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-09. [2025-09-16T19:07:05.961458] [INFO ] [REMOTE ] KLG: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-09-16T19:07:05.967066] [INFO ] [REMOTE ] KLG: Full fetch complete (45728 records). [2025-09-16T19:07:05.967230] [DEBUG] [VALIDATOR ] KLG: Validating primary data. [2025-09-16T19:07:06.369606] [DEBUG] [VALIDATOR ] KLG: All expected trading days are present. [2025-09-16T19:07:06.369723] [DEBUG] [VALIDATOR ] KLG: Primary data validation passed. [2025-09-16T19:07:06.370413] [INFO ] [MAIN ] KLG: Fetched primary data is valid. [2025-09-16T19:07:06.370466] [DEBUG] [LOCAL ] KLG: Saving primary data. [2025-09-16T19:07:06.371439] [DEBUG] [LOCAL ] KLG: Trimmed 45728 records to 31005 (keeping 150 days of data) [2025-09-16T19:07:06.573454] [DEBUG] [LOCAL ] KLG: Primary data saved (31005 records). [2025-09-16T19:07:06.573708] [INFO ] [MAIN ] KLG: Full history saved. [2025-09-16T19:07:06.573862] [INFO ] [MAIN ] Update Progress: 1199/2369 symbols completed. KLG took 20.5s. Estimated remaining: 2h 3m 59s. Rate: 9.4 symbols/min [2025-09-16T19:07:06.586614] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-09-16T19:07:06.591527] [INFO ] [REMOTE ] KLC: Full fetch complete (38552 records). [2025-09-16T19:07:06.591679] [DEBUG] [VALIDATOR ] KLC: Validating primary data. [2025-09-16T19:07:07.021385] [DEBUG] [VALIDATOR ] KLC: All expected trading days are present. [2025-09-16T19:07:07.021472] [DEBUG] [VALIDATOR ] KLC: Primary data validation passed. [2025-09-16T19:07:07.022199] [INFO ] [MAIN ] KLC: Fetched primary data is valid. [2025-09-16T19:07:07.022245] [DEBUG] [LOCAL ] KLC: Saving primary data. [2025-09-16T19:07:07.023257] [DEBUG] [LOCAL ] KLC: Trimmed 38552 records to 26417 (keeping 150 days of data) [2025-09-16T19:07:07.176606] [DEBUG] [LOCAL ] KLC: Primary data saved (26417 records). [2025-09-16T19:07:07.176758] [INFO ] [MAIN ] KLC: Full history saved. [2025-09-16T19:07:07.176845] [INFO ] [MAIN ] Update Progress: 1200/2369 symbols completed. KLC took 24.0s. Estimated remaining: 2h 3m 47s. Rate: 9.4 symbols/min [2025-09-16T19:07:07.749775] [INFO ] [MAIN ] KMPB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:07.857312] [INFO ] [MAIN ] KMT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:07.879160] [INFO ] [MAIN ] KMI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:07.913397] [INFO ] [MAIN ] KMPR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:07.916073] [DEBUG] [LOCAL ] KMPB: Loaded 2557 existing records from primary storage. [2025-09-16T19:07:07.916786] [INFO ] [MAIN ] KMPB: Loaded 2557 existing records. [2025-09-16T19:07:08.175589] [WARN ] [MAIN ] KMPB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:08.175994] [INFO ] [MAIN ] KMPB: Wiping existing primary data before full refresh. [2025-09-16T19:07:08.177890] [DEBUG] [LOCAL ] KMPR: Loaded 29002 existing records from primary storage. [2025-09-16T19:07:08.177285] [DEBUG] [LOCAL ] KMI: Loaded 39653 existing records from primary storage. [2025-09-16T19:07:08.178087] [INFO ] [LOCAL ] KMPB: Starting deletion of primary data. [2025-09-16T19:07:08.178191] [INFO ] [MAIN ] KMPR: Loaded 29002 existing records. [2025-09-16T19:07:08.178484] [INFO ] [MAIN ] KMI: Loaded 39653 existing records. [2025-09-16T19:07:08.179856] [DEBUG] [LOCAL ] KMT: Loaded 31670 existing records from primary storage. [2025-09-16T19:07:08.206145] [INFO ] [MAIN ] KMT: Loaded 31670 existing records. [2025-09-16T19:07:08.196288] [DEBUG] [LOCAL ] KMPB: Deleted primary data file. [2025-09-16T19:07:08.251429] [INFO ] [MAIN ] KMPB: Starting full primary refresh. [2025-09-16T19:07:08.856979] [WARN ] [MAIN ] KMPR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:08.902928] [WARN ] [MAIN ] KMI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:08.924068] [WARN ] [MAIN ] KMT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:08.924546] [INFO ] [MAIN ] KMPR: Wiping existing primary data before full refresh. [2025-09-16T19:07:08.926423] [INFO ] [MAIN ] KMT: Wiping existing primary data before full refresh. [2025-09-16T19:07:08.951245] [INFO ] [MAIN ] KMI: Wiping existing primary data before full refresh. [2025-09-16T19:07:08.951165] [INFO ] [LOCAL ] KMPR: Starting deletion of primary data. [2025-09-16T19:07:08.979963] [INFO ] [LOCAL ] KMI: Starting deletion of primary data. [2025-09-16T19:07:08.996378] [DEBUG] [LOCAL ] KMI: Deleted primary data file. [2025-09-16T19:07:08.996520] [INFO ] [MAIN ] KMI: Starting full primary refresh. [2025-09-16T19:07:08.979853] [INFO ] [LOCAL ] KMT: Starting deletion of primary data. [2025-09-16T19:07:09.005800] [DEBUG] [LOCAL ] KMPR: Deleted primary data file. [2025-09-16T19:07:09.057116] [INFO ] [MAIN ] KMPR: Starting full primary refresh. [2025-09-16T19:07:09.077289] [DEBUG] [LOCAL ] KMT: Deleted primary data file. [2025-09-16T19:07:09.136332] [INFO ] [MAIN ] KMT: Starting full primary refresh. [2025-09-16T19:07:09.381030] [INFO ] [REMOTE ] KMPB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:09.434920] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-02. [2025-09-16T19:07:10.091580] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-09-16T19:07:10.134331] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-03. [2025-09-16T19:07:10.108666] [INFO ] [REMOTE ] KMPR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:10.153610] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-02. [2025-09-16T19:07:10.222163] [INFO ] [REMOTE ] KMI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:10.226290] [INFO ] [REMOTE ] KMT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:10.227357] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-02. [2025-09-16T19:07:10.227620] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-02. [2025-09-16T19:07:10.773474] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-09-16T19:07:10.775120] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-04. [2025-09-16T19:07:11.460249] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T19:07:11.468936] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-03. [2025-09-16T19:07:11.480285] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-09-16T19:07:11.574934] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-05. [2025-09-16T19:07:11.783606] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T19:07:12.041148] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-09-16T19:07:12.172787] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-09-16T19:07:12.374908] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-03. [2025-09-16T19:07:12.828919] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T19:07:13.174954] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-03. [2025-09-16T19:07:13.974914] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-04. [2025-09-16T19:07:14.356573] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-09-16T19:07:14.774908] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-06. [2025-09-16T19:07:14.883711] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T19:07:15.117118] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-09-16T19:07:15.353180] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-09-16T19:07:15.574926] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-07. [2025-09-16T19:07:16.019909] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-09-16T19:07:16.374907] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-04. [2025-09-16T19:07:17.174947] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-05. [2025-09-16T19:07:17.760818] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T19:07:17.974877] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-04. [2025-09-16T19:07:18.447131] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-09-16T19:07:18.774919] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-08. [2025-09-16T19:07:19.401703] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-09-16T19:07:19.574872] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-05. [2025-09-16T19:07:19.970794] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-09-16T19:07:20.374906] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-05. [2025-09-16T19:07:21.116137] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T19:07:21.174861] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-06. [2025-09-16T19:07:21.974930] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-06. [2025-09-16T19:07:22.283094] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-09-16T19:07:22.684517] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-09-16T19:07:22.774945] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-07. [2025-09-16T19:07:23.389288] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T19:07:23.574927] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-07. [2025-09-16T19:07:23.967933] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-09-16T19:07:24.374892] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-08. [2025-09-16T19:07:24.976062] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-09-16T19:07:25.174974] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-06. [2025-09-16T19:07:25.948817] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T19:07:25.974970] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-09. [2025-09-16T19:07:26.774947] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-08. [2025-09-16T19:07:27.023887] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-09-16T19:07:27.080899] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T19:07:27.086304] [INFO ] [REMOTE ] KMT: Full fetch complete (46907 records). [2025-09-16T19:07:27.086458] [DEBUG] [VALIDATOR ] KMT: Validating primary data. [2025-09-16T19:07:27.465521] [DEBUG] [VALIDATOR ] KMT: All expected trading days are present. [2025-09-16T19:07:27.465606] [DEBUG] [VALIDATOR ] KMT: Primary data validation passed. [2025-09-16T19:07:27.466375] [INFO ] [MAIN ] KMT: Fetched primary data is valid. [2025-09-16T19:07:27.466432] [DEBUG] [LOCAL ] KMT: Saving primary data. [2025-09-16T19:07:27.467536] [DEBUG] [LOCAL ] KMT: Trimmed 46907 records to 32017 (keeping 150 days of data) [2025-09-16T19:07:27.576839] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-09. [2025-09-16T19:07:27.670278] [DEBUG] [LOCAL ] KMT: Primary data saved (32017 records). [2025-09-16T19:07:27.670470] [INFO ] [MAIN ] KMT: Full history saved. [2025-09-16T19:07:27.671536] [INFO ] [MAIN ] Update Progress: 1201/2369 symbols completed. KMT took 20.5s. Estimated remaining: 2h 3m 55s. Rate: 9.4 symbols/min [2025-09-16T19:07:27.840148] [INFO ] [MAIN ] KMX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:27.881484] [DEBUG] [LOCAL ] KMX: Loaded 39282 existing records from primary storage. [2025-09-16T19:07:27.881627] [INFO ] [MAIN ] KMX: Loaded 39282 existing records. [2025-09-16T19:07:28.058750] [WARN ] [MAIN ] KMX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:28.061891] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-09-16T19:07:28.061992] [INFO ] [MAIN ] KMX: Wiping existing primary data before full refresh. [2025-09-16T19:07:28.063069] [INFO ] [REMOTE ] KMPB: Full fetch complete (3858 records). [2025-09-16T19:07:28.063131] [INFO ] [LOCAL ] KMX: Starting deletion of primary data. [2025-09-16T19:07:28.063288] [DEBUG] [VALIDATOR ] KMPB: Validating primary data. [2025-09-16T19:07:28.095286] [DEBUG] [LOCAL ] KMX: Deleted primary data file. [2025-09-16T19:07:28.127887] [INFO ] [MAIN ] KMX: Starting full primary refresh. [2025-09-16T19:07:28.379079] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-07. [2025-09-16T19:07:28.728823] [DEBUG] [VALIDATOR ] KMPB: All expected trading days are present. [2025-09-16T19:07:28.763637] [DEBUG] [VALIDATOR ] KMPB: Primary data validation passed. [2025-09-16T19:07:28.767943] [INFO ] [MAIN ] KMPB: Fetched primary data is valid. [2025-09-16T19:07:28.773410] [DEBUG] [LOCAL ] KMPB: Saving primary data. [2025-09-16T19:07:28.800727] [INFO ] [REMOTE ] KMX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:28.801599] [DEBUG] [LOCAL ] KMPB: Trimmed 3858 records to 2577 (keeping 150 days of data) [2025-09-16T19:07:28.818667] [DEBUG] [LOCAL ] KMPB: Primary data saved (2577 records). [2025-09-16T19:07:28.818851] [INFO ] [MAIN ] KMPB: Full history saved. [2025-09-16T19:07:28.819022] [INFO ] [MAIN ] Update Progress: 1202/2369 symbols completed. KMPB took 21.6s. Estimated remaining: 2h 3m 43s. Rate: 9.4 symbols/min [2025-09-16T19:07:28.953107] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-09-16T19:07:29.033507] [INFO ] [MAIN ] KN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:29.061180] [DEBUG] [LOCAL ] KN: Loaded 27770 existing records from primary storage. [2025-09-16T19:07:29.061348] [INFO ] [MAIN ] KN: Loaded 27770 existing records. [2025-09-16T19:07:29.180019] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-02. [2025-09-16T19:07:29.252877] [WARN ] [MAIN ] KN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:29.253149] [INFO ] [MAIN ] KN: Wiping existing primary data before full refresh. [2025-09-16T19:07:29.253199] [INFO ] [LOCAL ] KN: Starting deletion of primary data. [2025-09-16T19:07:29.253867] [DEBUG] [LOCAL ] KN: Deleted primary data file. [2025-09-16T19:07:29.253925] [INFO ] [MAIN ] KN: Starting full primary refresh. [2025-09-16T19:07:29.614339] [INFO ] [REMOTE ] KN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:29.974947] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-02. [2025-09-16T19:07:30.668722] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-07 (2.29 seconds). [2025-09-16T19:07:30.774857] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-08. [2025-09-16T19:07:31.528079] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T19:07:31.530086] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-09-16T19:07:31.574847] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-03. [2025-09-16T19:07:32.374917] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-03. [2025-09-16T19:07:32.777663] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-09-16T19:07:33.022331] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T19:07:33.174910] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-09. [2025-09-16T19:07:33.974900] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-04. [2025-09-16T19:07:34.121680] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T19:07:34.665996] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T19:07:34.672832] [INFO ] [REMOTE ] KMPR: Full fetch complete (39894 records). [2025-09-16T19:07:34.672963] [DEBUG] [VALIDATOR ] KMPR: Validating primary data. [2025-09-16T19:07:34.776716] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-09. [2025-09-16T19:07:35.052280] [DEBUG] [VALIDATOR ] KMPR: All expected trading days are present. [2025-09-16T19:07:35.052365] [DEBUG] [VALIDATOR ] KMPR: Primary data validation passed. [2025-09-16T19:07:35.053074] [INFO ] [MAIN ] KMPR: Fetched primary data is valid. [2025-09-16T19:07:35.053118] [DEBUG] [LOCAL ] KMPR: Saving primary data. [2025-09-16T19:07:35.054051] [DEBUG] [LOCAL ] KMPR: Trimmed 39894 records to 29360 (keeping 150 days of data) [2025-09-16T19:07:35.262326] [DEBUG] [LOCAL ] KMPR: Primary data saved (29360 records). [2025-09-16T19:07:35.262908] [INFO ] [MAIN ] KMPR: Full history saved. [2025-09-16T19:07:35.264077] [INFO ] [MAIN ] Update Progress: 1203/2369 symbols completed. KMPR took 28.1s. Estimated remaining: 2h 3m 37s. Rate: 9.4 symbols/min [2025-09-16T19:07:35.263932] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-09-16T19:07:35.449365] [INFO ] [MAIN ] KNF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:35.476030] [DEBUG] [LOCAL ] KNF: Loaded 26473 existing records from primary storage. [2025-09-16T19:07:35.476169] [INFO ] [MAIN ] KNF: Loaded 26473 existing records. [2025-09-16T19:07:35.577088] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-04. [2025-09-16T19:07:35.656908] [WARN ] [MAIN ] KNF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:35.657168] [INFO ] [MAIN ] KNF: Wiping existing primary data before full refresh. [2025-09-16T19:07:35.657230] [INFO ] [LOCAL ] KNF: Starting deletion of primary data. [2025-09-16T19:07:35.657905] [DEBUG] [LOCAL ] KNF: Deleted primary data file. [2025-09-16T19:07:35.657968] [INFO ] [MAIN ] KNF: Starting full primary refresh. [2025-09-16T19:07:36.025656] [INFO ] [REMOTE ] KNF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:36.374874] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-05. [2025-09-16T19:07:36.791326] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-09-16T19:07:36.798000] [INFO ] [REMOTE ] KMI: Full fetch complete (58373 records). [2025-09-16T19:07:36.798144] [DEBUG] [VALIDATOR ] KMI: Validating primary data. [2025-09-16T19:07:37.185422] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-02. [2025-09-16T19:07:37.188323] [DEBUG] [VALIDATOR ] KMI: All expected trading days are present. [2025-09-16T19:07:37.189628] [DEBUG] [VALIDATOR ] KMI: Primary data validation passed. [2025-09-16T19:07:37.190575] [INFO ] [MAIN ] KMI: Fetched primary data is valid. [2025-09-16T19:07:37.190758] [DEBUG] [LOCAL ] KMI: Saving primary data. [2025-09-16T19:07:37.192208] [DEBUG] [LOCAL ] KMI: Trimmed 58373 records to 40043 (keeping 150 days of data) [2025-09-16T19:07:37.448357] [DEBUG] [LOCAL ] KMI: Primary data saved (40043 records). [2025-09-16T19:07:37.448490] [INFO ] [MAIN ] KMI: Full history saved. [2025-09-16T19:07:37.448584] [INFO ] [MAIN ] Update Progress: 1204/2369 symbols completed. KMI took 30.3s. Estimated remaining: 2h 3m 26s. Rate: 9.4 symbols/min [2025-09-16T19:07:37.622255] [INFO ] [MAIN ] KNOP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:37.655772] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-09-16T19:07:37.659202] [DEBUG] [LOCAL ] KNOP: Loaded 6609 existing records from primary storage. [2025-09-16T19:07:37.665322] [INFO ] [MAIN ] KNOP: Loaded 6609 existing records. [2025-09-16T19:07:37.799314] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-04 (2.22 seconds). [2025-09-16T19:07:37.886436] [WARN ] [MAIN ] KNOP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:37.886715] [INFO ] [MAIN ] KNOP: Wiping existing primary data before full refresh. [2025-09-16T19:07:37.886765] [INFO ] [LOCAL ] KNOP: Starting deletion of primary data. [2025-09-16T19:07:37.887122] [DEBUG] [LOCAL ] KNOP: Deleted primary data file. [2025-09-16T19:07:37.887172] [INFO ] [MAIN ] KNOP: Starting full primary refresh. [2025-09-16T19:07:37.982902] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-05. [2025-09-16T19:07:38.248171] [INFO ] [REMOTE ] KNOP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:38.267187] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-09-16T19:07:38.774903] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-02. [2025-09-16T19:07:39.530281] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T19:07:39.574854] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-03. [2025-09-16T19:07:39.916280] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-09-16T19:07:40.092292] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-09-16T19:07:40.374884] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-06. [2025-09-16T19:07:41.174935] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-03. [2025-09-16T19:07:41.759560] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-09-16T19:07:41.974912] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-07. [2025-09-16T19:07:42.485814] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-09-16T19:07:42.774944] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-04. [2025-09-16T19:07:43.347632] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T19:07:43.574922] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-08. [2025-09-16T19:07:44.217957] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-09-16T19:07:44.374999] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-04. [2025-09-16T19:07:44.981862] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-09-16T19:07:45.001559] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-09-16T19:07:45.174921] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-09. [2025-09-16T19:07:45.974921] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-05. [2025-09-16T19:07:46.498737] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T19:07:46.503439] [INFO ] [REMOTE ] KN: Full fetch complete (42843 records). [2025-09-16T19:07:46.503563] [DEBUG] [VALIDATOR ] KN: Validating primary data. [2025-09-16T19:07:46.780207] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-05. [2025-09-16T19:07:46.904129] [DEBUG] [VALIDATOR ] KN: All expected trading days are present. [2025-09-16T19:07:46.904225] [DEBUG] [VALIDATOR ] KN: Primary data validation passed. [2025-09-16T19:07:46.905130] [INFO ] [MAIN ] KN: Fetched primary data is valid. [2025-09-16T19:07:46.905196] [DEBUG] [LOCAL ] KN: Saving primary data. [2025-09-16T19:07:46.906129] [DEBUG] [LOCAL ] KN: Trimmed 42843 records to 28017 (keeping 150 days of data) [2025-09-16T19:07:47.077466] [DEBUG] [LOCAL ] KN: Primary data saved (28017 records). [2025-09-16T19:07:47.077706] [INFO ] [MAIN ] KN: Full history saved. [2025-09-16T19:07:47.077860] [INFO ] [MAIN ] Update Progress: 1205/2369 symbols completed. KN took 18.3s. Estimated remaining: 2h 3m 23s. Rate: 9.4 symbols/min [2025-09-16T19:07:47.247935] [INFO ] [MAIN ] KNSL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:47.267373] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-09-16T19:07:47.286610] [DEBUG] [LOCAL ] KNSL: Loaded 13501 existing records from primary storage. [2025-09-16T19:07:47.286760] [INFO ] [MAIN ] KNSL: Loaded 13501 existing records. [2025-09-16T19:07:47.466539] [WARN ] [MAIN ] KNSL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:47.466841] [INFO ] [MAIN ] KNSL: Wiping existing primary data before full refresh. [2025-09-16T19:07:47.466889] [INFO ] [LOCAL ] KNSL: Starting deletion of primary data. [2025-09-16T19:07:47.467382] [DEBUG] [LOCAL ] KNSL: Deleted primary data file. [2025-09-16T19:07:47.467440] [INFO ] [MAIN ] KNSL: Starting full primary refresh. [2025-09-16T19:07:47.580095] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-06. [2025-09-16T19:07:47.806501] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-09-16T19:07:47.832606] [INFO ] [REMOTE ] KNSL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:48.374973] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-06. [2025-09-16T19:07:49.174963] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-06. [2025-09-16T19:07:49.375616] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-09-16T19:07:49.825132] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T19:07:49.974839] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-09-16T19:07:49.974937] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-07. [2025-09-16T19:07:50.774917] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-07. [2025-09-16T19:07:51.339387] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-09-16T19:07:51.574905] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-07. [2025-09-16T19:07:51.627219] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-09-16T19:07:52.374925] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-08. [2025-09-16T19:07:53.174916] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-02. [2025-09-16T19:07:53.678906] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-09-16T19:07:53.791580] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-09-16T19:07:53.974879] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-09. [2025-09-16T19:07:54.313200] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-09-16T19:07:54.774916] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-08. [2025-09-16T19:07:55.058365] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T19:07:55.063276] [INFO ] [REMOTE ] KNF: Full fetch complete (38670 records). [2025-09-16T19:07:55.063374] [DEBUG] [VALIDATOR ] KNF: Validating primary data. [2025-09-16T19:07:55.434888] [DEBUG] [VALIDATOR ] KNF: All expected trading days are present. [2025-09-16T19:07:55.434968] [DEBUG] [VALIDATOR ] KNF: Primary data validation passed. [2025-09-16T19:07:55.435684] [INFO ] [MAIN ] KNF: Fetched primary data is valid. [2025-09-16T19:07:55.435731] [DEBUG] [LOCAL ] KNF: Saving primary data. [2025-09-16T19:07:55.436607] [DEBUG] [LOCAL ] KNF: Trimmed 38670 records to 26715 (keeping 150 days of data) [2025-09-16T19:07:55.583547] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-03. [2025-09-16T19:07:55.589395] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-09-16T19:07:55.613365] [DEBUG] [LOCAL ] KNF: Primary data saved (26715 records). [2025-09-16T19:07:55.613501] [INFO ] [MAIN ] KNF: Full history saved. [2025-09-16T19:07:55.613592] [INFO ] [MAIN ] Update Progress: 1206/2369 symbols completed. KNF took 20.3s. Estimated remaining: 2h 3m 19s. Rate: 9.4 symbols/min [2025-09-16T19:07:55.785374] [INFO ] [MAIN ] KNTK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:07:55.820882] [DEBUG] [LOCAL ] KNTK: Loaded 34724 existing records from primary storage. [2025-09-16T19:07:55.821027] [INFO ] [MAIN ] KNTK: Loaded 34724 existing records. [2025-09-16T19:07:55.997189] [WARN ] [MAIN ] KNTK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:07:55.997438] [INFO ] [MAIN ] KNTK: Wiping existing primary data before full refresh. [2025-09-16T19:07:55.997503] [INFO ] [LOCAL ] KNTK: Starting deletion of primary data. [2025-09-16T19:07:55.998360] [DEBUG] [LOCAL ] KNTK: Deleted primary data file. [2025-09-16T19:07:55.998409] [INFO ] [MAIN ] KNTK: Starting full primary refresh. [2025-09-16T19:07:56.349866] [INFO ] [REMOTE ] KNTK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:07:56.374878] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-08. [2025-09-16T19:07:56.587542] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-09-16T19:07:57.174913] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-02. [2025-09-16T19:07:57.974902] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-04. [2025-09-16T19:07:58.289697] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-09-16T19:07:58.594357] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T19:07:58.774903] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-09. [2025-09-16T19:07:58.812040] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-09-16T19:07:59.550969] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-09-16T19:07:59.552770] [INFO ] [REMOTE ] KNOP: Full fetch complete (10644 records). [2025-09-16T19:07:59.552838] [DEBUG] [VALIDATOR ] KNOP: Validating primary data. [2025-09-16T19:07:59.582056] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-05. [2025-09-16T19:07:59.916891] [DEBUG] [VALIDATOR ] KNOP: All expected trading days are present. [2025-09-16T19:07:59.916990] [DEBUG] [VALIDATOR ] KNOP: Primary data validation passed. [2025-09-16T19:07:59.917253] [INFO ] [MAIN ] KNOP: Fetched primary data is valid. [2025-09-16T19:07:59.917295] [DEBUG] [LOCAL ] KNOP: Saving primary data. [2025-09-16T19:07:59.917764] [DEBUG] [LOCAL ] KNOP: Trimmed 10644 records to 6647 (keeping 150 days of data) [2025-09-16T19:07:59.958831] [DEBUG] [LOCAL ] KNOP: Primary data saved (6647 records). [2025-09-16T19:07:59.958943] [INFO ] [MAIN ] KNOP: Full history saved. [2025-09-16T19:07:59.959042] [INFO ] [MAIN ] Update Progress: 1207/2369 symbols completed. KNOP took 22.5s. Estimated remaining: 2h 3m 11s. Rate: 9.4 symbols/min [2025-09-16T19:08:00.129072] [INFO ] [MAIN ] KNX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:00.171559] [DEBUG] [LOCAL ] KNX: Loaded 38684 existing records from primary storage. [2025-09-16T19:08:00.171720] [INFO ] [MAIN ] KNX: Loaded 38684 existing records. [2025-09-16T19:08:00.349669] [WARN ] [MAIN ] KNX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:00.349925] [INFO ] [MAIN ] KNX: Wiping existing primary data before full refresh. [2025-09-16T19:08:00.349970] [INFO ] [LOCAL ] KNX: Starting deletion of primary data. [2025-09-16T19:08:00.350813] [DEBUG] [LOCAL ] KNX: Deleted primary data file. [2025-09-16T19:08:00.350862] [INFO ] [MAIN ] KNX: Starting full primary refresh. [2025-09-16T19:08:00.381230] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-09. [2025-09-16T19:08:00.705725] [INFO ] [REMOTE ] KNX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:00.727800] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T19:08:01.174905] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-02. [2025-09-16T19:08:01.974919] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-06. [2025-09-16T19:08:02.226061] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-09-16T19:08:02.234224] [INFO ] [REMOTE ] KMX: Full fetch complete (57412 records). [2025-09-16T19:08:02.234393] [DEBUG] [VALIDATOR ] KMX: Validating primary data. [2025-09-16T19:08:02.634734] [DEBUG] [VALIDATOR ] KMX: All expected trading days are present. [2025-09-16T19:08:02.634815] [DEBUG] [VALIDATOR ] KMX: Primary data validation passed. [2025-09-16T19:08:02.635720] [INFO ] [MAIN ] KMX: Fetched primary data is valid. [2025-09-16T19:08:02.635778] [DEBUG] [LOCAL ] KMX: Saving primary data. [2025-09-16T19:08:02.637072] [DEBUG] [LOCAL ] KMX: Trimmed 57412 records to 39668 (keeping 150 days of data) [2025-09-16T19:08:02.812622] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-03. [2025-09-16T19:08:02.899251] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-09-16T19:08:02.922040] [DEBUG] [LOCAL ] KMX: Primary data saved (39668 records). [2025-09-16T19:08:02.922214] [INFO ] [MAIN ] KMX: Full history saved. [2025-09-16T19:08:02.923799] [INFO ] [MAIN ] Update Progress: 1208/2369 symbols completed. KMX took 35.3s. Estimated remaining: 2h 3m 1s. Rate: 9.4 symbols/min [2025-09-16T19:08:03.093554] [INFO ] [MAIN ] KO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:03.104762] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-09-16T19:08:03.147025] [DEBUG] [LOCAL ] KO: Loaded 39778 existing records from primary storage. [2025-09-16T19:08:03.147217] [INFO ] [MAIN ] KO: Loaded 39778 existing records. [2025-09-16T19:08:03.320110] [WARN ] [MAIN ] KO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:03.320375] [INFO ] [MAIN ] KO: Wiping existing primary data before full refresh. [2025-09-16T19:08:03.320421] [INFO ] [LOCAL ] KO: Starting deletion of primary data. [2025-09-16T19:08:03.321273] [DEBUG] [LOCAL ] KO: Deleted primary data file. [2025-09-16T19:08:03.321321] [INFO ] [MAIN ] KO: Starting full primary refresh. [2025-09-16T19:08:03.580023] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-07. [2025-09-16T19:08:03.681692] [INFO ] [REMOTE ] KO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:04.224411] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-09-16T19:08:04.374906] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-04. [2025-09-16T19:08:04.914823] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-09-16T19:08:05.174931] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-02. [2025-09-16T19:08:05.946076] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-09-16T19:08:05.974840] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-05. [2025-09-16T19:08:06.774900] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-03. [2025-09-16T19:08:07.574895] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-08. [2025-09-16T19:08:07.630870] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T19:08:07.681746] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-02 (2.51 seconds). [2025-09-16T19:08:08.374951] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-03. [2025-09-16T19:08:08.615089] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T19:08:08.637809] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-09-16T19:08:09.174925] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-09. [2025-09-16T19:08:09.974946] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-04. [2025-09-16T19:08:10.030707] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T19:08:10.033309] [INFO ] [REMOTE ] KNSL: Full fetch complete (19973 records). [2025-09-16T19:08:10.033412] [DEBUG] [VALIDATOR ] KNSL: Validating primary data. [2025-09-16T19:08:10.399357] [DEBUG] [VALIDATOR ] KNSL: All expected trading days are present. [2025-09-16T19:08:10.399436] [DEBUG] [VALIDATOR ] KNSL: Primary data validation passed. [2025-09-16T19:08:10.399742] [INFO ] [MAIN ] KNSL: Fetched primary data is valid. [2025-09-16T19:08:10.399785] [DEBUG] [LOCAL ] KNSL: Saving primary data. [2025-09-16T19:08:10.400313] [DEBUG] [LOCAL ] KNSL: Trimmed 19973 records to 13618 (keeping 150 days of data) [2025-09-16T19:08:10.488998] [DEBUG] [LOCAL ] KNSL: Primary data saved (13618 records). [2025-09-16T19:08:10.489139] [INFO ] [MAIN ] KNSL: Full history saved. [2025-09-16T19:08:10.489226] [INFO ] [MAIN ] Update Progress: 1209/2369 symbols completed. KNSL took 23.4s. Estimated remaining: 2h 2m 56s. Rate: 9.4 symbols/min [2025-09-16T19:08:10.658016] [INFO ] [MAIN ] KODK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:10.692007] [DEBUG] [LOCAL ] KODK: Loaded 31957 existing records from primary storage. [2025-09-16T19:08:10.692176] [INFO ] [MAIN ] KODK: Loaded 31957 existing records. [2025-09-16T19:08:10.784159] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-06. [2025-09-16T19:08:10.876225] [WARN ] [MAIN ] KODK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:10.876524] [INFO ] [MAIN ] KODK: Wiping existing primary data before full refresh. [2025-09-16T19:08:10.876672] [INFO ] [LOCAL ] KODK: Starting deletion of primary data. [2025-09-16T19:08:10.877505] [DEBUG] [LOCAL ] KODK: Deleted primary data file. [2025-09-16T19:08:10.877568] [INFO ] [MAIN ] KODK: Starting full primary refresh. [2025-09-16T19:08:11.230054] [INFO ] [REMOTE ] KODK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:11.260547] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-03 (2.89 seconds). [2025-09-16T19:08:11.574914] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-02. [2025-09-16T19:08:11.815303] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T19:08:12.374879] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-05. [2025-09-16T19:08:12.444717] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T19:08:13.103960] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T19:08:13.174867] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-03. [2025-09-16T19:08:13.974915] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-04. [2025-09-16T19:08:14.143014] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-09-16T19:08:14.776864] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-07. [2025-09-16T19:08:14.782124] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T19:08:15.574943] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-04. [2025-09-16T19:08:16.374902] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-06. [2025-09-16T19:08:16.573905] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-09-16T19:08:16.647014] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-04 (2.67 seconds). [2025-09-16T19:08:17.174955] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-08. [2025-09-16T19:08:17.238913] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-09-16T19:08:17.974982] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-05. [2025-09-16T19:08:18.039199] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T19:08:18.753939] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T19:08:18.774875] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-09. [2025-09-16T19:08:19.574934] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-05. [2025-09-16T19:08:20.284203] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-09-16T19:08:20.289881] [INFO ] [REMOTE ] KNTK: Full fetch complete (49790 records). [2025-09-16T19:08:20.290008] [DEBUG] [VALIDATOR ] KNTK: Validating primary data. [2025-09-16T19:08:20.379994] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-07. [2025-09-16T19:08:20.678797] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-05 (2.70 seconds). [2025-09-16T19:08:20.719469] [DEBUG] [VALIDATOR ] KNTK: All expected trading days are present. [2025-09-16T19:08:20.719622] [DEBUG] [VALIDATOR ] KNTK: Primary data validation passed. [2025-09-16T19:08:20.720380] [INFO ] [MAIN ] KNTK: Fetched primary data is valid. [2025-09-16T19:08:20.720435] [DEBUG] [LOCAL ] KNTK: Saving primary data. [2025-09-16T19:08:20.721583] [DEBUG] [LOCAL ] KNTK: Trimmed 49790 records to 35014 (keeping 150 days of data) [2025-09-16T19:08:20.955292] [DEBUG] [LOCAL ] KNTK: Primary data saved (35014 records). [2025-09-16T19:08:20.955458] [INFO ] [MAIN ] KNTK: Full history saved. [2025-09-16T19:08:20.955572] [INFO ] [MAIN ] Update Progress: 1210/2369 symbols completed. KNTK took 25.3s. Estimated remaining: 2h 2m 53s. Rate: 9.4 symbols/min [2025-09-16T19:08:21.126332] [INFO ] [MAIN ] KOF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:21.144825] [DEBUG] [LOCAL ] KOF: Loaded 19529 existing records from primary storage. [2025-09-16T19:08:21.144988] [INFO ] [MAIN ] KOF: Loaded 19529 existing records. [2025-09-16T19:08:21.177007] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-06. [2025-09-16T19:08:21.326855] [WARN ] [MAIN ] KOF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:21.327131] [INFO ] [MAIN ] KOF: Wiping existing primary data before full refresh. [2025-09-16T19:08:21.327181] [INFO ] [LOCAL ] KOF: Starting deletion of primary data. [2025-09-16T19:08:21.327778] [DEBUG] [LOCAL ] KOF: Deleted primary data file. [2025-09-16T19:08:21.327892] [INFO ] [MAIN ] KOF: Starting full primary refresh. [2025-09-16T19:08:21.523760] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-09-16T19:08:21.755770] [INFO ] [REMOTE ] KOF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:21.974886] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-06. [2025-09-16T19:08:22.180406] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-09-16T19:08:22.774930] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-02. [2025-09-16T19:08:23.437148] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T19:08:23.475593] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-06 (2.30 seconds). [2025-09-16T19:08:23.574891] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-07. [2025-09-16T19:08:23.962849] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T19:08:24.374902] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-07. [2025-09-16T19:08:25.174934] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-08. [2025-09-16T19:08:25.974924] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-03. [2025-09-16T19:08:26.089859] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T19:08:26.278662] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-07 (2.70 seconds). [2025-09-16T19:08:26.387745] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-09-16T19:08:26.774888] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-08. [2025-09-16T19:08:27.058827] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T19:08:27.574906] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-04. [2025-09-16T19:08:28.374914] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-09. [2025-09-16T19:08:28.881540] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-09-16T19:08:29.174901] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-08. [2025-09-16T19:08:29.362563] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-08 (2.59 seconds). [2025-09-16T19:08:29.620564] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-09-16T19:08:29.627923] [INFO ] [REMOTE ] KNX: Full fetch complete (57177 records). [2025-09-16T19:08:29.628131] [DEBUG] [VALIDATOR ] KNX: Validating primary data. [2025-09-16T19:08:29.980505] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-09. [2025-09-16T19:08:30.022190] [DEBUG] [VALIDATOR ] KNX: All expected trading days are present. [2025-09-16T19:08:30.022284] [DEBUG] [VALIDATOR ] KNX: Primary data validation passed. [2025-09-16T19:08:30.023161] [INFO ] [MAIN ] KNX: Fetched primary data is valid. [2025-09-16T19:08:30.023211] [DEBUG] [LOCAL ] KNX: Saving primary data. [2025-09-16T19:08:30.025601] [DEBUG] [LOCAL ] KNX: Trimmed 57177 records to 39070 (keeping 150 days of data) [2025-09-16T19:08:30.270612] [DEBUG] [LOCAL ] KNX: Primary data saved (39070 records). [2025-09-16T19:08:30.270795] [INFO ] [MAIN ] KNX: Full history saved. [2025-09-16T19:08:30.270889] [INFO ] [MAIN ] Update Progress: 1211/2369 symbols completed. KNX took 30.3s. Estimated remaining: 2h 2m 50s. Rate: 9.4 symbols/min [2025-09-16T19:08:30.448143] [INFO ] [MAIN ] KOP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:30.461319] [DEBUG] [LOCAL ] KOP: Loaded 12639 existing records from primary storage. [2025-09-16T19:08:30.461431] [INFO ] [MAIN ] KOP: Loaded 12639 existing records. [2025-09-16T19:08:30.646198] [WARN ] [MAIN ] KOP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:30.646444] [INFO ] [MAIN ] KOP: Wiping existing primary data before full refresh. [2025-09-16T19:08:30.646493] [INFO ] [LOCAL ] KOP: Starting deletion of primary data. [2025-09-16T19:08:30.646962] [DEBUG] [LOCAL ] KOP: Deleted primary data file. [2025-09-16T19:08:30.647012] [INFO ] [MAIN ] KOP: Starting full primary refresh. [2025-09-16T19:08:30.780008] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-05. [2025-09-16T19:08:31.024270] [INFO ] [REMOTE ] KOP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:31.053725] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-09-16T19:08:31.574901] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-09. [2025-09-16T19:08:31.832551] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-09-16T19:08:32.374919] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-06. [2025-09-16T19:08:32.674841] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-09 (2.69 seconds). [2025-09-16T19:08:32.681562] [INFO ] [REMOTE ] KO: Full fetch complete (58498 records). [2025-09-16T19:08:32.681717] [DEBUG] [VALIDATOR ] KO: Validating primary data. [2025-09-16T19:08:33.058938] [DEBUG] [VALIDATOR ] KO: All expected trading days are present. [2025-09-16T19:08:33.059147] [DEBUG] [VALIDATOR ] KO: Primary data validation passed. [2025-09-16T19:08:33.059994] [INFO ] [MAIN ] KO: Fetched primary data is valid. [2025-09-16T19:08:33.060060] [DEBUG] [LOCAL ] KO: Saving primary data. [2025-09-16T19:08:33.073140] [INFO ] [REMOTE ] KODK: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T19:08:33.077821] [DEBUG] [LOCAL ] KO: Trimmed 58498 records to 40168 (keeping 150 days of data) [2025-09-16T19:08:33.078831] [INFO ] [REMOTE ] KODK: Full fetch complete (47193 records). [2025-09-16T19:08:33.078921] [DEBUG] [VALIDATOR ] KODK: Validating primary data. [2025-09-16T19:08:33.181895] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-02. [2025-09-16T19:08:33.569705] [DEBUG] [VALIDATOR ] KODK: All expected trading days are present. [2025-09-16T19:08:33.570343] [DEBUG] [VALIDATOR ] KODK: Primary data validation passed. [2025-09-16T19:08:33.571618] [INFO ] [MAIN ] KODK: Fetched primary data is valid. [2025-09-16T19:08:33.571746] [DEBUG] [LOCAL ] KODK: Saving primary data. [2025-09-16T19:08:33.576092] [DEBUG] [LOCAL ] KODK: Trimmed 47193 records to 32256 (keeping 150 days of data) [2025-09-16T19:08:33.689489] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T19:08:33.925076] [DEBUG] [LOCAL ] KODK: Primary data saved (32256 records). [2025-09-16T19:08:33.929186] [INFO ] [MAIN ] KODK: Full history saved. [2025-09-16T19:08:33.929605] [INFO ] [MAIN ] Update Progress: 1212/2369 symbols completed. KODK took 23.4s. Estimated remaining: 2h 2m 41s. Rate: 9.4 symbols/min [2025-09-16T19:08:33.929463] [DEBUG] [LOCAL ] KO: Primary data saved (40168 records). [2025-09-16T19:08:33.929736] [INFO ] [MAIN ] KO: Full history saved. [2025-09-16T19:08:33.962835] [INFO ] [MAIN ] Update Progress: 1213/2369 symbols completed. KO took 31.0s. Estimated remaining: 2h 2m 29s. Rate: 9.4 symbols/min [2025-09-16T19:08:33.986115] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-07. [2025-09-16T19:08:34.208185] [INFO ] [MAIN ] KORE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:34.280768] [INFO ] [MAIN ] KOS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:34.282757] [DEBUG] [LOCAL ] KORE: Loaded 1525 existing records from primary storage. [2025-09-16T19:08:34.282888] [INFO ] [MAIN ] KORE: Loaded 1525 existing records. [2025-09-16T19:08:34.369413] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T19:08:34.493065] [WARN ] [MAIN ] KORE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:34.494010] [INFO ] [MAIN ] KORE: Wiping existing primary data before full refresh. [2025-09-16T19:08:34.494202] [INFO ] [LOCAL ] KORE: Starting deletion of primary data. [2025-09-16T19:08:34.505775] [DEBUG] [LOCAL ] KORE: Deleted primary data file. [2025-09-16T19:08:34.505735] [DEBUG] [LOCAL ] KOS: Loaded 39184 existing records from primary storage. [2025-09-16T19:08:34.505842] [INFO ] [MAIN ] KORE: Starting full primary refresh. [2025-09-16T19:08:34.537745] [INFO ] [MAIN ] KOS: Loaded 39184 existing records. [2025-09-16T19:08:34.781401] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-03. [2025-09-16T19:08:34.896814] [WARN ] [MAIN ] KOS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:34.958251] [INFO ] [MAIN ] KOS: Wiping existing primary data before full refresh. [2025-09-16T19:08:34.970044] [INFO ] [LOCAL ] KOS: Starting deletion of primary data. [2025-09-16T19:08:34.976163] [DEBUG] [LOCAL ] KOS: Deleted primary data file. [2025-09-16T19:08:34.986413] [INFO ] [MAIN ] KOS: Starting full primary refresh. [2025-09-16T19:08:35.127394] [INFO ] [REMOTE ] KORE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:35.409458] [INFO ] [REMOTE ] KOS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:35.425668] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T19:08:35.574900] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-08. [2025-09-16T19:08:35.932821] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T19:08:36.374906] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-02. [2025-09-16T19:08:36.704316] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T19:08:36.862834] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-09-16T19:08:37.174906] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-03. [2025-09-16T19:08:37.602981] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-09-16T19:08:37.974901] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-09. [2025-09-16T19:08:38.774888] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-04. [2025-09-16T19:08:39.125408] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T19:08:39.129238] [INFO ] [REMOTE ] KOF: Full fetch complete (28660 records). [2025-09-16T19:08:39.129347] [DEBUG] [VALIDATOR ] KOF: Validating primary data. [2025-09-16T19:08:39.502864] [DEBUG] [VALIDATOR ] KOF: All expected trading days are present. [2025-09-16T19:08:39.502980] [DEBUG] [VALIDATOR ] KOF: Primary data validation passed. [2025-09-16T19:08:39.503583] [INFO ] [MAIN ] KOF: Fetched primary data is valid. [2025-09-16T19:08:39.503624] [DEBUG] [LOCAL ] KOF: Saving primary data. [2025-09-16T19:08:39.504352] [DEBUG] [LOCAL ] KOF: Trimmed 28660 records to 19629 (keeping 150 days of data) [2025-09-16T19:08:39.592330] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-02. [2025-09-16T19:08:39.630015] [DEBUG] [LOCAL ] KOF: Primary data saved (19629 records). [2025-09-16T19:08:39.630265] [INFO ] [MAIN ] KOF: Full history saved. [2025-09-16T19:08:39.630364] [INFO ] [MAIN ] Update Progress: 1214/2369 symbols completed. KOF took 18.7s. Estimated remaining: 2h 2m 22s. Rate: 9.4 symbols/min [2025-09-16T19:08:39.801870] [INFO ] [MAIN ] KR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:39.819166] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T19:08:39.858922] [DEBUG] [LOCAL ] KR: Loaded 39643 existing records from primary storage. [2025-09-16T19:08:39.859137] [INFO ] [MAIN ] KR: Loaded 39643 existing records. [2025-09-16T19:08:40.033300] [WARN ] [MAIN ] KR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:40.033561] [INFO ] [MAIN ] KR: Wiping existing primary data before full refresh. [2025-09-16T19:08:40.033608] [INFO ] [LOCAL ] KR: Starting deletion of primary data. [2025-09-16T19:08:40.034908] [DEBUG] [LOCAL ] KR: Deleted primary data file. [2025-09-16T19:08:40.034960] [INFO ] [MAIN ] KR: Starting full primary refresh. [2025-09-16T19:08:40.384118] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-04. [2025-09-16T19:08:40.388784] [INFO ] [REMOTE ] KR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:40.846612] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-09-16T19:08:41.180430] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-02. [2025-09-16T19:08:41.202822] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T19:08:41.974971] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-05. [2025-09-16T19:08:42.405681] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-09-16T19:08:42.772950] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T19:08:42.782450] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-03. [2025-09-16T19:08:43.574960] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-06. [2025-09-16T19:08:43.995769] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-09-16T19:08:44.291807] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-09-16T19:08:44.374891] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-05. [2025-09-16T19:08:45.174920] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-04. [2025-09-16T19:08:45.337416] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-09-16T19:08:45.974909] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-03. [2025-09-16T19:08:46.691109] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T19:08:46.774888] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-05. [2025-09-16T19:08:47.574921] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-07. [2025-09-16T19:08:47.713115] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T19:08:48.048428] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-09-16T19:08:48.380021] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-08. [2025-09-16T19:08:48.396882] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T19:08:48.826811] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-09-16T19:08:49.174959] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-06. [2025-09-16T19:08:49.968231] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-09-16T19:08:49.974852] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-07. [2025-09-16T19:08:50.774929] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-04. [2025-09-16T19:08:50.865042] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-09-16T19:08:51.574950] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-08. [2025-09-16T19:08:52.374958] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-09. [2025-09-16T19:08:52.511976] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-09-16T19:08:52.761542] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-09-16T19:08:52.822946] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-09-16T19:08:52.823917] [INFO ] [REMOTE ] KORE: Full fetch complete (2235 records). [2025-09-16T19:08:52.824020] [DEBUG] [VALIDATOR ] KORE: Validating primary data. [2025-09-16T19:08:53.177170] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-05. [2025-09-16T19:08:53.199195] [DEBUG] [VALIDATOR ] KORE: All expected trading days are present. [2025-09-16T19:08:53.199326] [DEBUG] [VALIDATOR ] KORE: Primary data validation passed. [2025-09-16T19:08:53.199469] [INFO ] [MAIN ] KORE: Fetched primary data is valid. [2025-09-16T19:08:53.201033] [DEBUG] [LOCAL ] KORE: Saving primary data. [2025-09-16T19:08:53.202014] [DEBUG] [LOCAL ] KORE: Trimmed 2235 records to 1535 (keeping 150 days of data) [2025-09-16T19:08:53.213012] [DEBUG] [LOCAL ] KORE: Primary data saved (1535 records). [2025-09-16T19:08:53.213125] [INFO ] [MAIN ] KORE: Full history saved. [2025-09-16T19:08:53.213229] [INFO ] [MAIN ] Update Progress: 1215/2369 symbols completed. KORE took 19.3s. Estimated remaining: 2h 2m 22s. Rate: 9.4 symbols/min [2025-09-16T19:08:53.398172] [INFO ] [MAIN ] KRC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:53.431609] [DEBUG] [LOCAL ] KRC: Loaded 35359 existing records from primary storage. [2025-09-16T19:08:53.431772] [INFO ] [MAIN ] KRC: Loaded 35359 existing records. [2025-09-16T19:08:53.624909] [WARN ] [MAIN ] KRC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:53.625179] [INFO ] [MAIN ] KRC: Wiping existing primary data before full refresh. [2025-09-16T19:08:53.625237] [INFO ] [LOCAL ] KRC: Starting deletion of primary data. [2025-09-16T19:08:53.626119] [DEBUG] [LOCAL ] KRC: Deleted primary data file. [2025-09-16T19:08:53.626174] [INFO ] [MAIN ] KRC: Starting full primary refresh. [2025-09-16T19:08:53.985276] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-09. [2025-09-16T19:08:53.990612] [INFO ] [REMOTE ] KRC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:54.774959] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-02. [2025-09-16T19:08:54.977467] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T19:08:54.980532] [INFO ] [REMOTE ] KOP: Full fetch complete (19539 records). [2025-09-16T19:08:54.980606] [DEBUG] [VALIDATOR ] KOP: Validating primary data. [2025-09-16T19:08:55.347730] [DEBUG] [VALIDATOR ] KOP: All expected trading days are present. [2025-09-16T19:08:55.347838] [DEBUG] [VALIDATOR ] KOP: Primary data validation passed. [2025-09-16T19:08:55.348357] [INFO ] [MAIN ] KOP: Fetched primary data is valid. [2025-09-16T19:08:55.348446] [DEBUG] [LOCAL ] KOP: Saving primary data. [2025-09-16T19:08:55.349248] [DEBUG] [LOCAL ] KOP: Trimmed 19539 records to 12808 (keeping 150 days of data) [2025-09-16T19:08:55.447439] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-05 (2.27 seconds). [2025-09-16T19:08:55.474354] [DEBUG] [LOCAL ] KOP: Primary data saved (12808 records). [2025-09-16T19:08:55.474528] [INFO ] [MAIN ] KOP: Full history saved. [2025-09-16T19:08:55.474624] [INFO ] [MAIN ] Update Progress: 1216/2369 symbols completed. KOP took 25.2s. Estimated remaining: 2h 2m 12s. Rate: 9.4 symbols/min [2025-09-16T19:08:55.575727] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-06. [2025-09-16T19:08:55.663731] [INFO ] [MAIN ] KREF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:08:55.690301] [DEBUG] [LOCAL ] KREF: Loaded 27019 existing records from primary storage. [2025-09-16T19:08:55.690488] [INFO ] [MAIN ] KREF: Loaded 27019 existing records. [2025-09-16T19:08:55.866465] [WARN ] [MAIN ] KREF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:08:55.866721] [INFO ] [MAIN ] KREF: Wiping existing primary data before full refresh. [2025-09-16T19:08:55.866771] [INFO ] [LOCAL ] KREF: Starting deletion of primary data. [2025-09-16T19:08:55.867394] [DEBUG] [LOCAL ] KREF: Deleted primary data file. [2025-09-16T19:08:55.867441] [INFO ] [MAIN ] KREF: Starting full primary refresh. [2025-09-16T19:08:56.220585] [INFO ] [REMOTE ] KREF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:08:56.245841] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T19:08:56.374859] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-02. [2025-09-16T19:08:57.174914] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-03. [2025-09-16T19:08:57.219629] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T19:08:57.658408] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-09-16T19:08:57.974914] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-06. [2025-09-16T19:08:58.774864] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-07. [2025-09-16T19:08:58.803876] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T19:08:59.574958] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-04. [2025-09-16T19:08:59.888194] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-09-16T19:09:00.374911] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-07. [2025-09-16T19:09:00.602464] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-09-16T19:09:00.990285] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T19:09:01.174952] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-08. [2025-09-16T19:09:01.974953] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-05. [2025-09-16T19:09:02.325396] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-09-16T19:09:02.774935] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-03. [2025-09-16T19:09:03.088632] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-09-16T19:09:03.344312] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-09-16T19:09:03.574908] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-06. [2025-09-16T19:09:04.210583] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T19:09:04.374945] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-09. [2025-09-16T19:09:04.888889] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T19:09:05.174957] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-07. [2025-09-16T19:09:05.974964] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-04. [2025-09-16T19:09:06.179282] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-09-16T19:09:06.186358] [INFO ] [REMOTE ] KOS: Full fetch complete (57711 records). [2025-09-16T19:09:06.186524] [DEBUG] [VALIDATOR ] KOS: Validating primary data. [2025-09-16T19:09:06.581702] [DEBUG] [VALIDATOR ] KOS: All expected trading days are present. [2025-09-16T19:09:06.581801] [DEBUG] [VALIDATOR ] KOS: Primary data validation passed. [2025-09-16T19:09:06.582606] [INFO ] [MAIN ] KOS: Fetched primary data is valid. [2025-09-16T19:09:06.582686] [DEBUG] [LOCAL ] KOS: Saving primary data. [2025-09-16T19:09:06.584015] [DEBUG] [LOCAL ] KOS: Trimmed 57711 records to 39572 (keeping 150 days of data) [2025-09-16T19:09:06.779534] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-08. [2025-09-16T19:09:06.815623] [DEBUG] [LOCAL ] KOS: Primary data saved (39572 records). [2025-09-16T19:09:06.815795] [INFO ] [MAIN ] KOS: Full history saved. [2025-09-16T19:09:06.816697] [INFO ] [MAIN ] Update Progress: 1217/2369 symbols completed. KOS took 32.8s. Estimated remaining: 2h 2m 10s. Rate: 9.4 symbols/min [2025-09-16T19:09:06.998910] [INFO ] [MAIN ] KRG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:09:07.031380] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-09-16T19:09:07.077038] [DEBUG] [LOCAL ] KRG: Loaded 36082 existing records from primary storage. [2025-09-16T19:09:07.077188] [INFO ] [MAIN ] KRG: Loaded 36082 existing records. [2025-09-16T19:09:07.261762] [WARN ] [MAIN ] KRG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:09:07.262018] [INFO ] [MAIN ] KRG: Wiping existing primary data before full refresh. [2025-09-16T19:09:07.262065] [INFO ] [LOCAL ] KRG: Starting deletion of primary data. [2025-09-16T19:09:07.262943] [DEBUG] [LOCAL ] KRG: Deleted primary data file. [2025-09-16T19:09:07.262993] [INFO ] [MAIN ] KRG: Starting full primary refresh. [2025-09-16T19:09:07.583256] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-08. [2025-09-16T19:09:07.671492] [INFO ] [REMOTE ] KRG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:09:07.696814] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T19:09:08.374941] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-05. [2025-09-16T19:09:08.650589] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-09-16T19:09:09.174932] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-09. [2025-09-16T19:09:09.334982] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T19:09:09.719932] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T19:09:09.974916] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-02. [2025-09-16T19:09:10.689976] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T19:09:10.697022] [INFO ] [REMOTE ] KR: Full fetch complete (58358 records). [2025-09-16T19:09:10.697154] [DEBUG] [VALIDATOR ] KR: Validating primary data. [2025-09-16T19:09:10.780054] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-06. [2025-09-16T19:09:11.119549] [DEBUG] [VALIDATOR ] KR: All expected trading days are present. [2025-09-16T19:09:11.119632] [DEBUG] [VALIDATOR ] KR: Primary data validation passed. [2025-09-16T19:09:11.120471] [INFO ] [MAIN ] KR: Fetched primary data is valid. [2025-09-16T19:09:11.120532] [DEBUG] [LOCAL ] KR: Saving primary data. [2025-09-16T19:09:11.121766] [DEBUG] [LOCAL ] KR: Trimmed 58358 records to 40033 (keeping 150 days of data) [2025-09-16T19:09:11.374581] [DEBUG] [LOCAL ] KR: Primary data saved (40033 records). [2025-09-16T19:09:11.374729] [INFO ] [MAIN ] KR: Full history saved. [2025-09-16T19:09:11.374848] [INFO ] [MAIN ] Update Progress: 1218/2369 symbols completed. KR took 31.7s. Estimated remaining: 2h 2m 2s. Rate: 9.4 symbols/min [2025-09-16T19:09:11.551920] [INFO ] [MAIN ] KRMN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:09:11.552199] [DEBUG] [LOCAL ] KRMN: No existing data file found in primary storage. [2025-09-16T19:09:11.552267] [WARN ] [MAIN ] KRMN: No existing data found, performing full refresh instead. [2025-09-16T19:09:11.552701] [INFO ] [MAIN ] KRMN: Wiping existing primary data before full refresh. [2025-09-16T19:09:11.552754] [INFO ] [LOCAL ] KRMN: Starting deletion of primary data. [2025-09-16T19:09:11.552806] [DEBUG] [LOCAL ] KRMN: No primary data file found to delete. [2025-09-16T19:09:11.552848] [INFO ] [MAIN ] KRMN: Starting full primary refresh. [2025-09-16T19:09:11.583119] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-09. [2025-09-16T19:09:11.867983] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-09-16T19:09:11.956720] [INFO ] [REMOTE ] KRMN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:09:11.981804] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-09-16T19:09:12.374904] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-03. [2025-09-16T19:09:13.174969] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-07. [2025-09-16T19:09:13.223260] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T19:09:13.228994] [INFO ] [REMOTE ] KRC: Full fetch complete (52789 records). [2025-09-16T19:09:13.229086] [DEBUG] [VALIDATOR ] KRC: Validating primary data. [2025-09-16T19:09:13.609076] [DEBUG] [VALIDATOR ] KRC: All expected trading days are present. [2025-09-16T19:09:13.609198] [DEBUG] [VALIDATOR ] KRC: Primary data validation passed. [2025-09-16T19:09:13.610058] [INFO ] [MAIN ] KRC: Fetched primary data is valid. [2025-09-16T19:09:13.610150] [DEBUG] [LOCAL ] KRC: Saving primary data. [2025-09-16T19:09:13.611447] [DEBUG] [LOCAL ] KRC: Trimmed 52789 records to 35713 (keeping 150 days of data) [2025-09-16T19:09:13.736379] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T19:09:13.862440] [DEBUG] [LOCAL ] KRC: Primary data saved (35713 records). [2025-09-16T19:09:13.862582] [INFO ] [MAIN ] KRC: Full history saved. [2025-09-16T19:09:13.862702] [INFO ] [MAIN ] Update Progress: 1219/2369 symbols completed. KRC took 20.6s. Estimated remaining: 2h 1m 52s. Rate: 9.4 symbols/min [2025-09-16T19:09:13.979975] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-02. [2025-09-16T19:09:14.035438] [INFO ] [MAIN ] KRO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:09:14.054419] [DEBUG] [LOCAL ] KRO: Loaded 19369 existing records from primary storage. [2025-09-16T19:09:14.054548] [INFO ] [MAIN ] KRO: Loaded 19369 existing records. [2025-09-16T19:09:14.231260] [WARN ] [MAIN ] KRO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:09:14.231529] [INFO ] [MAIN ] KRO: Wiping existing primary data before full refresh. [2025-09-16T19:09:14.231585] [INFO ] [LOCAL ] KRO: Starting deletion of primary data. [2025-09-16T19:09:14.232081] [DEBUG] [LOCAL ] KRO: Deleted primary data file. [2025-09-16T19:09:14.232130] [INFO ] [MAIN ] KRO: Starting full primary refresh. [2025-09-16T19:09:14.589042] [INFO ] [REMOTE ] KRO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:09:14.629946] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T19:09:14.774874] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-04. [2025-09-16T19:09:14.804221] [INFO ] [REMOTE ] KRMN: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T19:09:15.574902] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-03. [2025-09-16T19:09:16.374907] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-02. [2025-09-16T19:09:16.511049] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-09-16T19:09:16.933955] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-09-16T19:09:17.173976] [INFO ] [REMOTE ] KRMN: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T19:09:17.182492] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-05. [2025-09-16T19:09:17.974910] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-03. [2025-09-16T19:09:18.522594] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T19:09:18.711086] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-09-16T19:09:18.774850] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-04. [2025-09-16T19:09:19.574944] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-04. [2025-09-16T19:09:20.342377] [INFO ] [REMOTE ] KRMN: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-09-16T19:09:20.374843] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-05. [2025-09-16T19:09:21.146300] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-09-16T19:09:21.174859] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-05. [2025-09-16T19:09:21.838126] [INFO ] [REMOTE ] KRMN: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T19:09:21.974923] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-08. [2025-09-16T19:09:22.396773] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T19:09:22.774903] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-06. [2025-09-16T19:09:23.288560] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T19:09:23.574920] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-09. [2025-09-16T19:09:24.237048] [INFO ] [REMOTE ] KRMN: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T19:09:24.374939] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-07. [2025-09-16T19:09:24.937073] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-09-16T19:09:24.941409] [INFO ] [REMOTE ] KREF: Full fetch complete (39329 records). [2025-09-16T19:09:24.941514] [DEBUG] [VALIDATOR ] KREF: Validating primary data. [2025-09-16T19:09:25.180126] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-06. [2025-09-16T19:09:25.340438] [DEBUG] [VALIDATOR ] KREF: All expected trading days are present. [2025-09-16T19:09:25.340547] [DEBUG] [VALIDATOR ] KREF: Primary data validation passed. [2025-09-16T19:09:25.341250] [INFO ] [MAIN ] KREF: Fetched primary data is valid. [2025-09-16T19:09:25.341297] [DEBUG] [LOCAL ] KREF: Saving primary data. [2025-09-16T19:09:25.342496] [DEBUG] [LOCAL ] KREF: Trimmed 39329 records to 27305 (keeping 150 days of data) [2025-09-16T19:09:25.509248] [DEBUG] [LOCAL ] KREF: Primary data saved (27305 records). [2025-09-16T19:09:25.509426] [INFO ] [MAIN ] KREF: Full history saved. [2025-09-16T19:09:25.509558] [INFO ] [MAIN ] Update Progress: 1220/2369 symbols completed. KREF took 30.0s. Estimated remaining: 2h 1m 51s. Rate: 9.4 symbols/min [2025-09-16T19:09:25.686656] [INFO ] [MAIN ] KRP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:09:25.712752] [DEBUG] [LOCAL ] KRP: Loaded 26642 existing records from primary storage. [2025-09-16T19:09:25.712891] [INFO ] [MAIN ] KRP: Loaded 26642 existing records. [2025-09-16T19:09:25.888715] [WARN ] [MAIN ] KRP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:09:25.888996] [INFO ] [MAIN ] KRP: Wiping existing primary data before full refresh. [2025-09-16T19:09:25.889091] [INFO ] [LOCAL ] KRP: Starting deletion of primary data. [2025-09-16T19:09:25.889766] [DEBUG] [LOCAL ] KRP: Deleted primary data file. [2025-09-16T19:09:25.889815] [INFO ] [MAIN ] KRP: Starting full primary refresh. [2025-09-16T19:09:25.984803] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-06. [2025-09-16T19:09:26.259134] [INFO ] [REMOTE ] KRP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:09:26.295769] [INFO ] [REMOTE ] KRMN: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-09-16T19:09:26.332566] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-09-16T19:09:26.774928] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-02. [2025-09-16T19:09:27.574937] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-08. [2025-09-16T19:09:27.666914] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T19:09:27.740931] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-09-16T19:09:28.374906] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-03. [2025-09-16T19:09:29.039894] [INFO ] [REMOTE ] KRMN: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-09-16T19:09:29.174932] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-07. [2025-09-16T19:09:29.937549] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T19:09:29.975028] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-09. [2025-09-16T19:09:30.454134] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-09-16T19:09:30.774933] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-04. [2025-09-16T19:09:31.326854] [INFO ] [REMOTE ] KRMN: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T19:09:31.332848] [INFO ] [REMOTE ] KRMN: Full fetch complete (47758 records). [2025-09-16T19:09:31.332983] [DEBUG] [VALIDATOR ] KRMN: Validating primary data. [2025-09-16T19:09:31.520359] [WARN ] [VALIDATOR ] KRMN: Validation failed - insufficient historical data. Have 148 unique days, need 150 days [2025-09-16T19:09:31.520933] [FATAL] [MAIN ] KRMN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:09:31.520999] [WARN ] [MAIN ] KRMN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:09:31.521055] [INFO ] [LOCAL ] KRMN: Starting deletion of primary data. [2025-09-16T19:09:31.521120] [DEBUG] [LOCAL ] KRMN: No primary data file found to delete. [2025-09-16T19:09:31.521166] [INFO ] [LOCAL ] KRMN: Starting deletion of derivative data. [2025-09-16T19:09:31.521218] [DEBUG] [LOCAL ] KRMN: No derivative data file found to delete. [2025-09-16T19:09:31.521287] [INFO ] [MAIN ] Update Progress: 1221/2369 symbols completed. KRMN took 20.1s. Estimated remaining: 2h 1m 44s. Rate: 9.4 symbols/min [2025-09-16T19:09:31.585159] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-08. [2025-09-16T19:09:31.696089] [INFO ] [MAIN ] KSS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:09:31.733539] [DEBUG] [LOCAL ] KSS: Loaded 39575 existing records from primary storage. [2025-09-16T19:09:31.733709] [INFO ] [MAIN ] KSS: Loaded 39575 existing records. [2025-09-16T19:09:31.917281] [WARN ] [MAIN ] KSS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:09:31.917584] [INFO ] [MAIN ] KSS: Wiping existing primary data before full refresh. [2025-09-16T19:09:31.917635] [INFO ] [LOCAL ] KSS: Starting deletion of primary data. [2025-09-16T19:09:31.918446] [DEBUG] [LOCAL ] KSS: Deleted primary data file. [2025-09-16T19:09:31.918505] [INFO ] [MAIN ] KSS: Starting full primary refresh. [2025-09-16T19:09:32.276048] [INFO ] [REMOTE ] KSS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:09:32.297765] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T19:09:32.374850] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-02. [2025-09-16T19:09:32.910194] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-09-16T19:09:33.174910] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-07. [2025-09-16T19:09:33.974936] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-05. [2025-09-16T19:09:34.157667] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T19:09:34.774942] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-03. [2025-09-16T19:09:34.877692] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-09-16T19:09:35.404792] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-09-16T19:09:35.574895] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-06. [2025-09-16T19:09:36.374910] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-09. [2025-09-16T19:09:36.785175] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-09-16T19:09:37.024843] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T19:09:37.174902] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-07. [2025-09-16T19:09:37.242988] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-09-16T19:09:37.246473] [INFO ] [REMOTE ] KRO: Full fetch complete (27067 records). [2025-09-16T19:09:37.246544] [DEBUG] [VALIDATOR ] KRO: Validating primary data. [2025-09-16T19:09:37.614852] [DEBUG] [VALIDATOR ] KRO: All expected trading days are present. [2025-09-16T19:09:37.614935] [DEBUG] [VALIDATOR ] KRO: Primary data validation passed. [2025-09-16T19:09:37.615615] [INFO ] [MAIN ] KRO: Fetched primary data is valid. [2025-09-16T19:09:37.615682] [DEBUG] [LOCAL ] KRO: Saving primary data. [2025-09-16T19:09:37.616475] [DEBUG] [LOCAL ] KRO: Trimmed 27067 records to 19557 (keeping 150 days of data) [2025-09-16T19:09:37.732171] [DEBUG] [LOCAL ] KRO: Primary data saved (19557 records). [2025-09-16T19:09:37.732321] [INFO ] [MAIN ] KRO: Full history saved. [2025-09-16T19:09:37.732432] [INFO ] [MAIN ] Update Progress: 1222/2369 symbols completed. KRO took 23.9s. Estimated remaining: 2h 1m 37s. Rate: 9.4 symbols/min [2025-09-16T19:09:37.905055] [INFO ] [MAIN ] KT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:09:37.940405] [DEBUG] [LOCAL ] KT: Loaded 33182 existing records from primary storage. [2025-09-16T19:09:37.940544] [INFO ] [MAIN ] KT: Loaded 33182 existing records. [2025-09-16T19:09:37.982572] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-04. [2025-09-16T19:09:38.120397] [WARN ] [MAIN ] KT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:09:38.120699] [INFO ] [MAIN ] KT: Wiping existing primary data before full refresh. [2025-09-16T19:09:38.120751] [INFO ] [LOCAL ] KT: Starting deletion of primary data. [2025-09-16T19:09:38.121690] [DEBUG] [LOCAL ] KT: Deleted primary data file. [2025-09-16T19:09:38.121748] [INFO ] [MAIN ] KT: Starting full primary refresh. [2025-09-16T19:09:38.486750] [INFO ] [REMOTE ] KT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:09:38.521553] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T19:09:38.774915] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-08. [2025-09-16T19:09:39.574896] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-08. [2025-09-16T19:09:40.117363] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-04 (2.13 seconds). [2025-09-16T19:09:40.311325] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T19:09:40.374882] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-09. [2025-09-16T19:09:41.046823] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-09-16T19:09:41.174946] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-02. [2025-09-16T19:09:41.663049] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T19:09:41.669579] [INFO ] [REMOTE ] KRG: Full fetch complete (53446 records). [2025-09-16T19:09:41.669735] [DEBUG] [VALIDATOR ] KRG: Validating primary data. [2025-09-16T19:09:41.980093] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-05. [2025-09-16T19:09:42.058944] [DEBUG] [VALIDATOR ] KRG: All expected trading days are present. [2025-09-16T19:09:42.059036] [DEBUG] [VALIDATOR ] KRG: Primary data validation passed. [2025-09-16T19:09:42.059859] [INFO ] [MAIN ] KRG: Fetched primary data is valid. [2025-09-16T19:09:42.059931] [DEBUG] [LOCAL ] KRG: Saving primary data. [2025-09-16T19:09:42.061304] [DEBUG] [LOCAL ] KRG: Trimmed 53446 records to 36453 (keeping 150 days of data) [2025-09-16T19:09:42.291278] [DEBUG] [LOCAL ] KRG: Primary data saved (36453 records). [2025-09-16T19:09:42.291430] [INFO ] [MAIN ] KRG: Full history saved. [2025-09-16T19:09:42.292186] [INFO ] [MAIN ] Update Progress: 1223/2369 symbols completed. KRG took 35.5s. Estimated remaining: 2h 1m 29s. Rate: 9.4 symbols/min [2025-09-16T19:09:42.467501] [INFO ] [MAIN ] KTB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:09:42.503835] [DEBUG] [LOCAL ] KTB: Loaded 30183 existing records from primary storage. [2025-09-16T19:09:42.503979] [INFO ] [MAIN ] KTB: Loaded 30183 existing records. [2025-09-16T19:09:42.687390] [WARN ] [MAIN ] KTB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:09:42.687718] [INFO ] [MAIN ] KTB: Wiping existing primary data before full refresh. [2025-09-16T19:09:42.687772] [INFO ] [LOCAL ] KTB: Starting deletion of primary data. [2025-09-16T19:09:42.688431] [DEBUG] [LOCAL ] KTB: Deleted primary data file. [2025-09-16T19:09:42.688510] [INFO ] [MAIN ] KTB: Starting full primary refresh. [2025-09-16T19:09:42.783157] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-09. [2025-09-16T19:09:43.068992] [INFO ] [REMOTE ] KTB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:09:43.095871] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-09-16T19:09:43.574932] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-03. [2025-09-16T19:09:43.712188] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T19:09:43.738198] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-09-16T19:09:43.742585] [INFO ] [REMOTE ] KRP: Full fetch complete (41621 records). [2025-09-16T19:09:43.742718] [DEBUG] [VALIDATOR ] KRP: Validating primary data. [2025-09-16T19:09:44.121578] [DEBUG] [VALIDATOR ] KRP: All expected trading days are present. [2025-09-16T19:09:44.121681] [DEBUG] [VALIDATOR ] KRP: Primary data validation passed. [2025-09-16T19:09:44.122521] [INFO ] [MAIN ] KRP: Fetched primary data is valid. [2025-09-16T19:09:44.122567] [DEBUG] [LOCAL ] KRP: Saving primary data. [2025-09-16T19:09:44.123605] [DEBUG] [LOCAL ] KRP: Trimmed 41621 records to 26905 (keeping 150 days of data) [2025-09-16T19:09:44.297111] [DEBUG] [LOCAL ] KRP: Primary data saved (26905 records). [2025-09-16T19:09:44.297252] [INFO ] [MAIN ] KRP: Full history saved. [2025-09-16T19:09:44.297522] [INFO ] [MAIN ] Update Progress: 1224/2369 symbols completed. KRP took 18.8s. Estimated remaining: 2h 1m 19s. Rate: 9.4 symbols/min [2025-09-16T19:09:44.380115] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-06. [2025-09-16T19:09:44.469631] [INFO ] [MAIN ] KTF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:09:44.479410] [DEBUG] [LOCAL ] KTF: Loaded 8631 existing records from primary storage. [2025-09-16T19:09:44.479540] [INFO ] [MAIN ] KTF: Loaded 8631 existing records. [2025-09-16T19:09:44.655960] [WARN ] [MAIN ] KTF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:09:44.656218] [INFO ] [MAIN ] KTF: Wiping existing primary data before full refresh. [2025-09-16T19:09:44.656287] [INFO ] [LOCAL ] KTF: Starting deletion of primary data. [2025-09-16T19:09:44.656593] [DEBUG] [LOCAL ] KTF: Deleted primary data file. [2025-09-16T19:09:44.656636] [INFO ] [MAIN ] KTF: Starting full primary refresh. [2025-09-16T19:09:45.007373] [INFO ] [REMOTE ] KTF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:09:45.178071] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-02. [2025-09-16T19:09:45.182968] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T19:09:45.974755] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-09-16T19:09:45.974912] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-04. [2025-09-16T19:09:46.376043] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-09-16T19:09:46.774859] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-02. [2025-09-16T19:09:47.556572] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T19:09:47.574941] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-05. [2025-09-16T19:09:48.292207] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-09-16T19:09:48.374874] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-03. [2025-09-16T19:09:49.174888] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-03. [2025-09-16T19:09:49.206087] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-09-16T19:09:49.971427] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T19:09:49.980408] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-07. [2025-09-16T19:09:49.994703] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-09-16T19:09:50.774969] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-06. [2025-09-16T19:09:51.574908] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-04. [2025-09-16T19:09:52.107799] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-09-16T19:09:52.374927] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-04. [2025-09-16T19:09:52.443190] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-09-16T19:09:52.636148] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-09-16T19:09:53.174937] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-08. [2025-09-16T19:09:53.881772] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T19:09:53.974866] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-05. [2025-09-16T19:09:54.774880] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-07. [2025-09-16T19:09:55.286820] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-09-16T19:09:55.491162] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-09-16T19:09:55.574907] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-06. [2025-09-16T19:09:56.374962] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-09. [2025-09-16T19:09:56.376313] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T19:09:57.039456] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T19:09:57.174944] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-08. [2025-09-16T19:09:57.974913] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-07. [2025-09-16T19:09:58.289766] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-09-16T19:09:58.296454] [INFO ] [REMOTE ] KSS: Full fetch complete (58288 records). [2025-09-16T19:09:58.296572] [DEBUG] [VALIDATOR ] KSS: Validating primary data. [2025-09-16T19:09:58.681630] [DEBUG] [VALIDATOR ] KSS: All expected trading days are present. [2025-09-16T19:09:58.681831] [DEBUG] [VALIDATOR ] KSS: Primary data validation passed. [2025-09-16T19:09:58.682720] [INFO ] [MAIN ] KSS: Fetched primary data is valid. [2025-09-16T19:09:58.682807] [DEBUG] [LOCAL ] KSS: Saving primary data. [2025-09-16T19:09:58.693424] [DEBUG] [LOCAL ] KSS: Trimmed 58288 records to 39965 (keeping 150 days of data) [2025-09-16T19:09:58.710348] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T19:09:58.779047] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-05. [2025-09-16T19:09:58.964303] [DEBUG] [LOCAL ] KSS: Primary data saved (39965 records). [2025-09-16T19:09:58.964460] [INFO ] [MAIN ] KSS: Full history saved. [2025-09-16T19:09:58.964561] [INFO ] [MAIN ] Update Progress: 1225/2369 symbols completed. KSS took 27.4s. Estimated remaining: 2h 1m 20s. Rate: 9.4 symbols/min [2025-09-16T19:09:59.140271] [INFO ] [MAIN ] KUKE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:09:59.140464] [DEBUG] [LOCAL ] KUKE: No existing data file found in primary storage. [2025-09-16T19:09:59.140509] [WARN ] [MAIN ] KUKE: No existing data found, performing full refresh instead. [2025-09-16T19:09:59.140632] [INFO ] [MAIN ] KUKE: Wiping existing primary data before full refresh. [2025-09-16T19:09:59.140699] [INFO ] [LOCAL ] KUKE: Starting deletion of primary data. [2025-09-16T19:09:59.140754] [DEBUG] [LOCAL ] KUKE: No primary data file found to delete. [2025-09-16T19:09:59.140797] [INFO ] [MAIN ] KUKE: Starting full primary refresh. [2025-09-16T19:09:59.497561] [INFO ] [REMOTE ] KUKE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:09:59.534571] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-09-16T19:09:59.577877] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-02. [2025-09-16T19:09:59.578944] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-09-16T19:10:00.235362] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-09-16T19:10:00.374955] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-09. [2025-09-16T19:10:01.174950] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-08. [2025-09-16T19:10:01.974949] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-06. [2025-09-16T19:10:02.067197] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-09-16T19:10:02.073280] [INFO ] [REMOTE ] KT: Full fetch complete (50074 records). [2025-09-16T19:10:02.073445] [DEBUG] [VALIDATOR ] KT: Validating primary data. [2025-09-16T19:10:02.477962] [DEBUG] [VALIDATOR ] KT: All expected trading days are present. [2025-09-16T19:10:02.478063] [DEBUG] [VALIDATOR ] KT: Primary data validation passed. [2025-09-16T19:10:02.478926] [INFO ] [MAIN ] KT: Fetched primary data is valid. [2025-09-16T19:10:02.478978] [DEBUG] [LOCAL ] KT: Saving primary data. [2025-09-16T19:10:02.480238] [DEBUG] [LOCAL ] KT: Trimmed 50074 records to 33570 (keeping 150 days of data) [2025-09-16T19:10:02.704385] [DEBUG] [LOCAL ] KT: Primary data saved (33570 records). [2025-09-16T19:10:02.704608] [INFO ] [MAIN ] KT: Full history saved. [2025-09-16T19:10:02.704729] [INFO ] [MAIN ] Update Progress: 1226/2369 symbols completed. KT took 25.0s. Estimated remaining: 2h 1m 12s. Rate: 9.4 symbols/min [2025-09-16T19:10:02.779976] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-03. [2025-09-16T19:10:02.894601] [INFO ] [MAIN ] KVUE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:10:02.906230] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-09-16T19:10:02.940539] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T19:10:02.978091] [DEBUG] [LOCAL ] KVUE: Loaded 39627 existing records from primary storage. [2025-09-16T19:10:02.978252] [INFO ] [MAIN ] KVUE: Loaded 39627 existing records. [2025-09-16T19:10:03.158565] [WARN ] [MAIN ] KVUE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:10:03.158860] [INFO ] [MAIN ] KVUE: Wiping existing primary data before full refresh. [2025-09-16T19:10:03.158909] [INFO ] [LOCAL ] KVUE: Starting deletion of primary data. [2025-09-16T19:10:03.160121] [DEBUG] [LOCAL ] KVUE: Deleted primary data file. [2025-09-16T19:10:03.160176] [INFO ] [MAIN ] KVUE: Starting full primary refresh. [2025-09-16T19:10:03.529265] [INFO ] [REMOTE ] KVUE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:10:03.574866] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-02. [2025-09-16T19:10:03.778449] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-09-16T19:10:04.374910] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-07. [2025-09-16T19:10:05.174880] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-09. [2025-09-16T19:10:05.188374] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-09-16T19:10:05.262957] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T19:10:05.974910] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-03. [2025-09-16T19:10:06.596032] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T19:10:06.600707] [INFO ] [REMOTE ] KTB: Full fetch complete (45366 records). [2025-09-16T19:10:06.600807] [DEBUG] [VALIDATOR ] KTB: Validating primary data. [2025-09-16T19:10:06.779772] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-08. [2025-09-16T19:10:06.984082] [DEBUG] [VALIDATOR ] KTB: All expected trading days are present. [2025-09-16T19:10:06.984181] [DEBUG] [VALIDATOR ] KTB: Primary data validation passed. [2025-09-16T19:10:06.985078] [INFO ] [MAIN ] KTB: Fetched primary data is valid. [2025-09-16T19:10:06.985125] [DEBUG] [LOCAL ] KTB: Saving primary data. [2025-09-16T19:10:06.986287] [DEBUG] [LOCAL ] KTB: Trimmed 45366 records to 30470 (keeping 150 days of data) [2025-09-16T19:10:07.180679] [DEBUG] [LOCAL ] KTB: Primary data saved (30470 records). [2025-09-16T19:10:07.180844] [INFO ] [MAIN ] KTB: Full history saved. [2025-09-16T19:10:07.181801] [INFO ] [MAIN ] Update Progress: 1227/2369 symbols completed. KTB took 24.9s. Estimated remaining: 2h 1m 3s. Rate: 9.4 symbols/min [2025-09-16T19:10:07.353707] [INFO ] [MAIN ] KVYO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:10:07.392204] [DEBUG] [LOCAL ] KVYO: Loaded 37223 existing records from primary storage. [2025-09-16T19:10:07.392363] [INFO ] [MAIN ] KVYO: Loaded 37223 existing records. [2025-09-16T19:10:07.575519] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-04. [2025-09-16T19:10:07.578719] [WARN ] [MAIN ] KVYO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:10:07.580227] [INFO ] [MAIN ] KVYO: Wiping existing primary data before full refresh. [2025-09-16T19:10:07.580284] [INFO ] [LOCAL ] KVYO: Starting deletion of primary data. [2025-09-16T19:10:07.581145] [DEBUG] [LOCAL ] KVYO: Deleted primary data file. [2025-09-16T19:10:07.581253] [INFO ] [MAIN ] KVYO: Starting full primary refresh. [2025-09-16T19:10:07.955581] [INFO ] [REMOTE ] KVYO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:10:07.957233] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T19:10:07.992733] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-09-16T19:10:08.219286] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-09-16T19:10:08.374903] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-02. [2025-09-16T19:10:09.174922] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-05. [2025-09-16T19:10:09.687392] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-09-16T19:10:09.982663] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-06. [2025-09-16T19:10:09.985374] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T19:10:10.428593] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-09-16T19:10:10.774922] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-07. [2025-09-16T19:10:10.930438] [DEBUG] [REMOTE ] KUKE: Error fetching primary data for 2025-07: 'API call returned an empty response.' [2025-09-16T19:10:10.930777] [WARN ] [REMOTE ] KUKE: No data returned for month 2025-07 [2025-09-16T19:10:11.574897] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-09. [2025-09-16T19:10:12.374934] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-03. [2025-09-16T19:10:12.392305] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-09-16T19:10:12.394777] [INFO ] [REMOTE ] KTF: Full fetch complete (14197 records). [2025-09-16T19:10:12.394864] [DEBUG] [VALIDATOR ] KTF: Validating primary data. [2025-09-16T19:10:12.759796] [DEBUG] [VALIDATOR ] KTF: All expected trading days are present. [2025-09-16T19:10:12.759872] [DEBUG] [VALIDATOR ] KTF: Primary data validation passed. [2025-09-16T19:10:12.760183] [INFO ] [MAIN ] KTF: Fetched primary data is valid. [2025-09-16T19:10:12.760223] [DEBUG] [LOCAL ] KTF: Saving primary data. [2025-09-16T19:10:12.760706] [DEBUG] [LOCAL ] KTF: Trimmed 14197 records to 8672 (keeping 150 days of data) [2025-09-16T19:10:12.814166] [DEBUG] [LOCAL ] KTF: Primary data saved (8672 records). [2025-09-16T19:10:12.814301] [INFO ] [MAIN ] KTF: Full history saved. [2025-09-16T19:10:12.814384] [INFO ] [MAIN ] Update Progress: 1228/2369 symbols completed. KTF took 28.5s. Estimated remaining: 2h 0m 56s. Rate: 9.4 symbols/min [2025-09-16T19:10:12.986353] [INFO ] [MAIN ] KW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:10:13.014170] [DEBUG] [LOCAL ] KW: Loaded 29877 existing records from primary storage. [2025-09-16T19:10:13.014313] [INFO ] [MAIN ] KW: Loaded 29877 existing records. [2025-09-16T19:10:13.191139] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-04. [2025-09-16T19:10:13.196905] [WARN ] [MAIN ] KW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:10:13.197101] [INFO ] [MAIN ] KW: Wiping existing primary data before full refresh. [2025-09-16T19:10:13.197146] [INFO ] [LOCAL ] KW: Starting deletion of primary data. [2025-09-16T19:10:13.197845] [DEBUG] [LOCAL ] KW: Deleted primary data file. [2025-09-16T19:10:13.197893] [INFO ] [MAIN ] KW: Starting full primary refresh. [2025-09-16T19:10:13.556127] [INFO ] [REMOTE ] KW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:10:13.974902] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-02. [2025-09-16T19:10:14.078045] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-09-16T19:10:14.692733] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T19:10:14.774895] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-05. [2025-09-16T19:10:15.286433] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T19:10:15.574903] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-03. [2025-09-16T19:10:16.374907] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-04. [2025-09-16T19:10:16.490999] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-09-16T19:10:17.146050] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-09-16T19:10:17.174888] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-04. [2025-09-16T19:10:17.791018] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T19:10:17.974914] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-05. [2025-09-16T19:10:18.387311] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-09-16T19:10:18.774892] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-05. [2025-09-16T19:10:19.451078] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T19:10:19.574852] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-06. [2025-09-16T19:10:20.333235] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T19:10:20.374851] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-06. [2025-09-16T19:10:21.160170] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T19:10:21.174853] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-07. [2025-09-16T19:10:21.415456] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-09-16T19:10:21.974916] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-06. [2025-09-16T19:10:22.541206] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-09-16T19:10:22.774983] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-07. [2025-09-16T19:10:23.575000] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-08. [2025-09-16T19:10:23.826933] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-09-16T19:10:23.958121] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-09-16T19:10:24.238101] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T19:10:24.374901] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-08. [2025-09-16T19:10:25.174920] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-09. [2025-09-16T19:10:25.784145] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-09-16T19:10:25.784296] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-09-16T19:10:25.793493] [INFO ] [REMOTE ] KUKE: Full fetch complete (4644 records). [2025-09-16T19:10:25.793753] [DEBUG] [VALIDATOR ] KUKE: Validating primary data. [2025-09-16T19:10:25.984225] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-07. [2025-09-16T19:10:25.991886] [WARN ] [VALIDATOR ] KUKE: Validation failed - insufficient historical data. Have 128 unique days, need 150 days [2025-09-16T19:10:25.993098] [FATAL] [MAIN ] KUKE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:10:25.993210] [WARN ] [MAIN ] KUKE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:10:25.993291] [INFO ] [LOCAL ] KUKE: Starting deletion of primary data. [2025-09-16T19:10:25.993366] [DEBUG] [LOCAL ] KUKE: No primary data file found to delete. [2025-09-16T19:10:25.993417] [INFO ] [LOCAL ] KUKE: Starting deletion of derivative data. [2025-09-16T19:10:25.993492] [DEBUG] [LOCAL ] KUKE: No derivative data file found to delete. [2025-09-16T19:10:25.993714] [INFO ] [MAIN ] Update Progress: 1229/2369 symbols completed. KUKE took 27.0s. Estimated remaining: 2h 0m 56s. Rate: 9.4 symbols/min [2025-09-16T19:10:26.180932] [INFO ] [MAIN ] KWR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:10:26.194784] [DEBUG] [LOCAL ] KWR: Loaded 12280 existing records from primary storage. [2025-09-16T19:10:26.194919] [INFO ] [MAIN ] KWR: Loaded 12280 existing records. [2025-09-16T19:10:26.371466] [WARN ] [MAIN ] KWR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:10:26.371721] [INFO ] [MAIN ] KWR: Wiping existing primary data before full refresh. [2025-09-16T19:10:26.371767] [INFO ] [LOCAL ] KWR: Starting deletion of primary data. [2025-09-16T19:10:26.372238] [DEBUG] [LOCAL ] KWR: Deleted primary data file. [2025-09-16T19:10:26.372283] [INFO ] [MAIN ] KWR: Starting full primary refresh. [2025-09-16T19:10:26.728063] [INFO ] [REMOTE ] KWR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:10:26.774902] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-09. [2025-09-16T19:10:27.574916] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-08. [2025-09-16T19:10:27.890930] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T19:10:28.026105] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-09-16T19:10:28.030945] [INFO ] [REMOTE ] KW: Full fetch complete (43766 records). [2025-09-16T19:10:28.031046] [DEBUG] [VALIDATOR ] KW: Validating primary data. [2025-09-16T19:10:28.400266] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-02. [2025-09-16T19:10:28.416446] [DEBUG] [VALIDATOR ] KW: All expected trading days are present. [2025-09-16T19:10:28.416536] [DEBUG] [VALIDATOR ] KW: Primary data validation passed. [2025-09-16T19:10:28.417363] [INFO ] [MAIN ] KW: Fetched primary data is valid. [2025-09-16T19:10:28.417411] [DEBUG] [LOCAL ] KW: Saving primary data. [2025-09-16T19:10:28.418415] [DEBUG] [LOCAL ] KW: Trimmed 43766 records to 30155 (keeping 150 days of data) [2025-09-16T19:10:28.592503] [DEBUG] [LOCAL ] KW: Primary data saved (30155 records). [2025-09-16T19:10:28.592668] [INFO ] [MAIN ] KW: Full history saved. [2025-09-16T19:10:28.592779] [INFO ] [MAIN ] Update Progress: 1230/2369 symbols completed. KW took 15.8s. Estimated remaining: 2h 0m 47s. Rate: 9.4 symbols/min [2025-09-16T19:10:28.765232] [INFO ] [MAIN ] KYN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:10:28.787301] [DEBUG] [LOCAL ] KYN: Loaded 21836 existing records from primary storage. [2025-09-16T19:10:28.787438] [INFO ] [MAIN ] KYN: Loaded 21836 existing records. [2025-09-16T19:10:28.964374] [WARN ] [MAIN ] KYN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:10:28.964685] [INFO ] [MAIN ] KYN: Wiping existing primary data before full refresh. [2025-09-16T19:10:28.964735] [INFO ] [LOCAL ] KYN: Starting deletion of primary data. [2025-09-16T19:10:28.965471] [DEBUG] [LOCAL ] KYN: Deleted primary data file. [2025-09-16T19:10:28.965520] [INFO ] [MAIN ] KYN: Starting full primary refresh. [2025-09-16T19:10:29.174914] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-08. [2025-09-16T19:10:29.334669] [INFO ] [REMOTE ] KYN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:10:29.370934] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-09-16T19:10:29.402016] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-09-16T19:10:29.974911] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-02. [2025-09-16T19:10:30.774897] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-09. [2025-09-16T19:10:30.922381] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T19:10:30.950506] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-09-16T19:10:31.574900] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-09. [2025-09-16T19:10:32.374915] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-03. [2025-09-16T19:10:32.420204] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-09-16T19:10:32.426376] [INFO ] [REMOTE ] KVYO: Full fetch complete (54096 records). [2025-09-16T19:10:32.426465] [DEBUG] [VALIDATOR ] KVYO: Validating primary data. [2025-09-16T19:10:32.808663] [DEBUG] [VALIDATOR ] KVYO: All expected trading days are present. [2025-09-16T19:10:32.808794] [DEBUG] [VALIDATOR ] KVYO: Primary data validation passed. [2025-09-16T19:10:32.809601] [INFO ] [MAIN ] KVYO: Fetched primary data is valid. [2025-09-16T19:10:32.809658] [DEBUG] [LOCAL ] KVYO: Saving primary data. [2025-09-16T19:10:32.810868] [DEBUG] [LOCAL ] KVYO: Trimmed 54096 records to 37611 (keeping 150 days of data) [2025-09-16T19:10:33.029606] [DEBUG] [LOCAL ] KVYO: Primary data saved (37611 records). [2025-09-16T19:10:33.029758] [INFO ] [MAIN ] KVYO: Full history saved. [2025-09-16T19:10:33.031339] [INFO ] [MAIN ] Update Progress: 1231/2369 symbols completed. KVYO took 25.8s. Estimated remaining: 2h 0m 38s. Rate: 9.4 symbols/min [2025-09-16T19:10:33.178400] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-03. [2025-09-16T19:10:33.201114] [INFO ] [MAIN ] L: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:10:33.229112] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-09-16T19:10:33.239819] [INFO ] [REMOTE ] KVUE: Full fetch complete (58347 records). [2025-09-16T19:10:33.240034] [DEBUG] [VALIDATOR ] KVUE: Validating primary data. [2025-09-16T19:10:33.448236] [DEBUG] [LOCAL ] L: Loaded 30026 existing records from primary storage. [2025-09-16T19:10:33.498194] [INFO ] [MAIN ] L: Loaded 30026 existing records. [2025-09-16T19:10:33.818897] [DEBUG] [VALIDATOR ] KVUE: All expected trading days are present. [2025-09-16T19:10:33.838407] [WARN ] [MAIN ] L: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:10:33.838538] [DEBUG] [VALIDATOR ] KVUE: Primary data validation passed. [2025-09-16T19:10:33.839629] [INFO ] [MAIN ] KVUE: Fetched primary data is valid. [2025-09-16T19:10:33.839767] [DEBUG] [LOCAL ] KVUE: Saving primary data. [2025-09-16T19:10:33.839797] [INFO ] [MAIN ] L: Wiping existing primary data before full refresh. [2025-09-16T19:10:33.839981] [INFO ] [LOCAL ] L: Starting deletion of primary data. [2025-09-16T19:10:33.840924] [DEBUG] [LOCAL ] L: Deleted primary data file. [2025-09-16T19:10:33.841123] [DEBUG] [LOCAL ] KVUE: Trimmed 58347 records to 40017 (keeping 150 days of data) [2025-09-16T19:10:33.841010] [INFO ] [MAIN ] L: Starting full primary refresh. [2025-09-16T19:10:34.054423] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-09-16T19:10:34.193602] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-04. [2025-09-16T19:10:34.471291] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-09-16T19:10:34.516297] [INFO ] [REMOTE ] L: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:10:34.530789] [DEBUG] [LOCAL ] KVUE: Primary data saved (40017 records). [2025-09-16T19:10:34.530881] [INFO ] [MAIN ] KVUE: Full history saved. [2025-09-16T19:10:34.530957] [INFO ] [MAIN ] Update Progress: 1232/2369 symbols completed. KVUE took 31.8s. Estimated remaining: 2h 0m 28s. Rate: 9.4 symbols/min [2025-09-16T19:10:34.699036] [INFO ] [MAIN ] LAC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:10:34.734629] [DEBUG] [LOCAL ] LAC: Loaded 38655 existing records from primary storage. [2025-09-16T19:10:34.734795] [INFO ] [MAIN ] LAC: Loaded 38655 existing records. [2025-09-16T19:10:34.779997] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-04. [2025-09-16T19:10:34.935770] [WARN ] [MAIN ] LAC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:10:34.936037] [INFO ] [MAIN ] LAC: Wiping existing primary data before full refresh. [2025-09-16T19:10:34.936082] [INFO ] [LOCAL ] LAC: Starting deletion of primary data. [2025-09-16T19:10:34.936958] [DEBUG] [LOCAL ] LAC: Deleted primary data file. [2025-09-16T19:10:34.937008] [INFO ] [MAIN ] LAC: Starting full primary refresh. [2025-09-16T19:10:35.288843] [INFO ] [REMOTE ] LAC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:10:35.574904] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-02. [2025-09-16T19:10:35.713044] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T19:10:35.886174] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-09-16T19:10:36.374906] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-05. [2025-09-16T19:10:37.174898] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-05. [2025-09-16T19:10:37.322290] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-09-16T19:10:37.402170] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-09-16T19:10:37.974917] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-06. [2025-09-16T19:10:38.428513] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-09-16T19:10:38.770976] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-09-16T19:10:38.775026] [INFO ] [REMOTE ] L: Fetching primary data for 2025-02. [2025-09-16T19:10:39.574920] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-06. [2025-09-16T19:10:40.256574] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T19:10:40.374885] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-07. [2025-09-16T19:10:40.708779] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-09-16T19:10:41.174969] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-03. [2025-09-16T19:10:41.344870] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T19:10:41.974921] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-07. [2025-09-16T19:10:42.774922] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-08. [2025-09-16T19:10:43.094511] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-09-16T19:10:43.266563] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-09-16T19:10:43.574955] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-04. [2025-09-16T19:10:43.796961] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-09-16T19:10:44.374912] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-08. [2025-09-16T19:10:45.174960] [INFO ] [REMOTE ] L: Fetching primary data for 2025-03. [2025-09-16T19:10:45.496799] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-09-16T19:10:45.559170] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T19:10:45.974932] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-05. [2025-09-16T19:10:46.309236] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-09-16T19:10:46.774937] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-09. [2025-09-16T19:10:47.574918] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-09. [2025-09-16T19:10:47.684249] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-09-16T19:10:47.686606] [INFO ] [REMOTE ] KWR: Full fetch complete (18429 records). [2025-09-16T19:10:47.686738] [DEBUG] [VALIDATOR ] KWR: Validating primary data. [2025-09-16T19:10:48.070329] [DEBUG] [VALIDATOR ] KWR: All expected trading days are present. [2025-09-16T19:10:48.070503] [DEBUG] [VALIDATOR ] KWR: Primary data validation passed. [2025-09-16T19:10:48.070934] [INFO ] [MAIN ] KWR: Fetched primary data is valid. [2025-09-16T19:10:48.070986] [DEBUG] [LOCAL ] KWR: Saving primary data. [2025-09-16T19:10:48.071732] [DEBUG] [LOCAL ] KWR: Trimmed 18429 records to 12353 (keeping 150 days of data) [2025-09-16T19:10:48.166980] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-05 (2.19 seconds). [2025-09-16T19:10:48.198109] [DEBUG] [LOCAL ] KWR: Primary data saved (12353 records). [2025-09-16T19:10:48.198248] [INFO ] [MAIN ] KWR: Full history saved. [2025-09-16T19:10:48.198352] [INFO ] [MAIN ] Update Progress: 1233/2369 symbols completed. KWR took 22.2s. Estimated remaining: 2h 0m 28s. Rate: 9.4 symbols/min [2025-09-16T19:10:48.366918] [INFO ] [MAIN ] LAD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:10:48.375940] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-06. [2025-09-16T19:10:48.384761] [DEBUG] [LOCAL ] LAD: Loaded 16862 existing records from primary storage. [2025-09-16T19:10:48.384903] [INFO ] [MAIN ] LAD: Loaded 16862 existing records. [2025-09-16T19:10:48.589964] [WARN ] [MAIN ] LAD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:10:48.591153] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T19:10:48.591382] [INFO ] [MAIN ] LAD: Wiping existing primary data before full refresh. [2025-09-16T19:10:48.594991] [INFO ] [LOCAL ] LAD: Starting deletion of primary data. [2025-09-16T19:10:48.595236] [INFO ] [REMOTE ] KYN: Full fetch complete (33162 records). [2025-09-16T19:10:48.595312] [DEBUG] [VALIDATOR ] KYN: Validating primary data. [2025-09-16T19:10:48.595526] [DEBUG] [LOCAL ] LAD: Deleted primary data file. [2025-09-16T19:10:48.595582] [INFO ] [MAIN ] LAD: Starting full primary refresh. [2025-09-16T19:10:49.175534] [INFO ] [REMOTE ] L: Fetching primary data for 2025-04. [2025-09-16T19:10:49.363866] [DEBUG] [VALIDATOR ] KYN: All expected trading days are present. [2025-09-16T19:10:49.363778] [INFO ] [REMOTE ] LAD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:10:49.364220] [DEBUG] [VALIDATOR ] KYN: Primary data validation passed. [2025-09-16T19:10:49.365042] [INFO ] [MAIN ] KYN: Fetched primary data is valid. [2025-09-16T19:10:49.365095] [DEBUG] [LOCAL ] KYN: Saving primary data. [2025-09-16T19:10:49.366016] [DEBUG] [LOCAL ] KYN: Trimmed 33162 records to 22074 (keeping 150 days of data) [2025-09-16T19:10:49.502263] [DEBUG] [LOCAL ] KYN: Primary data saved (22074 records). [2025-09-16T19:10:49.502400] [INFO ] [MAIN ] KYN: Full history saved. [2025-09-16T19:10:49.502492] [INFO ] [MAIN ] Update Progress: 1234/2369 symbols completed. KYN took 20.9s. Estimated remaining: 2h 0m 17s. Rate: 9.4 symbols/min [2025-09-16T19:10:49.675407] [INFO ] [MAIN ] LADR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:10:49.705131] [DEBUG] [LOCAL ] LADR: Loaded 28214 existing records from primary storage. [2025-09-16T19:10:49.705288] [INFO ] [MAIN ] LADR: Loaded 28214 existing records. [2025-09-16T19:10:49.897635] [WARN ] [MAIN ] LADR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:10:49.897984] [INFO ] [MAIN ] LADR: Wiping existing primary data before full refresh. [2025-09-16T19:10:49.898083] [INFO ] [LOCAL ] LADR: Starting deletion of primary data. [2025-09-16T19:10:49.898779] [DEBUG] [LOCAL ] LADR: Deleted primary data file. [2025-09-16T19:10:49.898834] [INFO ] [MAIN ] LADR: Starting full primary refresh. [2025-09-16T19:10:49.977636] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-02. [2025-09-16T19:10:50.262417] [INFO ] [REMOTE ] LADR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:10:50.291843] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-09-16T19:10:50.774924] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-07. [2025-09-16T19:10:50.893281] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T19:10:51.209488] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-09-16T19:10:51.574953] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-03. [2025-09-16T19:10:52.374894] [INFO ] [REMOTE ] L: Fetching primary data for 2025-05. [2025-09-16T19:10:52.771207] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-09-16T19:10:52.825744] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-09-16T19:10:53.174938] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-02. [2025-09-16T19:10:53.696808] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T19:10:53.974914] [INFO ] [REMOTE ] L: Fetching primary data for 2025-06. [2025-09-16T19:10:54.198733] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-09-16T19:10:54.774953] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-04. [2025-09-16T19:10:55.135740] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-09-16T19:10:55.574936] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-03. [2025-09-16T19:10:55.838305] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-09-16T19:10:56.374914] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-08. [2025-09-16T19:10:57.011583] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T19:10:57.174912] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-04. [2025-09-16T19:10:57.974908] [INFO ] [REMOTE ] L: Fetching primary data for 2025-07. [2025-09-16T19:10:58.518668] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-09-16T19:10:58.733395] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T19:10:58.774859] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-05. [2025-09-16T19:10:59.140148] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T19:10:59.574911] [INFO ] [REMOTE ] L: Fetching primary data for 2025-08. [2025-09-16T19:11:00.093874] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T19:11:00.374902] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-09. [2025-09-16T19:11:01.169225] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T19:11:01.177306] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-05. [2025-09-16T19:11:01.974900] [INFO ] [REMOTE ] L: Fetching primary data for 2025-09. [2025-09-16T19:11:02.299848] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-09-16T19:11:02.306292] [INFO ] [REMOTE ] LAC: Full fetch complete (56835 records). [2025-09-16T19:11:02.306457] [DEBUG] [VALIDATOR ] LAC: Validating primary data. [2025-09-16T19:11:02.663860] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-09-16T19:11:02.718819] [DEBUG] [VALIDATOR ] LAC: All expected trading days are present. [2025-09-16T19:11:02.718900] [DEBUG] [VALIDATOR ] LAC: Primary data validation passed. [2025-09-16T19:11:02.719777] [INFO ] [MAIN ] LAC: Fetched primary data is valid. [2025-09-16T19:11:02.719836] [DEBUG] [LOCAL ] LAC: Saving primary data. [2025-09-16T19:11:02.720993] [DEBUG] [LOCAL ] LAC: Trimmed 56835 records to 39040 (keeping 150 days of data) [2025-09-16T19:11:02.778716] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-06. [2025-09-16T19:11:02.951210] [DEBUG] [LOCAL ] LAC: Primary data saved (39040 records). [2025-09-16T19:11:02.951376] [INFO ] [MAIN ] LAC: Full history saved. [2025-09-16T19:11:02.951463] [INFO ] [MAIN ] Update Progress: 1235/2369 symbols completed. LAC took 28.4s. Estimated remaining: 2h 0m 17s. Rate: 9.4 symbols/min [2025-09-16T19:11:03.126505] [INFO ] [MAIN ] LANV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:03.129915] [DEBUG] [LOCAL ] LANV: Loaded 1908 existing records from primary storage. [2025-09-16T19:11:03.129995] [INFO ] [MAIN ] LANV: Loaded 1908 existing records. [2025-09-16T19:11:03.307307] [WARN ] [MAIN ] LANV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:03.307725] [INFO ] [MAIN ] LANV: Wiping existing primary data before full refresh. [2025-09-16T19:11:03.307809] [INFO ] [LOCAL ] LANV: Starting deletion of primary data. [2025-09-16T19:11:03.308104] [DEBUG] [LOCAL ] LANV: Deleted primary data file. [2025-09-16T19:11:03.308238] [INFO ] [MAIN ] LANV: Starting full primary refresh. [2025-09-16T19:11:03.428411] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-09-16T19:11:03.494079] [INFO ] [REMOTE ] L: Full fetch complete (44518 records). [2025-09-16T19:11:03.510420] [DEBUG] [VALIDATOR ] L: Validating primary data. [2025-09-16T19:11:03.578956] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-06. [2025-09-16T19:11:03.856635] [INFO ] [REMOTE ] LANV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:04.106714] [DEBUG] [VALIDATOR ] L: All expected trading days are present. [2025-09-16T19:11:04.106801] [DEBUG] [VALIDATOR ] L: Primary data validation passed. [2025-09-16T19:11:04.107505] [INFO ] [MAIN ] L: Fetched primary data is valid. [2025-09-16T19:11:04.107558] [DEBUG] [LOCAL ] L: Saving primary data. [2025-09-16T19:11:04.108660] [DEBUG] [LOCAL ] L: Trimmed 44518 records to 30321 (keeping 150 days of data) [2025-09-16T19:11:04.312481] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-09-16T19:11:04.338251] [DEBUG] [LOCAL ] L: Primary data saved (30321 records). [2025-09-16T19:11:04.338387] [INFO ] [MAIN ] L: Full history saved. [2025-09-16T19:11:04.339116] [INFO ] [MAIN ] Update Progress: 1236/2369 symbols completed. L took 31.3s. Estimated remaining: 2h 0m 6s. Rate: 9.4 symbols/min [2025-09-16T19:11:04.381426] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-02. [2025-09-16T19:11:04.510596] [INFO ] [MAIN ] LAR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:04.545427] [DEBUG] [LOCAL ] LAR: Loaded 33317 existing records from primary storage. [2025-09-16T19:11:04.545572] [INFO ] [MAIN ] LAR: Loaded 33317 existing records. [2025-09-16T19:11:04.719988] [WARN ] [MAIN ] LAR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:04.720252] [INFO ] [MAIN ] LAR: Wiping existing primary data before full refresh. [2025-09-16T19:11:04.720320] [INFO ] [LOCAL ] LAR: Starting deletion of primary data. [2025-09-16T19:11:04.721152] [DEBUG] [LOCAL ] LAR: Deleted primary data file. [2025-09-16T19:11:04.721214] [INFO ] [MAIN ] LAR: Starting full primary refresh. [2025-09-16T19:11:04.952664] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-09-16T19:11:05.103977] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-09-16T19:11:05.109035] [INFO ] [REMOTE ] LAR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:05.174911] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-07. [2025-09-16T19:11:05.974944] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-03. [2025-09-16T19:11:06.519993] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-09-16T19:11:06.726086] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-09-16T19:11:06.774850] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-08. [2025-09-16T19:11:07.574962] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-07. [2025-09-16T19:11:08.170267] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-09-16T19:11:08.374904] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-02. [2025-09-16T19:11:08.596545] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T19:11:09.174938] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-04. [2025-09-16T19:11:09.602970] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-09-16T19:11:09.982315] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-09. [2025-09-16T19:11:09.999265] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T19:11:10.774894] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-05. [2025-09-16T19:11:11.234114] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-09-16T19:11:11.556336] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T19:11:11.561178] [INFO ] [REMOTE ] LADR: Full fetch complete (41424 records). [2025-09-16T19:11:11.561279] [DEBUG] [VALIDATOR ] LADR: Validating primary data. [2025-09-16T19:11:11.582151] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-08. [2025-09-16T19:11:11.940673] [DEBUG] [VALIDATOR ] LADR: All expected trading days are present. [2025-09-16T19:11:11.940770] [DEBUG] [VALIDATOR ] LADR: Primary data validation passed. [2025-09-16T19:11:11.941434] [INFO ] [MAIN ] LADR: Fetched primary data is valid. [2025-09-16T19:11:11.941480] [DEBUG] [LOCAL ] LADR: Saving primary data. [2025-09-16T19:11:11.942567] [DEBUG] [LOCAL ] LADR: Trimmed 41424 records to 28533 (keeping 150 days of data) [2025-09-16T19:11:12.114826] [DEBUG] [LOCAL ] LADR: Primary data saved (28533 records). [2025-09-16T19:11:12.114973] [INFO ] [MAIN ] LADR: Full history saved. [2025-09-16T19:11:12.115059] [INFO ] [MAIN ] Update Progress: 1237/2369 symbols completed. LADR took 22.6s. Estimated remaining: 2h 0m 1s. Rate: 9.4 symbols/min [2025-09-16T19:11:12.374943] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-03. [2025-09-16T19:11:12.753670] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T19:11:13.174905] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-09. [2025-09-16T19:11:13.723950] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-09-16T19:11:13.974913] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-04. [2025-09-16T19:11:14.277404] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-09-16T19:11:14.280453] [INFO ] [REMOTE ] LAD: Full fetch complete (26576 records). [2025-09-16T19:11:14.280593] [DEBUG] [VALIDATOR ] LAD: Validating primary data. [2025-09-16T19:11:14.664300] [DEBUG] [VALIDATOR ] LAD: All expected trading days are present. [2025-09-16T19:11:14.664427] [DEBUG] [VALIDATOR ] LAD: Primary data validation passed. [2025-09-16T19:11:14.665095] [INFO ] [MAIN ] LAD: Fetched primary data is valid. [2025-09-16T19:11:14.665144] [DEBUG] [LOCAL ] LAD: Saving primary data. [2025-09-16T19:11:14.666089] [DEBUG] [LOCAL ] LAD: Trimmed 26576 records to 16994 (keeping 150 days of data) [2025-09-16T19:11:14.775868] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-06. [2025-09-16T19:11:14.778892] [DEBUG] [LOCAL ] LAD: Primary data saved (16994 records). [2025-09-16T19:11:14.780023] [INFO ] [MAIN ] LAD: Full history saved. [2025-09-16T19:11:14.780127] [INFO ] [MAIN ] Update Progress: 1238/2369 symbols completed. LAD took 26.6s. Estimated remaining: 1h 59m 51s. Rate: 9.4 symbols/min [2025-09-16T19:11:15.202956] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-09-16T19:11:15.574969] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-07. [2025-09-16T19:11:15.679530] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-09-16T19:11:16.139705] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-09-16T19:11:16.374900] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-08. [2025-09-16T19:11:16.931017] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-09-16T19:11:17.174958] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-09. [2025-09-16T19:11:17.769860] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-09-16T19:11:17.770951] [INFO ] [REMOTE ] LANV: Full fetch complete (2814 records). [2025-09-16T19:11:17.771020] [DEBUG] [VALIDATOR ] LANV: Validating primary data. [2025-09-16T19:11:17.980459] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-05. [2025-09-16T19:11:18.127102] [DEBUG] [VALIDATOR ] LANV: All expected trading days are present. [2025-09-16T19:11:18.127181] [DEBUG] [VALIDATOR ] LANV: Primary data validation passed. [2025-09-16T19:11:18.127293] [INFO ] [MAIN ] LANV: Fetched primary data is valid. [2025-09-16T19:11:18.127333] [DEBUG] [LOCAL ] LANV: Saving primary data. [2025-09-16T19:11:18.127699] [DEBUG] [LOCAL ] LANV: Trimmed 2814 records to 1929 (keeping 150 days of data) [2025-09-16T19:11:18.138674] [DEBUG] [LOCAL ] LANV: Primary data saved (1929 records). [2025-09-16T19:11:18.138737] [INFO ] [MAIN ] LANV: Full history saved. [2025-09-16T19:11:18.138819] [INFO ] [MAIN ] Update Progress: 1239/2369 symbols completed. LANV took 15.2s. Estimated remaining: 1h 59m 42s. Rate: 9.4 symbols/min [2025-09-16T19:11:19.632332] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-09-16T19:11:19.641870] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-06. [2025-09-16T19:11:21.201118] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T19:11:21.210305] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-07. [2025-09-16T19:11:22.895508] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T19:11:22.906991] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-08. [2025-09-16T19:11:24.339477] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-09-16T19:11:24.349713] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-09. [2025-09-16T19:11:25.918481] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T19:11:25.924341] [INFO ] [REMOTE ] LAR: Full fetch complete (48912 records). [2025-09-16T19:11:25.924469] [DEBUG] [VALIDATOR ] LAR: Validating primary data. [2025-09-16T19:11:26.308304] [DEBUG] [VALIDATOR ] LAR: All expected trading days are present. [2025-09-16T19:11:26.308386] [DEBUG] [VALIDATOR ] LAR: Primary data validation passed. [2025-09-16T19:11:26.309143] [INFO ] [MAIN ] LAR: Fetched primary data is valid. [2025-09-16T19:11:26.309189] [DEBUG] [LOCAL ] LAR: Saving primary data. [2025-09-16T19:11:26.310285] [DEBUG] [LOCAL ] LAR: Trimmed 48912 records to 33661 (keeping 150 days of data) [2025-09-16T19:11:26.507275] [DEBUG] [LOCAL ] LAR: Primary data saved (33661 records). [2025-09-16T19:11:26.507429] [INFO ] [MAIN ] LAR: Full history saved. [2025-09-16T19:11:26.508135] [INFO ] [MAIN ] Update Progress: 1240/2369 symbols completed. LAR took 22.2s. Estimated remaining: 1h 59m 38s. Rate: 9.4 symbols/min [2025-09-16T19:11:27.169084] [INFO ] [MAIN ] LAW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:27.201521] [INFO ] [MAIN ] LAZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:27.228332] [INFO ] [MAIN ] LB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:27.261699] [INFO ] [MAIN ] LBRT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:27.274029] [DEBUG] [LOCAL ] LAW: Loaded 10545 existing records from primary storage. [2025-09-16T19:11:27.274343] [INFO ] [MAIN ] LAW: Loaded 10545 existing records. [2025-09-16T19:11:27.464016] [DEBUG] [LOCAL ] LB: Loaded 27501 existing records from primary storage. [2025-09-16T19:11:27.511580] [DEBUG] [LOCAL ] LAZ: Loaded 33117 existing records from primary storage. [2025-09-16T19:11:27.511141] [INFO ] [MAIN ] LB: Loaded 27501 existing records. [2025-09-16T19:11:27.510693] [WARN ] [MAIN ] LAW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:27.511730] [INFO ] [MAIN ] LAZ: Loaded 33117 existing records. [2025-09-16T19:11:27.573256] [INFO ] [MAIN ] LAW: Wiping existing primary data before full refresh. [2025-09-16T19:11:27.583707] [INFO ] [LOCAL ] LAW: Starting deletion of primary data. [2025-09-16T19:11:27.589561] [DEBUG] [LOCAL ] LBRT: Loaded 39012 existing records from primary storage. [2025-09-16T19:11:27.632249] [DEBUG] [LOCAL ] LAW: Deleted primary data file. [2025-09-16T19:11:27.650867] [INFO ] [MAIN ] LBRT: Loaded 39012 existing records. [2025-09-16T19:11:27.669709] [INFO ] [MAIN ] LAW: Starting full primary refresh. [2025-09-16T19:11:28.246491] [WARN ] [MAIN ] LAZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:28.264368] [WARN ] [MAIN ] LB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:28.330946] [INFO ] [MAIN ] LB: Wiping existing primary data before full refresh. [2025-09-16T19:11:28.336586] [INFO ] [MAIN ] LAZ: Wiping existing primary data before full refresh. [2025-09-16T19:11:28.390543] [INFO ] [LOCAL ] LAZ: Starting deletion of primary data. [2025-09-16T19:11:28.400940] [INFO ] [LOCAL ] LB: Starting deletion of primary data. [2025-09-16T19:11:28.415532] [WARN ] [MAIN ] LBRT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:28.435150] [DEBUG] [LOCAL ] LAZ: Deleted primary data file. [2025-09-16T19:11:28.435269] [DEBUG] [LOCAL ] LB: Deleted primary data file. [2025-09-16T19:11:28.435329] [INFO ] [MAIN ] LBRT: Wiping existing primary data before full refresh. [2025-09-16T19:11:28.445782] [INFO ] [MAIN ] LAZ: Starting full primary refresh. [2025-09-16T19:11:28.485011] [INFO ] [MAIN ] LB: Starting full primary refresh. [2025-09-16T19:11:28.467201] [INFO ] [LOCAL ] LBRT: Starting deletion of primary data. [2025-09-16T19:11:28.539864] [INFO ] [REMOTE ] LAW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:28.563143] [DEBUG] [LOCAL ] LBRT: Deleted primary data file. [2025-09-16T19:11:28.624761] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-02. [2025-09-16T19:11:28.683476] [INFO ] [MAIN ] LBRT: Starting full primary refresh. [2025-09-16T19:11:29.604184] [INFO ] [REMOTE ] LAZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:29.675952] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-02. [2025-09-16T19:11:29.707160] [INFO ] [REMOTE ] LBRT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:29.742176] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-02. [2025-09-16T19:11:29.727422] [INFO ] [REMOTE ] LB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:29.743275] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-02. [2025-09-16T19:11:29.747784] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-09-16T19:11:29.754284] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-03. [2025-09-16T19:11:30.619221] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-09-16T19:11:30.623115] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-04. [2025-09-16T19:11:31.188957] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-09-16T19:11:31.193015] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-05. [2025-09-16T19:11:31.217170] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T19:11:31.226611] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-03. [2025-09-16T19:11:31.361910] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T19:11:31.371023] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-03. [2025-09-16T19:11:31.416565] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-09-16T19:11:31.426690] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-03. [2025-09-16T19:11:32.171419] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-09-16T19:11:32.175113] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-06. [2025-09-16T19:11:32.828890] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T19:11:32.837618] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-04. [2025-09-16T19:11:33.013278] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T19:11:33.023294] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-04. [2025-09-16T19:11:33.066717] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T19:11:33.080084] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-04. [2025-09-16T19:11:33.096410] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-09-16T19:11:33.174897] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-07. [2025-09-16T19:11:34.170070] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T19:11:34.173850] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-08. [2025-09-16T19:11:34.317427] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T19:11:34.635085] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-09-16T19:11:34.774873] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-05. [2025-09-16T19:11:34.861837] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-09-16T19:11:34.953990] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-09-16T19:11:35.574927] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-05. [2025-09-16T19:11:36.374918] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-05. [2025-09-16T19:11:36.431436] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T19:11:36.885300] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T19:11:37.174916] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-06. [2025-09-16T19:11:37.395176] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-09-16T19:11:37.974893] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-06. [2025-09-16T19:11:38.435728] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-09-16T19:11:38.774830] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-06. [2025-09-16T19:11:39.574868] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-07. [2025-09-16T19:11:39.678051] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T19:11:40.186325] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-09-16T19:11:40.374855] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-07. [2025-09-16T19:11:41.165738] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T19:11:41.175485] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-09. [2025-09-16T19:11:41.579612] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T19:11:41.800758] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-09-16T19:11:41.802964] [INFO ] [REMOTE ] LAW: Full fetch complete (15466 records). [2025-09-16T19:11:41.803079] [DEBUG] [VALIDATOR ] LAW: Validating primary data. [2025-09-16T19:11:41.980272] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-08. [2025-09-16T19:11:42.172521] [DEBUG] [VALIDATOR ] LAW: All expected trading days are present. [2025-09-16T19:11:42.172601] [DEBUG] [VALIDATOR ] LAW: Primary data validation passed. [2025-09-16T19:11:42.173050] [INFO ] [MAIN ] LAW: Fetched primary data is valid. [2025-09-16T19:11:42.173122] [DEBUG] [LOCAL ] LAW: Saving primary data. [2025-09-16T19:11:42.174196] [DEBUG] [LOCAL ] LAW: Trimmed 15466 records to 10628 (keeping 150 days of data) [2025-09-16T19:11:42.236490] [DEBUG] [LOCAL ] LAW: Primary data saved (10628 records). [2025-09-16T19:11:42.236663] [INFO ] [MAIN ] LAW: Full history saved. [2025-09-16T19:11:42.236806] [INFO ] [MAIN ] Update Progress: 1241/2369 symbols completed. LAW took 15.7s. Estimated remaining: 1h 59m 40s. Rate: 9.4 symbols/min [2025-09-16T19:11:42.413131] [INFO ] [MAIN ] LC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:42.456837] [DEBUG] [LOCAL ] LC: Loaded 35229 existing records from primary storage. [2025-09-16T19:11:42.457000] [INFO ] [MAIN ] LC: Loaded 35229 existing records. [2025-09-16T19:11:42.648341] [WARN ] [MAIN ] LC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:42.648617] [INFO ] [MAIN ] LC: Wiping existing primary data before full refresh. [2025-09-16T19:11:42.648689] [INFO ] [LOCAL ] LC: Starting deletion of primary data. [2025-09-16T19:11:42.649410] [DEBUG] [LOCAL ] LC: Deleted primary data file. [2025-09-16T19:11:42.649470] [INFO ] [MAIN ] LC: Starting full primary refresh. [2025-09-16T19:11:42.780148] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-07. [2025-09-16T19:11:43.026990] [INFO ] [REMOTE ] LC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:43.556529] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-09-16T19:11:43.574844] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-09. [2025-09-16T19:11:44.374913] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-08. [2025-09-16T19:11:44.769815] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-09-16T19:11:44.995149] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T19:11:44.999666] [INFO ] [REMOTE ] LB: Full fetch complete (41404 records). [2025-09-16T19:11:44.999763] [DEBUG] [VALIDATOR ] LB: Validating primary data. [2025-09-16T19:11:45.179711] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-08. [2025-09-16T19:11:45.382857] [DEBUG] [VALIDATOR ] LB: All expected trading days are present. [2025-09-16T19:11:45.382958] [DEBUG] [VALIDATOR ] LB: Primary data validation passed. [2025-09-16T19:11:45.383747] [INFO ] [MAIN ] LB: Fetched primary data is valid. [2025-09-16T19:11:45.383853] [DEBUG] [LOCAL ] LB: Saving primary data. [2025-09-16T19:11:45.384815] [DEBUG] [LOCAL ] LB: Trimmed 41404 records to 27672 (keeping 150 days of data) [2025-09-16T19:11:45.561147] [DEBUG] [LOCAL ] LB: Primary data saved (27672 records). [2025-09-16T19:11:45.561306] [INFO ] [MAIN ] LB: Full history saved. [2025-09-16T19:11:45.561419] [INFO ] [MAIN ] Update Progress: 1242/2369 symbols completed. LB took 19.1s. Estimated remaining: 1h 59m 31s. Rate: 9.4 symbols/min [2025-09-16T19:11:45.735153] [INFO ] [MAIN ] LCII: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:45.755813] [DEBUG] [LOCAL ] LCII: Loaded 20124 existing records from primary storage. [2025-09-16T19:11:45.755962] [INFO ] [MAIN ] LCII: Loaded 20124 existing records. [2025-09-16T19:11:45.932941] [WARN ] [MAIN ] LCII: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:45.933233] [INFO ] [MAIN ] LCII: Wiping existing primary data before full refresh. [2025-09-16T19:11:45.933298] [INFO ] [LOCAL ] LCII: Starting deletion of primary data. [2025-09-16T19:11:45.934188] [DEBUG] [LOCAL ] LCII: Deleted primary data file. [2025-09-16T19:11:45.934303] [INFO ] [MAIN ] LCII: Starting full primary refresh. [2025-09-16T19:11:45.997875] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-02. [2025-09-16T19:11:46.253565] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-09-16T19:11:46.350562] [INFO ] [REMOTE ] LCII: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:46.676435] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T19:11:46.774865] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-02. [2025-09-16T19:11:47.574984] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-09. [2025-09-16T19:11:47.760216] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-09-16T19:11:47.910858] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-09-16T19:11:48.374901] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-03. [2025-09-16T19:11:49.060321] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T19:11:49.067343] [INFO ] [REMOTE ] LAZ: Full fetch complete (49565 records). [2025-09-16T19:11:49.067506] [DEBUG] [VALIDATOR ] LAZ: Validating primary data. [2025-09-16T19:11:49.180021] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-09. [2025-09-16T19:11:49.454326] [DEBUG] [VALIDATOR ] LAZ: All expected trading days are present. [2025-09-16T19:11:49.454449] [DEBUG] [VALIDATOR ] LAZ: Primary data validation passed. [2025-09-16T19:11:49.455246] [INFO ] [MAIN ] LAZ: Fetched primary data is valid. [2025-09-16T19:11:49.455303] [DEBUG] [LOCAL ] LAZ: Saving primary data. [2025-09-16T19:11:49.456681] [DEBUG] [LOCAL ] LAZ: Trimmed 49565 records to 33463 (keeping 150 days of data) [2025-09-16T19:11:49.669453] [DEBUG] [LOCAL ] LAZ: Primary data saved (33463 records). [2025-09-16T19:11:49.669606] [INFO ] [MAIN ] LAZ: Full history saved. [2025-09-16T19:11:49.669719] [INFO ] [MAIN ] Update Progress: 1243/2369 symbols completed. LAZ took 23.2s. Estimated remaining: 1h 59m 22s. Rate: 9.4 symbols/min [2025-09-16T19:11:49.842924] [INFO ] [MAIN ] LDI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:49.876012] [DEBUG] [LOCAL ] LDI: Loaded 31881 existing records from primary storage. [2025-09-16T19:11:49.876184] [INFO ] [MAIN ] LDI: Loaded 31881 existing records. [2025-09-16T19:11:49.978968] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-03. [2025-09-16T19:11:50.057471] [WARN ] [MAIN ] LDI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:50.057876] [INFO ] [MAIN ] LDI: Wiping existing primary data before full refresh. [2025-09-16T19:11:50.057963] [INFO ] [LOCAL ] LDI: Starting deletion of primary data. [2025-09-16T19:11:50.058582] [DEBUG] [LOCAL ] LDI: Deleted primary data file. [2025-09-16T19:11:50.058629] [INFO ] [MAIN ] LDI: Starting full primary refresh. [2025-09-16T19:11:50.384196] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-09-16T19:11:50.481132] [INFO ] [REMOTE ] LDI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:50.774926] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-04. [2025-09-16T19:11:51.047710] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-09-16T19:11:51.055629] [INFO ] [REMOTE ] LBRT: Full fetch complete (57542 records). [2025-09-16T19:11:51.055791] [DEBUG] [VALIDATOR ] LBRT: Validating primary data. [2025-09-16T19:11:51.440614] [DEBUG] [VALIDATOR ] LBRT: All expected trading days are present. [2025-09-16T19:11:51.440756] [DEBUG] [VALIDATOR ] LBRT: Primary data validation passed. [2025-09-16T19:11:51.441669] [INFO ] [MAIN ] LBRT: Fetched primary data is valid. [2025-09-16T19:11:51.441755] [DEBUG] [LOCAL ] LBRT: Saving primary data. [2025-09-16T19:11:51.443187] [DEBUG] [LOCAL ] LBRT: Trimmed 57542 records to 39400 (keeping 150 days of data) [2025-09-16T19:11:51.524091] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T19:11:51.574927] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-04. [2025-09-16T19:11:51.717821] [DEBUG] [LOCAL ] LBRT: Primary data saved (39400 records). [2025-09-16T19:11:51.718026] [INFO ] [MAIN ] LBRT: Full history saved. [2025-09-16T19:11:51.718127] [INFO ] [MAIN ] Update Progress: 1244/2369 symbols completed. LBRT took 25.2s. Estimated remaining: 1h 59m 12s. Rate: 9.4 symbols/min [2025-09-16T19:11:51.890312] [INFO ] [MAIN ] LDOS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:11:51.926652] [DEBUG] [LOCAL ] LDOS: Loaded 33182 existing records from primary storage. [2025-09-16T19:11:51.926803] [INFO ] [MAIN ] LDOS: Loaded 33182 existing records. [2025-09-16T19:11:52.101305] [WARN ] [MAIN ] LDOS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:11:52.101557] [INFO ] [MAIN ] LDOS: Wiping existing primary data before full refresh. [2025-09-16T19:11:52.101610] [INFO ] [LOCAL ] LDOS: Starting deletion of primary data. [2025-09-16T19:11:52.102433] [DEBUG] [LOCAL ] LDOS: Deleted primary data file. [2025-09-16T19:11:52.102480] [INFO ] [MAIN ] LDOS: Starting full primary refresh. [2025-09-16T19:11:52.379281] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-02. [2025-09-16T19:11:52.457300] [INFO ] [REMOTE ] LDOS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:11:52.503311] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T19:11:53.055882] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T19:11:53.174952] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-02. [2025-09-16T19:11:53.684389] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T19:11:53.974909] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-03. [2025-09-16T19:11:54.774875] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-05. [2025-09-16T19:11:54.880865] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-09-16T19:11:55.100757] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-09-16T19:11:55.574898] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-04. [2025-09-16T19:11:56.374930] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-03. [2025-09-16T19:11:56.502960] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T19:11:57.065580] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-09-16T19:11:57.174903] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-05. [2025-09-16T19:11:57.974925] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-05. [2025-09-16T19:11:58.110498] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T19:11:58.590437] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-09-16T19:11:58.774924] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-06. [2025-09-16T19:11:59.018919] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-09-16T19:11:59.574938] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-06. [2025-09-16T19:12:00.259573] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T19:12:00.374876] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-06. [2025-09-16T19:12:00.818846] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-09-16T19:12:01.174930] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-07. [2025-09-16T19:12:01.951329] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-09-16T19:12:01.974868] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-07. [2025-09-16T19:12:02.104051] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-09-16T19:12:02.774902] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-08. [2025-09-16T19:12:03.574889] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-04. [2025-09-16T19:12:03.829466] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T19:12:03.972451] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-09-16T19:12:04.374917] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-08. [2025-09-16T19:12:05.022753] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T19:12:05.174949] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-07. [2025-09-16T19:12:05.974898] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-05. [2025-09-16T19:12:06.023891] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T19:12:06.774923] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-09. [2025-09-16T19:12:06.865259] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-09-16T19:12:07.575030] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-08. [2025-09-16T19:12:07.756079] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-09-16T19:12:08.342390] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T19:12:08.348398] [INFO ] [REMOTE ] LC: Full fetch complete (52437 records). [2025-09-16T19:12:08.348562] [DEBUG] [VALIDATOR ] LC: Validating primary data. [2025-09-16T19:12:08.380962] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-06. [2025-09-16T19:12:08.761619] [DEBUG] [VALIDATOR ] LC: All expected trading days are present. [2025-09-16T19:12:08.761718] [DEBUG] [VALIDATOR ] LC: Primary data validation passed. [2025-09-16T19:12:08.762663] [INFO ] [MAIN ] LC: Fetched primary data is valid. [2025-09-16T19:12:08.762709] [DEBUG] [LOCAL ] LC: Saving primary data. [2025-09-16T19:12:08.763944] [DEBUG] [LOCAL ] LC: Trimmed 52437 records to 35580 (keeping 150 days of data) [2025-09-16T19:12:09.002043] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-09-16T19:12:09.023046] [DEBUG] [LOCAL ] LC: Primary data saved (35580 records). [2025-09-16T19:12:09.023210] [INFO ] [MAIN ] LC: Full history saved. [2025-09-16T19:12:09.023297] [INFO ] [MAIN ] Update Progress: 1245/2369 symbols completed. LC took 26.8s. Estimated remaining: 1h 59m 16s. Rate: 9.4 symbols/min [2025-09-16T19:12:09.182022] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-09. [2025-09-16T19:12:09.201088] [INFO ] [MAIN ] LDP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:12:09.213585] [DEBUG] [LOCAL ] LDP: Loaded 9342 existing records from primary storage. [2025-09-16T19:12:09.213778] [INFO ] [MAIN ] LDP: Loaded 9342 existing records. [2025-09-16T19:12:09.408257] [WARN ] [MAIN ] LDP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:12:09.408575] [INFO ] [MAIN ] LDP: Wiping existing primary data before full refresh. [2025-09-16T19:12:09.408634] [INFO ] [LOCAL ] LDP: Starting deletion of primary data. [2025-09-16T19:12:09.409095] [DEBUG] [LOCAL ] LDP: Deleted primary data file. [2025-09-16T19:12:09.409150] [INFO ] [MAIN ] LDP: Starting full primary refresh. [2025-09-16T19:12:09.841487] [INFO ] [REMOTE ] LDP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:12:09.927052] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-09-16T19:12:09.974932] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-07. [2025-09-16T19:12:10.319935] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T19:12:10.322850] [INFO ] [REMOTE ] LCII: Full fetch complete (30976 records). [2025-09-16T19:12:10.322943] [DEBUG] [VALIDATOR ] LCII: Validating primary data. [2025-09-16T19:12:10.701007] [DEBUG] [VALIDATOR ] LCII: All expected trading days are present. [2025-09-16T19:12:10.701086] [DEBUG] [VALIDATOR ] LCII: Primary data validation passed. [2025-09-16T19:12:10.701701] [INFO ] [MAIN ] LCII: Fetched primary data is valid. [2025-09-16T19:12:10.701744] [DEBUG] [LOCAL ] LCII: Saving primary data. [2025-09-16T19:12:10.702429] [DEBUG] [LOCAL ] LCII: Trimmed 30976 records to 20266 (keeping 150 days of data) [2025-09-16T19:12:10.792193] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-09. [2025-09-16T19:12:10.828809] [DEBUG] [LOCAL ] LCII: Primary data saved (20266 records). [2025-09-16T19:12:10.828936] [INFO ] [MAIN ] LCII: Full history saved. [2025-09-16T19:12:10.829036] [INFO ] [MAIN ] Update Progress: 1246/2369 symbols completed. LCII took 25.3s. Estimated remaining: 1h 59m 5s. Rate: 9.4 symbols/min [2025-09-16T19:12:10.997071] [INFO ] [MAIN ] LEA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:12:11.031474] [DEBUG] [LOCAL ] LEA: Loaded 30256 existing records from primary storage. [2025-09-16T19:12:11.031612] [INFO ] [MAIN ] LEA: Loaded 30256 existing records. [2025-09-16T19:12:11.209692] [WARN ] [MAIN ] LEA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:12:11.209936] [INFO ] [MAIN ] LEA: Wiping existing primary data before full refresh. [2025-09-16T19:12:11.209982] [INFO ] [LOCAL ] LEA: Starting deletion of primary data. [2025-09-16T19:12:11.210744] [DEBUG] [LOCAL ] LEA: Deleted primary data file. [2025-09-16T19:12:11.210804] [INFO ] [MAIN ] LEA: Starting full primary refresh. [2025-09-16T19:12:11.576774] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-02. [2025-09-16T19:12:11.582733] [INFO ] [REMOTE ] LEA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:12:11.616119] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T19:12:12.374874] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-02. [2025-09-16T19:12:12.434904] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-09-16T19:12:12.686084] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-09-16T19:12:12.692390] [INFO ] [REMOTE ] LDI: Full fetch complete (46391 records). [2025-09-16T19:12:12.692543] [DEBUG] [VALIDATOR ] LDI: Validating primary data. [2025-09-16T19:12:13.066097] [DEBUG] [VALIDATOR ] LDI: All expected trading days are present. [2025-09-16T19:12:13.066176] [DEBUG] [VALIDATOR ] LDI: Primary data validation passed. [2025-09-16T19:12:13.066902] [INFO ] [MAIN ] LDI: Fetched primary data is valid. [2025-09-16T19:12:13.066947] [DEBUG] [LOCAL ] LDI: Saving primary data. [2025-09-16T19:12:13.067967] [DEBUG] [LOCAL ] LDI: Trimmed 46391 records to 32271 (keeping 150 days of data) [2025-09-16T19:12:13.180392] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-03. [2025-09-16T19:12:13.253342] [DEBUG] [LOCAL ] LDI: Primary data saved (32271 records). [2025-09-16T19:12:13.253491] [INFO ] [MAIN ] LDI: Full history saved. [2025-09-16T19:12:13.253588] [INFO ] [MAIN ] Update Progress: 1247/2369 symbols completed. LDI took 23.6s. Estimated remaining: 1h 58m 55s. Rate: 9.4 symbols/min [2025-09-16T19:12:13.422662] [INFO ] [MAIN ] LEG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:12:13.422833] [DEBUG] [LOCAL ] LEG: No existing data file found in primary storage. [2025-09-16T19:12:13.422871] [WARN ] [MAIN ] LEG: No existing data found, performing full refresh instead. [2025-09-16T19:12:13.422995] [INFO ] [MAIN ] LEG: Wiping existing primary data before full refresh. [2025-09-16T19:12:13.423036] [INFO ] [LOCAL ] LEG: Starting deletion of primary data. [2025-09-16T19:12:13.423079] [DEBUG] [LOCAL ] LEG: No primary data file found to delete. [2025-09-16T19:12:13.423113] [INFO ] [MAIN ] LEG: Starting full primary refresh. [2025-09-16T19:12:13.794661] [INFO ] [REMOTE ] LEG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:12:13.817602] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T19:12:13.974908] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-08. [2025-09-16T19:12:14.109368] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-09-16T19:12:14.774928] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-04. [2025-09-16T19:12:15.548373] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T19:12:15.574883] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-02. [2025-09-16T19:12:15.668102] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-09-16T19:12:16.374907] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-03. [2025-09-16T19:12:17.033516] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-09-16T19:12:17.174920] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-09. [2025-09-16T19:12:17.977067] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-03. [2025-09-16T19:12:17.982452] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T19:12:18.642944] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T19:12:18.648436] [INFO ] [REMOTE ] LDOS: Full fetch complete (50528 records). [2025-09-16T19:12:18.648603] [DEBUG] [VALIDATOR ] LDOS: Validating primary data. [2025-09-16T19:12:18.790154] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-04. [2025-09-16T19:12:19.025186] [DEBUG] [VALIDATOR ] LDOS: All expected trading days are present. [2025-09-16T19:12:19.025267] [DEBUG] [VALIDATOR ] LDOS: Primary data validation passed. [2025-09-16T19:12:19.026096] [INFO ] [MAIN ] LDOS: Fetched primary data is valid. [2025-09-16T19:12:19.026162] [DEBUG] [LOCAL ] LDOS: Saving primary data. [2025-09-16T19:12:19.027229] [DEBUG] [LOCAL ] LDOS: Trimmed 50528 records to 33529 (keeping 150 days of data) [2025-09-16T19:12:19.258818] [DEBUG] [LOCAL ] LDOS: Primary data saved (33529 records). [2025-09-16T19:12:19.258958] [INFO ] [MAIN ] LDOS: Full history saved. [2025-09-16T19:12:19.259962] [INFO ] [MAIN ] Update Progress: 1248/2369 symbols completed. LDOS took 27.5s. Estimated remaining: 1h 58m 49s. Rate: 9.4 symbols/min [2025-09-16T19:12:19.431491] [INFO ] [MAIN ] LEN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:12:19.474503] [DEBUG] [LOCAL ] LEN: Loaded 39223 existing records from primary storage. [2025-09-16T19:12:19.474661] [INFO ] [MAIN ] LEN: Loaded 39223 existing records. [2025-09-16T19:12:19.576864] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-05. [2025-09-16T19:12:19.655094] [WARN ] [MAIN ] LEN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:12:19.655401] [INFO ] [MAIN ] LEN: Wiping existing primary data before full refresh. [2025-09-16T19:12:19.655501] [INFO ] [LOCAL ] LEN: Starting deletion of primary data. [2025-09-16T19:12:19.656328] [DEBUG] [LOCAL ] LEN: Deleted primary data file. [2025-09-16T19:12:19.656389] [INFO ] [MAIN ] LEN: Starting full primary refresh. [2025-09-16T19:12:20.040150] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-03 (2.06 seconds). [2025-09-16T19:12:20.038466] [INFO ] [REMOTE ] LEN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:12:20.319285] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-09-16T19:12:20.374863] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-05. [2025-09-16T19:12:20.499962] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-09-16T19:12:21.174904] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-02. [2025-09-16T19:12:21.816064] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T19:12:21.974933] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-06. [2025-09-16T19:12:22.777428] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-06. [2025-09-16T19:12:22.780608] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-09-16T19:12:22.940429] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-09-16T19:12:23.574932] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-07. [2025-09-16T19:12:24.300788] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-09-16T19:12:24.374943] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-04. [2025-09-16T19:12:24.545557] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T19:12:25.174904] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-07. [2025-09-16T19:12:25.974893] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-08. [2025-09-16T19:12:26.254334] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T19:12:26.594573] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-09-16T19:12:26.774917] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-05. [2025-09-16T19:12:26.837854] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T19:12:27.574914] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-03. [2025-09-16T19:12:28.374889] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-08. [2025-09-16T19:12:28.510515] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T19:12:29.174955] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-06. [2025-09-16T19:12:29.530106] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-09-16T19:12:29.574860] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-09-16T19:12:29.974941] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-04. [2025-09-16T19:12:30.780476] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-09. [2025-09-16T19:12:30.800950] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T19:12:31.440931] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T19:12:31.574904] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-09. [2025-09-16T19:12:31.894845] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T19:12:31.900170] [INFO ] [REMOTE ] LEA: Full fetch complete (45333 records). [2025-09-16T19:12:31.900274] [DEBUG] [VALIDATOR ] LEA: Validating primary data. [2025-09-16T19:12:32.322259] [DEBUG] [VALIDATOR ] LEA: All expected trading days are present. [2025-09-16T19:12:32.322384] [DEBUG] [VALIDATOR ] LEA: Primary data validation passed. [2025-09-16T19:12:32.323110] [INFO ] [MAIN ] LEA: Fetched primary data is valid. [2025-09-16T19:12:32.323157] [DEBUG] [LOCAL ] LEA: Saving primary data. [2025-09-16T19:12:32.324278] [DEBUG] [LOCAL ] LEA: Trimmed 45333 records to 30603 (keeping 150 days of data) [2025-09-16T19:12:32.386087] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-07. [2025-09-16T19:12:32.564094] [DEBUG] [LOCAL ] LEA: Primary data saved (30603 records). [2025-09-16T19:12:32.564259] [INFO ] [MAIN ] LEA: Full history saved. [2025-09-16T19:12:32.564358] [INFO ] [MAIN ] Update Progress: 1249/2369 symbols completed. LEA took 21.7s. Estimated remaining: 1h 58m 48s. Rate: 9.4 symbols/min [2025-09-16T19:12:32.709470] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-09-16T19:12:32.753084] [INFO ] [REMOTE ] LDP: Full fetch complete (14178 records). [2025-09-16T19:12:32.760480] [DEBUG] [VALIDATOR ] LDP: Validating primary data. [2025-09-16T19:12:32.786005] [INFO ] [MAIN ] LEO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:12:32.992777] [DEBUG] [LOCAL ] LEO: Loaded 11897 existing records from primary storage. [2025-09-16T19:12:33.060428] [INFO ] [MAIN ] LEO: Loaded 11897 existing records. [2025-09-16T19:12:33.183626] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-05. [2025-09-16T19:12:33.311630] [DEBUG] [VALIDATOR ] LDP: All expected trading days are present. [2025-09-16T19:12:33.321971] [DEBUG] [VALIDATOR ] LDP: Primary data validation passed. [2025-09-16T19:12:33.337706] [INFO ] [MAIN ] LDP: Fetched primary data is valid. [2025-09-16T19:12:33.372278] [DEBUG] [LOCAL ] LDP: Saving primary data. [2025-09-16T19:12:33.394231] [WARN ] [MAIN ] LEO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:12:33.394769] [DEBUG] [LOCAL ] LDP: Trimmed 14178 records to 9457 (keeping 150 days of data) [2025-09-16T19:12:33.395249] [INFO ] [MAIN ] LEO: Wiping existing primary data before full refresh. [2025-09-16T19:12:33.395302] [INFO ] [LOCAL ] LEO: Starting deletion of primary data. [2025-09-16T19:12:33.396071] [DEBUG] [LOCAL ] LEO: Deleted primary data file. [2025-09-16T19:12:33.396122] [INFO ] [MAIN ] LEO: Starting full primary refresh. [2025-09-16T19:12:33.838858] [DEBUG] [LOCAL ] LDP: Primary data saved (9457 records). [2025-09-16T19:12:33.856851] [INFO ] [MAIN ] LDP: Full history saved. [2025-09-16T19:12:33.876490] [INFO ] [REMOTE ] LEO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:12:33.876696] [INFO ] [MAIN ] Update Progress: 1250/2369 symbols completed. LDP took 24.9s. Estimated remaining: 1h 58m 38s. Rate: 9.4 symbols/min [2025-09-16T19:12:33.985882] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-02. [2025-09-16T19:12:34.059942] [INFO ] [MAIN ] LEVI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:12:34.096287] [DEBUG] [LOCAL ] LEVI: Loaded 37969 existing records from primary storage. [2025-09-16T19:12:34.096457] [INFO ] [MAIN ] LEVI: Loaded 37969 existing records. [2025-09-16T19:12:34.275234] [WARN ] [MAIN ] LEVI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:12:34.275530] [INFO ] [MAIN ] LEVI: Wiping existing primary data before full refresh. [2025-09-16T19:12:34.275613] [INFO ] [LOCAL ] LEVI: Starting deletion of primary data. [2025-09-16T19:12:34.276491] [DEBUG] [LOCAL ] LEVI: Deleted primary data file. [2025-09-16T19:12:34.276544] [INFO ] [MAIN ] LEVI: Starting full primary refresh. [2025-09-16T19:12:34.559787] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-09-16T19:12:34.685837] [INFO ] [REMOTE ] LEVI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:12:34.765874] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-09-16T19:12:34.776607] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-02. [2025-09-16T19:12:35.061984] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-09-16T19:12:35.574892] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-08. [2025-09-16T19:12:36.349227] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T19:12:36.374861] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-03. [2025-09-16T19:12:37.174890] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-06. [2025-09-16T19:12:37.347997] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-09-16T19:12:37.974907] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-03. [2025-09-16T19:12:38.125710] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T19:12:38.774883] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-09. [2025-09-16T19:12:39.019499] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-09-16T19:12:39.081913] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-09-16T19:12:39.574924] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-04. [2025-09-16T19:12:40.349106] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-09-16T19:12:40.355084] [INFO ] [REMOTE ] LEG: Full fetch complete (54874 records). [2025-09-16T19:12:40.355185] [DEBUG] [VALIDATOR ] LEG: Validating primary data. [2025-09-16T19:12:40.380025] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-07. [2025-09-16T19:12:40.772051] [DEBUG] [VALIDATOR ] LEG: All expected trading days are present. [2025-09-16T19:12:40.772291] [DEBUG] [VALIDATOR ] LEG: Primary data validation passed. [2025-09-16T19:12:40.773127] [INFO ] [MAIN ] LEG: Fetched primary data is valid. [2025-09-16T19:12:40.773182] [DEBUG] [LOCAL ] LEG: Saving primary data. [2025-09-16T19:12:40.788696] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-09-16T19:12:40.788137] [DEBUG] [LOCAL ] LEG: Trimmed 54874 records to 37283 (keeping 150 days of data) [2025-09-16T19:12:41.025459] [DEBUG] [LOCAL ] LEG: Primary data saved (37283 records). [2025-09-16T19:12:41.025619] [INFO ] [MAIN ] LEG: Full history saved. [2025-09-16T19:12:41.025719] [INFO ] [MAIN ] Update Progress: 1251/2369 symbols completed. LEG took 27.8s. Estimated remaining: 1h 58m 32s. Rate: 9.4 symbols/min [2025-09-16T19:12:41.174943] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-05. [2025-09-16T19:12:41.198327] [INFO ] [MAIN ] LFT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:12:41.212689] [DEBUG] [LOCAL ] LFT: Loaded 13970 existing records from primary storage. [2025-09-16T19:12:41.212813] [INFO ] [MAIN ] LFT: Loaded 13970 existing records. [2025-09-16T19:12:41.386968] [WARN ] [MAIN ] LFT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:12:41.387244] [INFO ] [MAIN ] LFT: Wiping existing primary data before full refresh. [2025-09-16T19:12:41.387290] [INFO ] [LOCAL ] LFT: Starting deletion of primary data. [2025-09-16T19:12:41.387829] [DEBUG] [LOCAL ] LFT: Deleted primary data file. [2025-09-16T19:12:41.387879] [INFO ] [MAIN ] LFT: Starting full primary refresh. [2025-09-16T19:12:41.739330] [INFO ] [REMOTE ] LFT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:12:41.974857] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-02. [2025-09-16T19:12:42.099942] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-09-16T19:12:42.371471] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-09-16T19:12:42.483631] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-09-16T19:12:42.774898] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-06. [2025-09-16T19:12:43.574912] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-08. [2025-09-16T19:12:44.236973] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T19:12:44.374905] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-04. [2025-09-16T19:12:45.174897] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-07. [2025-09-16T19:12:45.610707] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-08 (2.04 seconds). [2025-09-16T19:12:45.974888] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-03. [2025-09-16T19:12:46.177419] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-09-16T19:12:46.198423] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T19:12:46.774917] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-08. [2025-09-16T19:12:46.876433] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-09-16T19:12:47.574934] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-09. [2025-09-16T19:12:47.750820] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-09-16T19:12:48.374948] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-05. [2025-09-16T19:12:49.174898] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-09. [2025-09-16T19:12:49.772890] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-09-16T19:12:49.779283] [INFO ] [REMOTE ] LEN: Full fetch complete (57811 records). [2025-09-16T19:12:49.779383] [DEBUG] [VALIDATOR ] LEN: Validating primary data. [2025-09-16T19:12:49.980156] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-04. [2025-09-16T19:12:50.091549] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T19:12:50.115477] [INFO ] [REMOTE ] LEO: Full fetch complete (19620 records). [2025-09-16T19:12:50.144119] [DEBUG] [VALIDATOR ] LEO: Validating primary data. [2025-09-16T19:12:50.222847] [DEBUG] [VALIDATOR ] LEN: All expected trading days are present. [2025-09-16T19:12:50.223113] [DEBUG] [VALIDATOR ] LEN: Primary data validation passed. [2025-09-16T19:12:50.231161] [INFO ] [MAIN ] LEN: Fetched primary data is valid. [2025-09-16T19:12:50.231319] [DEBUG] [LOCAL ] LEN: Saving primary data. [2025-09-16T19:12:50.299027] [DEBUG] [LOCAL ] LEN: Trimmed 57811 records to 39613 (keeping 150 days of data) [2025-09-16T19:12:50.662784] [DEBUG] [VALIDATOR ] LEO: All expected trading days are present. [2025-09-16T19:12:50.663147] [DEBUG] [VALIDATOR ] LEO: Primary data validation passed. [2025-09-16T19:12:50.663863] [INFO ] [MAIN ] LEO: Fetched primary data is valid. [2025-09-16T19:12:50.664252] [DEBUG] [LOCAL ] LEO: Saving primary data. [2025-09-16T19:12:50.667438] [DEBUG] [LOCAL ] LEO: Trimmed 19620 records to 12057 (keeping 150 days of data) [2025-09-16T19:12:50.757343] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-05 (2.38 seconds). [2025-09-16T19:12:50.775596] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-06. [2025-09-16T19:12:50.860118] [DEBUG] [LOCAL ] LEO: Primary data saved (12057 records). [2025-09-16T19:12:50.868730] [INFO ] [MAIN ] LEO: Full history saved. [2025-09-16T19:12:50.868855] [INFO ] [MAIN ] Update Progress: 1252/2369 symbols completed. LEO took 18.3s. Estimated remaining: 1h 58m 29s. Rate: 9.4 symbols/min [2025-09-16T19:12:51.075607] [INFO ] [MAIN ] LGI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:12:51.090657] [DEBUG] [LOCAL ] LGI: Loaded 7281 existing records from primary storage. [2025-09-16T19:12:51.091864] [INFO ] [MAIN ] LGI: Loaded 7281 existing records. [2025-09-16T19:12:51.194079] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-09-16T19:12:51.301839] [WARN ] [MAIN ] LGI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:12:51.313306] [DEBUG] [LOCAL ] LEN: Primary data saved (39613 records). [2025-09-16T19:12:51.313482] [INFO ] [MAIN ] LEN: Full history saved. [2025-09-16T19:12:51.314302] [INFO ] [MAIN ] Update Progress: 1253/2369 symbols completed. LEN took 32.1s. Estimated remaining: 1h 58m 17s. Rate: 9.4 symbols/min [2025-09-16T19:12:51.313376] [INFO ] [MAIN ] LGI: Wiping existing primary data before full refresh. [2025-09-16T19:12:51.357505] [INFO ] [LOCAL ] LGI: Starting deletion of primary data. [2025-09-16T19:12:51.416416] [DEBUG] [LOCAL ] LGI: Deleted primary data file. [2025-09-16T19:12:51.416570] [INFO ] [MAIN ] LGI: Starting full primary refresh. [2025-09-16T19:12:51.587943] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-05. [2025-09-16T19:12:51.594186] [INFO ] [MAIN ] LH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:12:51.891032] [DEBUG] [LOCAL ] LH: Loaded 27924 existing records from primary storage. [2025-09-16T19:12:51.907037] [INFO ] [MAIN ] LH: Loaded 27924 existing records. [2025-09-16T19:12:51.909946] [INFO ] [REMOTE ] LGI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:12:52.097587] [WARN ] [MAIN ] LH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:12:52.097867] [INFO ] [MAIN ] LH: Wiping existing primary data before full refresh. [2025-09-16T19:12:52.097913] [INFO ] [LOCAL ] LH: Starting deletion of primary data. [2025-09-16T19:12:52.098684] [DEBUG] [LOCAL ] LH: Deleted primary data file. [2025-09-16T19:12:52.098741] [INFO ] [MAIN ] LH: Starting full primary refresh. [2025-09-16T19:12:52.375993] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-02. [2025-09-16T19:12:52.484985] [INFO ] [REMOTE ] LH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:12:52.517023] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-09-16T19:12:52.615050] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-09-16T19:12:53.174901] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-02. [2025-09-16T19:12:53.210597] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-09-16T19:12:53.974906] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-06. [2025-09-16T19:12:54.778393] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-03. [2025-09-16T19:12:54.783000] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T19:12:55.509952] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-09-16T19:12:55.579463] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-03. [2025-09-16T19:12:55.583275] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T19:12:56.374908] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-04. [2025-09-16T19:12:57.172016] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-09-16T19:12:57.175008] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-07. [2025-09-16T19:12:57.677610] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-09-16T19:12:57.974900] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-07. [2025-09-16T19:12:58.371062] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-09-16T19:12:58.774972] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-08. [2025-09-16T19:12:59.574927] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-05. [2025-09-16T19:12:59.758126] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-09-16T19:12:59.799939] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-09-16T19:13:00.081975] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-09-16T19:13:00.374929] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-09. [2025-09-16T19:13:01.174879] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-06. [2025-09-16T19:13:01.269831] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T19:13:01.271815] [INFO ] [REMOTE ] LFT: Full fetch complete (18600 records). [2025-09-16T19:13:01.271884] [DEBUG] [VALIDATOR ] LFT: Validating primary data. [2025-09-16T19:13:01.635084] [DEBUG] [VALIDATOR ] LFT: All expected trading days are present. [2025-09-16T19:13:01.635166] [DEBUG] [VALIDATOR ] LFT: Primary data validation passed. [2025-09-16T19:13:01.635606] [INFO ] [MAIN ] LFT: Fetched primary data is valid. [2025-09-16T19:13:01.635673] [DEBUG] [LOCAL ] LFT: Saving primary data. [2025-09-16T19:13:01.636286] [DEBUG] [LOCAL ] LFT: Trimmed 18600 records to 14075 (keeping 150 days of data) [2025-09-16T19:13:01.727010] [DEBUG] [LOCAL ] LFT: Primary data saved (14075 records). [2025-09-16T19:13:01.727180] [INFO ] [MAIN ] LFT: Full history saved. [2025-09-16T19:13:01.727286] [INFO ] [MAIN ] Update Progress: 1254/2369 symbols completed. LFT took 20.7s. Estimated remaining: 1h 58m 14s. Rate: 9.4 symbols/min [2025-09-16T19:13:01.914850] [INFO ] [MAIN ] LHX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:13:01.951968] [DEBUG] [LOCAL ] LHX: Loaded 36187 existing records from primary storage. [2025-09-16T19:13:01.952129] [INFO ] [MAIN ] LHX: Loaded 36187 existing records. [2025-09-16T19:13:01.982989] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-08. [2025-09-16T19:13:02.107942] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-09-16T19:13:02.146017] [WARN ] [MAIN ] LHX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:13:02.146282] [INFO ] [MAIN ] LHX: Wiping existing primary data before full refresh. [2025-09-16T19:13:02.146331] [INFO ] [LOCAL ] LHX: Starting deletion of primary data. [2025-09-16T19:13:02.147256] [DEBUG] [LOCAL ] LHX: Deleted primary data file. [2025-09-16T19:13:02.147306] [INFO ] [MAIN ] LHX: Starting full primary refresh. [2025-09-16T19:13:02.499704] [INFO ] [REMOTE ] LHX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:13:02.774892] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-02. [2025-09-16T19:13:03.574905] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-04. [2025-09-16T19:13:03.653903] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-09-16T19:13:04.064568] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T19:13:04.374907] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-03. [2025-09-16T19:13:05.174909] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-07. [2025-09-16T19:13:05.282730] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-09-16T19:13:05.892397] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-09-16T19:13:05.974888] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-08. [2025-09-16T19:13:06.155759] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T19:13:06.774992] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-09. [2025-09-16T19:13:06.920060] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-09-16T19:13:07.574895] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-05. [2025-09-16T19:13:08.374917] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-04. [2025-09-16T19:13:08.404493] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-09-16T19:13:08.410771] [INFO ] [REMOTE ] LEVI: Full fetch complete (55933 records). [2025-09-16T19:13:08.410882] [DEBUG] [VALIDATOR ] LEVI: Validating primary data. [2025-09-16T19:13:08.801351] [DEBUG] [VALIDATOR ] LEVI: All expected trading days are present. [2025-09-16T19:13:08.801444] [DEBUG] [VALIDATOR ] LEVI: Primary data validation passed. [2025-09-16T19:13:08.802393] [INFO ] [MAIN ] LEVI: Fetched primary data is valid. [2025-09-16T19:13:08.802525] [DEBUG] [LOCAL ] LEVI: Saving primary data. [2025-09-16T19:13:08.803989] [DEBUG] [LOCAL ] LEVI: Trimmed 55933 records to 38348 (keeping 150 days of data) [2025-09-16T19:13:08.917479] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T19:13:09.069677] [DEBUG] [LOCAL ] LEVI: Primary data saved (38348 records). [2025-09-16T19:13:09.069841] [INFO ] [MAIN ] LEVI: Full history saved. [2025-09-16T19:13:09.069956] [INFO ] [MAIN ] Update Progress: 1255/2369 symbols completed. LEVI took 35.2s. Estimated remaining: 1h 58m 9s. Rate: 9.4 symbols/min [2025-09-16T19:13:09.180095] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-06. [2025-09-16T19:13:09.244556] [INFO ] [MAIN ] LII: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:13:09.264175] [DEBUG] [LOCAL ] LII: Loaded 20183 existing records from primary storage. [2025-09-16T19:13:09.264301] [INFO ] [MAIN ] LII: Loaded 20183 existing records. [2025-09-16T19:13:09.442158] [WARN ] [MAIN ] LII: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:13:09.442425] [INFO ] [MAIN ] LII: Wiping existing primary data before full refresh. [2025-09-16T19:13:09.442473] [INFO ] [LOCAL ] LII: Starting deletion of primary data. [2025-09-16T19:13:09.443155] [DEBUG] [LOCAL ] LII: Deleted primary data file. [2025-09-16T19:13:09.443209] [INFO ] [MAIN ] LII: Starting full primary refresh. [2025-09-16T19:13:09.804094] [INFO ] [REMOTE ] LII: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:13:09.974866] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-02. [2025-09-16T19:13:10.274278] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-09-16T19:13:10.774909] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-05. [2025-09-16T19:13:11.311574] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-09-16T19:13:11.338479] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-06 (2.16 seconds). [2025-09-16T19:13:11.574948] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-07. [2025-09-16T19:13:12.374915] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-09. [2025-09-16T19:13:12.681775] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-09-16T19:13:13.130798] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-09-16T19:13:13.132257] [INFO ] [REMOTE ] LGI: Full fetch complete (11050 records). [2025-09-16T19:13:13.132325] [DEBUG] [VALIDATOR ] LGI: Validating primary data. [2025-09-16T19:13:13.175689] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-03. [2025-09-16T19:13:13.520270] [DEBUG] [VALIDATOR ] LGI: All expected trading days are present. [2025-09-16T19:13:13.520370] [DEBUG] [VALIDATOR ] LGI: Primary data validation passed. [2025-09-16T19:13:13.520620] [INFO ] [MAIN ] LGI: Fetched primary data is valid. [2025-09-16T19:13:13.521147] [DEBUG] [LOCAL ] LGI: Saving primary data. [2025-09-16T19:13:13.521990] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-09-16T19:13:13.531160] [DEBUG] [LOCAL ] LGI: Trimmed 11050 records to 7314 (keeping 150 days of data) [2025-09-16T19:13:13.577318] [DEBUG] [LOCAL ] LGI: Primary data saved (7314 records). [2025-09-16T19:13:13.577427] [INFO ] [MAIN ] LGI: Full history saved. [2025-09-16T19:13:13.577522] [INFO ] [MAIN ] Update Progress: 1256/2369 symbols completed. LGI took 22.7s. Estimated remaining: 1h 58m 1s. Rate: 9.4 symbols/min [2025-09-16T19:13:13.746581] [INFO ] [MAIN ] LION: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:13:13.778443] [DEBUG] [LOCAL ] LION: Loaded 35366 existing records from primary storage. [2025-09-16T19:13:13.778576] [INFO ] [MAIN ] LION: Loaded 35366 existing records. [2025-09-16T19:13:13.956716] [WARN ] [MAIN ] LION: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:13:13.956938] [INFO ] [MAIN ] LION: Wiping existing primary data before full refresh. [2025-09-16T19:13:13.956984] [INFO ] [LOCAL ] LION: Starting deletion of primary data. [2025-09-16T19:13:13.957689] [DEBUG] [LOCAL ] LION: Deleted primary data file. [2025-09-16T19:13:13.957741] [INFO ] [MAIN ] LION: Starting full primary refresh. [2025-09-16T19:13:13.980046] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-06. [2025-09-16T19:13:14.333347] [INFO ] [REMOTE ] LION: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:13:14.687819] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-09-16T19:13:14.774863] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-04. [2025-09-16T19:13:15.574889] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-02. [2025-09-16T19:13:15.968473] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-09-16T19:13:16.245784] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-09-16T19:13:16.279363] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T19:13:16.374929] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-05. [2025-09-16T19:13:17.174983] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-08. [2025-09-16T19:13:17.740234] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-09-16T19:13:17.974991] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-07. [2025-09-16T19:13:18.774926] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-03. [2025-09-16T19:13:18.868879] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T19:13:19.575015] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-09. [2025-09-16T19:13:19.840778] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-09-16T19:13:19.963013] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-09-16T19:13:20.374916] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-04. [2025-09-16T19:13:20.505312] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T19:13:20.509323] [INFO ] [REMOTE ] LH: Full fetch complete (42197 records). [2025-09-16T19:13:20.509426] [DEBUG] [VALIDATOR ] LH: Validating primary data. [2025-09-16T19:13:20.902000] [DEBUG] [VALIDATOR ] LH: All expected trading days are present. [2025-09-16T19:13:20.902084] [DEBUG] [VALIDATOR ] LH: Primary data validation passed. [2025-09-16T19:13:20.902854] [INFO ] [MAIN ] LH: Fetched primary data is valid. [2025-09-16T19:13:20.902901] [DEBUG] [LOCAL ] LH: Saving primary data. [2025-09-16T19:13:20.903984] [DEBUG] [LOCAL ] LH: Trimmed 42197 records to 28126 (keeping 150 days of data) [2025-09-16T19:13:21.089669] [DEBUG] [LOCAL ] LH: Primary data saved (28126 records). [2025-09-16T19:13:21.089824] [INFO ] [MAIN ] LH: Full history saved. [2025-09-16T19:13:21.089916] [INFO ] [MAIN ] Update Progress: 1257/2369 symbols completed. LH took 29.8s. Estimated remaining: 1h 57m 55s. Rate: 9.4 symbols/min [2025-09-16T19:13:21.180053] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-08. [2025-09-16T19:13:21.269181] [INFO ] [MAIN ] LITB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:13:21.273124] [DEBUG] [LOCAL ] LITB: Loaded 2317 existing records from primary storage. [2025-09-16T19:13:21.273202] [INFO ] [MAIN ] LITB: Loaded 2317 existing records. [2025-09-16T19:13:21.451709] [WARN ] [MAIN ] LITB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:13:21.452098] [INFO ] [MAIN ] LITB: Wiping existing primary data before full refresh. [2025-09-16T19:13:21.452212] [INFO ] [LOCAL ] LITB: Starting deletion of primary data. [2025-09-16T19:13:21.459184] [DEBUG] [LOCAL ] LITB: Deleted primary data file. [2025-09-16T19:13:21.469866] [INFO ] [MAIN ] LITB: Starting full primary refresh. [2025-09-16T19:13:21.531262] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-09-16T19:13:21.846021] [INFO ] [REMOTE ] LITB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:13:21.974934] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-06. [2025-09-16T19:13:22.720583] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T19:13:22.774935] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-05. [2025-09-16T19:13:22.837247] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-09-16T19:13:23.574958] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-07. [2025-09-16T19:13:24.374921] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-09. [2025-09-16T19:13:24.853215] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-09-16T19:13:24.944117] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-09-16T19:13:25.174901] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-06. [2025-09-16T19:13:25.761694] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-09-16T19:13:25.767874] [INFO ] [REMOTE ] LHX: Full fetch complete (53583 records). [2025-09-16T19:13:25.767997] [DEBUG] [VALIDATOR ] LHX: Validating primary data. [2025-09-16T19:13:25.985477] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-02. [2025-09-16T19:13:26.143291] [DEBUG] [VALIDATOR ] LHX: All expected trading days are present. [2025-09-16T19:13:26.143371] [DEBUG] [VALIDATOR ] LHX: Primary data validation passed. [2025-09-16T19:13:26.144162] [INFO ] [MAIN ] LHX: Fetched primary data is valid. [2025-09-16T19:13:26.144209] [DEBUG] [LOCAL ] LHX: Saving primary data. [2025-09-16T19:13:26.145327] [DEBUG] [LOCAL ] LHX: Trimmed 53583 records to 36532 (keeping 150 days of data) [2025-09-16T19:13:26.387631] [DEBUG] [LOCAL ] LHX: Primary data saved (36532 records). [2025-09-16T19:13:26.387817] [INFO ] [MAIN ] LHX: Full history saved. [2025-09-16T19:13:26.387945] [INFO ] [MAIN ] Update Progress: 1258/2369 symbols completed. LHX took 24.7s. Estimated remaining: 1h 57m 48s. Rate: 9.4 symbols/min [2025-09-16T19:13:26.558261] [INFO ] [MAIN ] LLY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:13:26.568857] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-09-16T19:13:26.610783] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T19:13:26.634905] [DEBUG] [LOCAL ] LLY: Loaded 39337 existing records from primary storage. [2025-09-16T19:13:26.635056] [INFO ] [MAIN ] LLY: Loaded 39337 existing records. [2025-09-16T19:13:26.784682] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-07. [2025-09-16T19:13:26.812479] [WARN ] [MAIN ] LLY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:13:26.812664] [INFO ] [MAIN ] LLY: Wiping existing primary data before full refresh. [2025-09-16T19:13:26.812711] [INFO ] [LOCAL ] LLY: Starting deletion of primary data. [2025-09-16T19:13:26.813564] [DEBUG] [LOCAL ] LLY: Deleted primary data file. [2025-09-16T19:13:26.813612] [INFO ] [MAIN ] LLY: Starting full primary refresh. [2025-09-16T19:13:27.172662] [INFO ] [REMOTE ] LLY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:13:27.574927] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-02. [2025-09-16T19:13:28.374897] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-03. [2025-09-16T19:13:28.566556] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-09-16T19:13:29.072815] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-09-16T19:13:29.174899] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-08. [2025-09-16T19:13:29.883832] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-02 (2.31 seconds). [2025-09-16T19:13:29.974897] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-03. [2025-09-16T19:13:30.193525] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-09-16T19:13:30.774886] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-04. [2025-09-16T19:13:31.545397] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-09-16T19:13:31.574843] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-05. [2025-09-16T19:13:32.128955] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-09-16T19:13:32.377717] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-09. [2025-09-16T19:13:32.383083] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-03 (2.41 seconds). [2025-09-16T19:13:33.174906] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-04. [2025-09-16T19:13:33.974920] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-08. [2025-09-16T19:13:34.002490] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T19:13:34.006370] [INFO ] [REMOTE ] LII: Full fetch complete (31483 records). [2025-09-16T19:13:34.006497] [DEBUG] [VALIDATOR ] LII: Validating primary data. [2025-09-16T19:13:34.382600] [DEBUG] [VALIDATOR ] LII: All expected trading days are present. [2025-09-16T19:13:34.382717] [DEBUG] [VALIDATOR ] LII: Primary data validation passed. [2025-09-16T19:13:34.383311] [INFO ] [MAIN ] LII: Fetched primary data is valid. [2025-09-16T19:13:34.383353] [DEBUG] [LOCAL ] LII: Saving primary data. [2025-09-16T19:13:34.384176] [DEBUG] [LOCAL ] LII: Trimmed 31483 records to 20348 (keeping 150 days of data) [2025-09-16T19:13:34.513925] [DEBUG] [LOCAL ] LII: Primary data saved (20348 records). [2025-09-16T19:13:34.514073] [INFO ] [MAIN ] LII: Full history saved. [2025-09-16T19:13:34.514189] [INFO ] [MAIN ] Update Progress: 1259/2369 symbols completed. LII took 25.4s. Estimated remaining: 1h 57m 43s. Rate: 9.4 symbols/min [2025-09-16T19:13:34.681518] [INFO ] [MAIN ] LMND: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:13:34.719748] [DEBUG] [LOCAL ] LMND: Loaded 38437 existing records from primary storage. [2025-09-16T19:13:34.719889] [INFO ] [MAIN ] LMND: Loaded 38437 existing records. [2025-09-16T19:13:34.782426] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-06. [2025-09-16T19:13:34.902424] [WARN ] [MAIN ] LMND: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:13:34.902700] [INFO ] [MAIN ] LMND: Wiping existing primary data before full refresh. [2025-09-16T19:13:34.902754] [INFO ] [LOCAL ] LMND: Starting deletion of primary data. [2025-09-16T19:13:34.903433] [DEBUG] [LOCAL ] LMND: Deleted primary data file. [2025-09-16T19:13:34.903477] [INFO ] [MAIN ] LMND: Starting full primary refresh. [2025-09-16T19:13:35.254866] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-09-16T19:13:35.257855] [INFO ] [REMOTE ] LMND: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:13:35.574857] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-02. [2025-09-16T19:13:35.684278] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-09-16T19:13:35.779887] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-04 (2.60 seconds). [2025-09-16T19:13:36.374924] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-05. [2025-09-16T19:13:37.174919] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-09. [2025-09-16T19:13:37.495585] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-09-16T19:13:37.974918] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-03. [2025-09-16T19:13:38.640954] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T19:13:38.646912] [INFO ] [REMOTE ] LION: Full fetch complete (43966 records). [2025-09-16T19:13:38.647032] [DEBUG] [VALIDATOR ] LION: Validating primary data. [2025-09-16T19:13:38.780001] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-07. [2025-09-16T19:13:39.045426] [DEBUG] [VALIDATOR ] LION: All expected trading days are present. [2025-09-16T19:13:39.045695] [DEBUG] [VALIDATOR ] LION: Primary data validation passed. [2025-09-16T19:13:39.046691] [INFO ] [MAIN ] LION: Fetched primary data is valid. [2025-09-16T19:13:39.046827] [DEBUG] [LOCAL ] LION: Saving primary data. [2025-09-16T19:13:39.049302] [DEBUG] [LOCAL ] LION: Trimmed 43966 records to 35711 (keeping 150 days of data) [2025-09-16T19:13:39.165941] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-05 (2.79 seconds). [2025-09-16T19:13:39.298264] [DEBUG] [LOCAL ] LION: Primary data saved (35711 records). [2025-09-16T19:13:39.298411] [INFO ] [MAIN ] LION: Full history saved. [2025-09-16T19:13:39.298542] [INFO ] [MAIN ] Update Progress: 1260/2369 symbols completed. LION took 25.7s. Estimated remaining: 1h 57m 36s. Rate: 9.4 symbols/min [2025-09-16T19:13:39.471133] [INFO ] [MAIN ] LMT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:13:39.475798] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-09-16T19:13:39.548530] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-09-16T19:13:39.546948] [DEBUG] [LOCAL ] LMT: Loaded 36693 existing records from primary storage. [2025-09-16T19:13:39.559115] [INFO ] [MAIN ] LMT: Loaded 36693 existing records. [2025-09-16T19:13:39.581000] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-08. [2025-09-16T19:13:39.788336] [WARN ] [MAIN ] LMT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:13:39.788695] [INFO ] [MAIN ] LMT: Wiping existing primary data before full refresh. [2025-09-16T19:13:39.788746] [INFO ] [LOCAL ] LMT: Starting deletion of primary data. [2025-09-16T19:13:39.789612] [DEBUG] [LOCAL ] LMT: Deleted primary data file. [2025-09-16T19:13:39.789690] [INFO ] [MAIN ] LMT: Starting full primary refresh. [2025-09-16T19:13:40.127858] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-09-16T19:13:40.146731] [INFO ] [REMOTE ] LMT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:13:40.374898] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-06. [2025-09-16T19:13:41.174911] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-09. [2025-09-16T19:13:41.902527] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T19:13:41.903961] [INFO ] [REMOTE ] LITB: Full fetch complete (4460 records). [2025-09-16T19:13:41.904050] [DEBUG] [VALIDATOR ] LITB: Validating primary data. [2025-09-16T19:13:41.980079] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-04. [2025-09-16T19:13:42.260290] [DEBUG] [VALIDATOR ] LITB: All expected trading days are present. [2025-09-16T19:13:42.260366] [DEBUG] [VALIDATOR ] LITB: Primary data validation passed. [2025-09-16T19:13:42.260522] [INFO ] [MAIN ] LITB: Fetched primary data is valid. [2025-09-16T19:13:42.260569] [DEBUG] [LOCAL ] LITB: Saving primary data. [2025-09-16T19:13:42.260959] [DEBUG] [LOCAL ] LITB: Trimmed 4460 records to 2344 (keeping 150 days of data) [2025-09-16T19:13:42.275337] [DEBUG] [LOCAL ] LITB: Primary data saved (2344 records). [2025-09-16T19:13:42.275457] [INFO ] [MAIN ] LITB: Full history saved. [2025-09-16T19:13:42.275570] [INFO ] [MAIN ] Update Progress: 1261/2369 symbols completed. LITB took 21.2s. Estimated remaining: 1h 57m 26s. Rate: 9.4 symbols/min [2025-09-16T19:13:42.444422] [INFO ] [MAIN ] LNC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:13:42.489593] [DEBUG] [LOCAL ] LNC: Loaded 37942 existing records from primary storage. [2025-09-16T19:13:42.489750] [INFO ] [MAIN ] LNC: Loaded 37942 existing records. [2025-09-16T19:13:42.674213] [WARN ] [MAIN ] LNC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:13:42.674657] [INFO ] [MAIN ] LNC: Wiping existing primary data before full refresh. [2025-09-16T19:13:42.674779] [INFO ] [LOCAL ] LNC: Starting deletion of primary data. [2025-09-16T19:13:42.675674] [DEBUG] [LOCAL ] LNC: Deleted primary data file. [2025-09-16T19:13:42.675810] [INFO ] [MAIN ] LNC: Starting full primary refresh. [2025-09-16T19:13:42.776443] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-02. [2025-09-16T19:13:43.010579] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-06 (2.64 seconds). [2025-09-16T19:13:43.080613] [INFO ] [REMOTE ] LNC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:13:43.574922] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-02. [2025-09-16T19:13:43.942932] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-09-16T19:13:44.374918] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-07. [2025-09-16T19:13:44.696610] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-09-16T19:13:44.870792] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-09-16T19:13:45.174920] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-05. [2025-09-16T19:13:45.974910] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-03. [2025-09-16T19:13:46.774918] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-03. [2025-09-16T19:13:46.861569] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-07 (2.49 seconds). [2025-09-16T19:13:47.574963] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-08. [2025-09-16T19:13:47.775282] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-05 (2.60 seconds). [2025-09-16T19:13:47.960585] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-09-16T19:13:48.374919] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-04. [2025-09-16T19:13:48.504330] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T19:13:49.174905] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-04. [2025-09-16T19:13:49.974867] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-06. [2025-09-16T19:13:50.077635] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-09-16T19:13:50.188767] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-08 (2.61 seconds). [2025-09-16T19:13:50.601324] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-09-16T19:13:50.774922] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-09. [2025-09-16T19:13:51.574901] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-05. [2025-09-16T19:13:51.800369] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-09-16T19:13:52.374938] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-05. [2025-09-16T19:13:52.975634] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-09-16T19:13:52.984205] [INFO ] [REMOTE ] LLY: Full fetch complete (57940 records). [2025-09-16T19:13:52.984343] [DEBUG] [VALIDATOR ] LLY: Validating primary data. [2025-09-16T19:13:53.180167] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-07. [2025-09-16T19:13:53.376370] [DEBUG] [VALIDATOR ] LLY: All expected trading days are present. [2025-09-16T19:13:53.376554] [DEBUG] [VALIDATOR ] LLY: Primary data validation passed. [2025-09-16T19:13:53.377417] [INFO ] [MAIN ] LLY: Fetched primary data is valid. [2025-09-16T19:13:53.377479] [DEBUG] [LOCAL ] LLY: Saving primary data. [2025-09-16T19:13:53.379011] [DEBUG] [LOCAL ] LLY: Trimmed 57940 records to 39727 (keeping 150 days of data) [2025-09-16T19:13:53.514442] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-09-16T19:13:53.590167] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T19:13:53.767255] [DEBUG] [LOCAL ] LLY: Primary data saved (39727 records). [2025-09-16T19:13:53.767428] [INFO ] [MAIN ] LLY: Full history saved. [2025-09-16T19:13:53.767520] [INFO ] [MAIN ] Update Progress: 1262/2369 symbols completed. LLY took 27.4s. Estimated remaining: 1h 57m 24s. Rate: 9.4 symbols/min [2025-09-16T19:13:53.969365] [INFO ] [MAIN ] LND: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:13:53.975701] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-06. [2025-09-16T19:13:53.975607] [DEBUG] [LOCAL ] LND: Loaded 3604 existing records from primary storage. [2025-09-16T19:13:53.976191] [INFO ] [MAIN ] LND: Loaded 3604 existing records. [2025-09-16T19:13:54.159014] [WARN ] [MAIN ] LND: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:13:54.159334] [INFO ] [MAIN ] LND: Wiping existing primary data before full refresh. [2025-09-16T19:13:54.159389] [INFO ] [LOCAL ] LND: Starting deletion of primary data. [2025-09-16T19:13:54.159633] [DEBUG] [LOCAL ] LND: Deleted primary data file. [2025-09-16T19:13:54.159704] [INFO ] [MAIN ] LND: Starting full primary refresh. [2025-09-16T19:13:54.508265] [INFO ] [REMOTE ] LND: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:13:54.774891] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-06. [2025-09-16T19:13:55.276675] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-09-16T19:13:55.574880] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-08. [2025-09-16T19:13:56.106715] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-09-16T19:13:56.198188] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T19:13:56.374889] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-07. [2025-09-16T19:13:57.174872] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-07. [2025-09-16T19:13:57.693497] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-09-16T19:13:57.974960] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-09. [2025-09-16T19:13:58.268735] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T19:13:58.775008] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-09-16T19:13:58.775130] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-02. [2025-09-16T19:13:59.239213] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-09-16T19:13:59.442124] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T19:13:59.449845] [INFO ] [REMOTE ] LMND: Full fetch complete (56478 records). [2025-09-16T19:13:59.450020] [DEBUG] [VALIDATOR ] LMND: Validating primary data. [2025-09-16T19:13:59.580042] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-08. [2025-09-16T19:13:59.840181] [DEBUG] [VALIDATOR ] LMND: All expected trading days are present. [2025-09-16T19:13:59.840257] [DEBUG] [VALIDATOR ] LMND: Primary data validation passed. [2025-09-16T19:13:59.841057] [INFO ] [MAIN ] LMND: Fetched primary data is valid. [2025-09-16T19:13:59.841101] [DEBUG] [LOCAL ] LMND: Saving primary data. [2025-09-16T19:13:59.843363] [DEBUG] [LOCAL ] LMND: Trimmed 56478 records to 38824 (keeping 150 days of data) [2025-09-16T19:14:00.078426] [DEBUG] [LOCAL ] LMND: Primary data saved (38824 records). [2025-09-16T19:14:00.078573] [INFO ] [MAIN ] LMND: Full history saved. [2025-09-16T19:14:00.080386] [INFO ] [MAIN ] Update Progress: 1263/2369 symbols completed. LMND took 25.6s. Estimated remaining: 1h 57m 18s. Rate: 9.4 symbols/min [2025-09-16T19:14:00.252156] [INFO ] [MAIN ] LNG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:14:00.294494] [DEBUG] [LOCAL ] LNG: Loaded 37954 existing records from primary storage. [2025-09-16T19:14:00.294682] [INFO ] [MAIN ] LNG: Loaded 37954 existing records. [2025-09-16T19:14:00.380127] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-08. [2025-09-16T19:14:00.474399] [WARN ] [MAIN ] LNG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:14:00.474595] [INFO ] [MAIN ] LNG: Wiping existing primary data before full refresh. [2025-09-16T19:14:00.474667] [INFO ] [LOCAL ] LNG: Starting deletion of primary data. [2025-09-16T19:14:00.475564] [DEBUG] [LOCAL ] LNG: Deleted primary data file. [2025-09-16T19:14:00.475613] [INFO ] [MAIN ] LNG: Starting full primary refresh. [2025-09-16T19:14:00.834625] [INFO ] [REMOTE ] LNG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:14:01.174988] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-02. [2025-09-16T19:14:01.646168] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-09-16T19:14:01.974961] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-09. [2025-09-16T19:14:02.246828] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-09-16T19:14:02.774931] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-03. [2025-09-16T19:14:03.066942] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-09-16T19:14:03.325939] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-09-16T19:14:03.572852] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T19:14:03.578166] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-04. [2025-09-16T19:14:03.580950] [INFO ] [REMOTE ] LMT: Full fetch complete (54556 records). [2025-09-16T19:14:03.581772] [DEBUG] [VALIDATOR ] LMT: Validating primary data. [2025-09-16T19:14:03.959846] [DEBUG] [VALIDATOR ] LMT: All expected trading days are present. [2025-09-16T19:14:03.959921] [DEBUG] [VALIDATOR ] LMT: Primary data validation passed. [2025-09-16T19:14:03.960706] [INFO ] [MAIN ] LMT: Fetched primary data is valid. [2025-09-16T19:14:03.960751] [DEBUG] [LOCAL ] LMT: Saving primary data. [2025-09-16T19:14:03.961891] [DEBUG] [LOCAL ] LMT: Trimmed 54556 records to 37058 (keeping 150 days of data) [2025-09-16T19:14:04.207218] [DEBUG] [LOCAL ] LMT: Primary data saved (37058 records). [2025-09-16T19:14:04.207364] [INFO ] [MAIN ] LMT: Full history saved. [2025-09-16T19:14:04.207463] [INFO ] [MAIN ] Update Progress: 1264/2369 symbols completed. LMT took 24.9s. Estimated remaining: 1h 57m 10s. Rate: 9.4 symbols/min [2025-09-16T19:14:04.375198] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-09. [2025-09-16T19:14:04.377780] [INFO ] [MAIN ] LNN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:14:04.384789] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-09-16T19:14:04.395167] [DEBUG] [LOCAL ] LNN: Loaded 8087 existing records from primary storage. [2025-09-16T19:14:04.395785] [INFO ] [MAIN ] LNN: Loaded 8087 existing records. [2025-09-16T19:14:04.571256] [WARN ] [MAIN ] LNN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:14:04.571530] [INFO ] [MAIN ] LNN: Wiping existing primary data before full refresh. [2025-09-16T19:14:04.571577] [INFO ] [LOCAL ] LNN: Starting deletion of primary data. [2025-09-16T19:14:04.572000] [DEBUG] [LOCAL ] LNN: Deleted primary data file. [2025-09-16T19:14:04.572078] [INFO ] [MAIN ] LNN: Starting full primary refresh. [2025-09-16T19:14:04.923445] [INFO ] [REMOTE ] LNN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:14:05.174935] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-02. [2025-09-16T19:14:05.945706] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-09-16T19:14:05.974847] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-03. [2025-09-16T19:14:06.407851] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-09-16T19:14:06.414416] [INFO ] [REMOTE ] LNC: Full fetch complete (56135 records). [2025-09-16T19:14:06.414526] [DEBUG] [VALIDATOR ] LNC: Validating primary data. [2025-09-16T19:14:06.777265] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-03. [2025-09-16T19:14:06.795871] [DEBUG] [VALIDATOR ] LNC: All expected trading days are present. [2025-09-16T19:14:06.796088] [DEBUG] [VALIDATOR ] LNC: Primary data validation passed. [2025-09-16T19:14:06.796948] [INFO ] [MAIN ] LNC: Fetched primary data is valid. [2025-09-16T19:14:06.797033] [DEBUG] [LOCAL ] LNC: Saving primary data. [2025-09-16T19:14:06.805430] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-09-16T19:14:06.808660] [DEBUG] [LOCAL ] LNC: Trimmed 56135 records to 38332 (keeping 150 days of data) [2025-09-16T19:14:07.042709] [DEBUG] [LOCAL ] LNC: Primary data saved (38332 records). [2025-09-16T19:14:07.042868] [INFO ] [MAIN ] LNC: Full history saved. [2025-09-16T19:14:07.043918] [INFO ] [MAIN ] Update Progress: 1265/2369 symbols completed. LNC took 24.8s. Estimated remaining: 1h 57m 0s. Rate: 9.4 symbols/min [2025-09-16T19:14:07.215243] [INFO ] [MAIN ] LOAR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:14:07.250586] [DEBUG] [LOCAL ] LOAR: Loaded 32217 existing records from primary storage. [2025-09-16T19:14:07.250784] [INFO ] [MAIN ] LOAR: Loaded 32217 existing records. [2025-09-16T19:14:07.427041] [WARN ] [MAIN ] LOAR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:14:07.427293] [INFO ] [MAIN ] LOAR: Wiping existing primary data before full refresh. [2025-09-16T19:14:07.427343] [INFO ] [LOCAL ] LOAR: Starting deletion of primary data. [2025-09-16T19:14:07.427944] [DEBUG] [LOCAL ] LOAR: Deleted primary data file. [2025-09-16T19:14:07.427996] [INFO ] [MAIN ] LOAR: Starting full primary refresh. [2025-09-16T19:14:07.580026] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-05. [2025-09-16T19:14:07.793083] [INFO ] [REMOTE ] LOAR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:14:08.259109] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-09-16T19:14:08.374917] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-04. [2025-09-16T19:14:08.571554] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-09-16T19:14:09.174883] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-04. [2025-09-16T19:14:09.382792] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-09-16T19:14:09.974932] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-02. [2025-09-16T19:14:10.774952] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-05. [2025-09-16T19:14:10.979279] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-09-16T19:14:11.033778] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T19:14:11.268450] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-09-16T19:14:11.574961] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-03. [2025-09-16T19:14:12.374975] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-05. [2025-09-16T19:14:12.482856] [DEBUG] [REMOTE ] LNG: Error fetching primary data for 2025-05: 'API call returned an empty response.' [2025-09-16T19:14:12.483267] [WARN ] [REMOTE ] LNG: No data returned for month 2025-05 [2025-09-16T19:14:13.174927] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-06. [2025-09-16T19:14:13.283391] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T19:14:13.758192] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-09-16T19:14:13.974913] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-06. [2025-09-16T19:14:14.774920] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-04. [2025-09-16T19:14:15.574914] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-07. [2025-09-16T19:14:15.721914] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-09-16T19:14:16.304555] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-09-16T19:14:16.335874] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T19:14:16.374889] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-06. [2025-09-16T19:14:17.174926] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-05. [2025-09-16T19:14:17.200058] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-09-16T19:14:17.974908] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-08. [2025-09-16T19:14:18.440787] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-09-16T19:14:18.774882] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-07. [2025-09-16T19:14:18.870457] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T19:14:19.574914] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-06. [2025-09-16T19:14:20.374900] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-07. [2025-09-16T19:14:20.744807] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-09-16T19:14:21.026001] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T19:14:21.174883] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-07. [2025-09-16T19:14:21.320704] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-09-16T19:14:21.974893] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-09. [2025-09-16T19:14:22.642661] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-09-16T19:14:22.644064] [INFO ] [REMOTE ] LND: Full fetch complete (5932 records). [2025-09-16T19:14:22.644141] [DEBUG] [VALIDATOR ] LND: Validating primary data. [2025-09-16T19:14:22.788704] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-08. [2025-09-16T19:14:23.020618] [DEBUG] [VALIDATOR ] LND: All expected trading days are present. [2025-09-16T19:14:23.020852] [DEBUG] [VALIDATOR ] LND: Primary data validation passed. [2025-09-16T19:14:23.021074] [INFO ] [MAIN ] LND: Fetched primary data is valid. [2025-09-16T19:14:23.021163] [DEBUG] [LOCAL ] LND: Saving primary data. [2025-09-16T19:14:23.021754] [DEBUG] [LOCAL ] LND: Trimmed 5932 records to 3632 (keeping 150 days of data) [2025-09-16T19:14:23.043636] [DEBUG] [LOCAL ] LND: Primary data saved (3632 records). [2025-09-16T19:14:23.043774] [INFO ] [MAIN ] LND: Full history saved. [2025-09-16T19:14:23.043867] [INFO ] [MAIN ] Update Progress: 1266/2369 symbols completed. LND took 29.3s. Estimated remaining: 1h 57m 2s. Rate: 9.4 symbols/min [2025-09-16T19:14:23.219430] [INFO ] [MAIN ] LOB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:14:23.252861] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-09-16T19:14:23.278165] [DEBUG] [LOCAL ] LOB: Loaded 19142 existing records from primary storage. [2025-09-16T19:14:23.278339] [INFO ] [MAIN ] LOB: Loaded 19142 existing records. [2025-09-16T19:14:23.467428] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-09-16T19:14:23.473045] [WARN ] [MAIN ] LOB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:14:23.476892] [INFO ] [MAIN ] LOB: Wiping existing primary data before full refresh. [2025-09-16T19:14:23.476967] [INFO ] [LOCAL ] LOB: Starting deletion of primary data. [2025-09-16T19:14:23.477719] [DEBUG] [LOCAL ] LOB: Deleted primary data file. [2025-09-16T19:14:23.477793] [INFO ] [MAIN ] LOB: Starting full primary refresh. [2025-09-16T19:14:23.580033] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-09. [2025-09-16T19:14:23.844507] [INFO ] [REMOTE ] LOB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:14:24.374901] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-08. [2025-09-16T19:14:24.619406] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-09-16T19:14:24.621136] [INFO ] [REMOTE ] LNN: Full fetch complete (11944 records). [2025-09-16T19:14:24.621205] [DEBUG] [VALIDATOR ] LNN: Validating primary data. [2025-09-16T19:14:24.986830] [DEBUG] [VALIDATOR ] LNN: All expected trading days are present. [2025-09-16T19:14:24.986923] [DEBUG] [VALIDATOR ] LNN: Primary data validation passed. [2025-09-16T19:14:24.987217] [INFO ] [MAIN ] LNN: Fetched primary data is valid. [2025-09-16T19:14:24.987258] [DEBUG] [LOCAL ] LNN: Saving primary data. [2025-09-16T19:14:24.987845] [DEBUG] [LOCAL ] LNN: Trimmed 11944 records to 8163 (keeping 150 days of data) [2025-09-16T19:14:25.040365] [DEBUG] [LOCAL ] LNN: Primary data saved (8163 records). [2025-09-16T19:14:25.040517] [INFO ] [MAIN ] LNN: Full history saved. [2025-09-16T19:14:25.040605] [INFO ] [MAIN ] Update Progress: 1267/2369 symbols completed. LNN took 20.8s. Estimated remaining: 1h 56m 52s. Rate: 9.4 symbols/min [2025-09-16T19:14:25.182881] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-08. [2025-09-16T19:14:25.213408] [INFO ] [MAIN ] LOCL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:14:25.216886] [DEBUG] [LOCAL ] LOCL: Loaded 1941 existing records from primary storage. [2025-09-16T19:14:25.216957] [INFO ] [MAIN ] LOCL: Loaded 1941 existing records. [2025-09-16T19:14:25.397381] [WARN ] [MAIN ] LOCL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:14:25.397639] [INFO ] [MAIN ] LOCL: Wiping existing primary data before full refresh. [2025-09-16T19:14:25.397705] [INFO ] [LOCAL ] LOCL: Starting deletion of primary data. [2025-09-16T19:14:25.397913] [DEBUG] [LOCAL ] LOCL: Deleted primary data file. [2025-09-16T19:14:25.397956] [INFO ] [MAIN ] LOCL: Starting full primary refresh. [2025-09-16T19:14:25.756584] [INFO ] [REMOTE ] LOCL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:14:25.966926] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T19:14:25.975846] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-02. [2025-09-16T19:14:26.304843] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-09-16T19:14:26.774963] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-03. [2025-09-16T19:14:27.152886] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-09-16T19:14:27.188136] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-09-16T19:14:27.574943] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-09. [2025-09-16T19:14:28.374913] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-09. [2025-09-16T19:14:29.073775] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T19:14:29.078996] [INFO ] [REMOTE ] LOAR: Full fetch complete (44360 records). [2025-09-16T19:14:29.079083] [DEBUG] [VALIDATOR ] LOAR: Validating primary data. [2025-09-16T19:14:29.178573] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-04. [2025-09-16T19:14:29.480178] [DEBUG] [VALIDATOR ] LOAR: All expected trading days are present. [2025-09-16T19:14:29.480266] [DEBUG] [VALIDATOR ] LOAR: Primary data validation passed. [2025-09-16T19:14:29.481040] [INFO ] [MAIN ] LOAR: Fetched primary data is valid. [2025-09-16T19:14:29.481087] [DEBUG] [LOCAL ] LOAR: Saving primary data. [2025-09-16T19:14:29.482137] [DEBUG] [LOCAL ] LOAR: Trimmed 44360 records to 32474 (keeping 150 days of data) [2025-09-16T19:14:29.674923] [DEBUG] [LOCAL ] LOAR: Primary data saved (32474 records). [2025-09-16T19:14:29.675078] [INFO ] [MAIN ] LOAR: Full history saved. [2025-09-16T19:14:29.675751] [INFO ] [MAIN ] Update Progress: 1268/2369 symbols completed. LOAR took 22.6s. Estimated remaining: 1h 56m 44s. Rate: 9.4 symbols/min [2025-09-16T19:14:29.845568] [INFO ] [MAIN ] LOMA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:14:29.869442] [DEBUG] [LOCAL ] LOMA: Loaded 22980 existing records from primary storage. [2025-09-16T19:14:29.869578] [INFO ] [MAIN ] LOMA: Loaded 22980 existing records. [2025-09-16T19:14:29.976841] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-02. [2025-09-16T19:14:30.055934] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-09-16T19:14:30.059679] [WARN ] [MAIN ] LOMA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:14:30.063360] [INFO ] [MAIN ] LOMA: Wiping existing primary data before full refresh. [2025-09-16T19:14:30.063417] [INFO ] [LOCAL ] LOMA: Starting deletion of primary data. [2025-09-16T19:14:30.063963] [DEBUG] [LOCAL ] LOMA: Deleted primary data file. [2025-09-16T19:14:30.064036] [INFO ] [MAIN ] LOMA: Starting full primary refresh. [2025-09-16T19:14:30.438345] [INFO ] [REMOTE ] LOMA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:14:30.440121] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-09-16T19:14:30.446470] [INFO ] [REMOTE ] LNG: Full fetch complete (48387 records). [2025-09-16T19:14:30.446661] [DEBUG] [VALIDATOR ] LNG: Validating primary data. [2025-09-16T19:14:30.638377] [WARN ] [VALIDATOR ] LNG: Validation failed - insufficient historical data. Have 129 unique days, need 150 days [2025-09-16T19:14:30.639011] [FATAL] [MAIN ] LNG: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:14:30.639090] [WARN ] [MAIN ] LNG: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:14:30.639145] [INFO ] [LOCAL ] LNG: Starting deletion of primary data. [2025-09-16T19:14:30.639219] [DEBUG] [LOCAL ] LNG: No primary data file found to delete. [2025-09-16T19:14:30.639265] [INFO ] [LOCAL ] LNG: Starting deletion of derivative data. [2025-09-16T19:14:30.639740] [DEBUG] [LOCAL ] LNG: Deleted derivative data file. [2025-09-16T19:14:30.640289] [INFO ] [MAIN ] Update Progress: 1269/2369 symbols completed. LNG took 30.6s. Estimated remaining: 1h 56m 33s. Rate: 9.4 symbols/min [2025-09-16T19:14:30.784630] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-05. [2025-09-16T19:14:30.814526] [INFO ] [MAIN ] LOW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:14:30.856612] [DEBUG] [LOCAL ] LOW: Loaded 39109 existing records from primary storage. [2025-09-16T19:14:30.856830] [INFO ] [MAIN ] LOW: Loaded 39109 existing records. [2025-09-16T19:14:31.034809] [WARN ] [MAIN ] LOW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:14:31.035073] [INFO ] [MAIN ] LOW: Wiping existing primary data before full refresh. [2025-09-16T19:14:31.035156] [INFO ] [LOCAL ] LOW: Starting deletion of primary data. [2025-09-16T19:14:31.036247] [DEBUG] [LOCAL ] LOW: Deleted primary data file. [2025-09-16T19:14:31.036297] [INFO ] [MAIN ] LOW: Starting full primary refresh. [2025-09-16T19:14:31.226521] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-09-16T19:14:31.357515] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-09-16T19:14:31.414099] [INFO ] [REMOTE ] LOW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:14:31.574955] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-03. [2025-09-16T19:14:32.374900] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-02. [2025-09-16T19:14:32.946192] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-09-16T19:14:33.174902] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-06. [2025-09-16T19:14:33.663932] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-09-16T19:14:33.805440] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-09-16T19:14:33.974907] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-07. [2025-09-16T19:14:34.294752] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-09-16T19:14:34.774907] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-03. [2025-09-16T19:14:35.574949] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-04. [2025-09-16T19:14:36.374878] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-08. [2025-09-16T19:14:36.719720] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-09-16T19:14:36.825892] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-09-16T19:14:36.846423] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-09-16T19:14:37.174907] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-04. [2025-09-16T19:14:37.974925] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-02. [2025-09-16T19:14:38.774954] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-05. [2025-09-16T19:14:39.076361] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-09-16T19:14:39.241161] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T19:14:39.574929] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-03. [2025-09-16T19:14:39.896539] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T19:14:40.374961] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-06. [2025-09-16T19:14:40.669216] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-09-16T19:14:41.174917] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-05. [2025-09-16T19:14:41.407040] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-09-16T19:14:41.974914] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-04. [2025-09-16T19:14:42.774867] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-07. [2025-09-16T19:14:42.830695] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-09-16T19:14:43.200956] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-09-16T19:14:43.574946] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-06. [2025-09-16T19:14:44.022680] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-09-16T19:14:44.374932] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-08. [2025-09-16T19:14:45.174889] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-09. [2025-09-16T19:14:45.217073] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-09-16T19:14:45.523633] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T19:14:45.765478] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-09-16T19:14:45.766754] [INFO ] [REMOTE ] LOCL: Full fetch complete (3360 records). [2025-09-16T19:14:45.766830] [DEBUG] [VALIDATOR ] LOCL: Validating primary data. [2025-09-16T19:14:45.979987] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-09. [2025-09-16T19:14:46.133270] [DEBUG] [VALIDATOR ] LOCL: All expected trading days are present. [2025-09-16T19:14:46.133357] [DEBUG] [VALIDATOR ] LOCL: Primary data validation passed. [2025-09-16T19:14:46.133546] [INFO ] [MAIN ] LOCL: Fetched primary data is valid. [2025-09-16T19:14:46.133589] [DEBUG] [LOCAL ] LOCL: Saving primary data. [2025-09-16T19:14:46.133963] [DEBUG] [LOCAL ] LOCL: Trimmed 3360 records to 1993 (keeping 150 days of data) [2025-09-16T19:14:46.145656] [DEBUG] [LOCAL ] LOCL: Primary data saved (1993 records). [2025-09-16T19:14:46.145721] [INFO ] [MAIN ] LOCL: Full history saved. [2025-09-16T19:14:46.145796] [INFO ] [MAIN ] Update Progress: 1270/2369 symbols completed. LOCL took 21.1s. Estimated remaining: 1h 56m 35s. Rate: 9.4 symbols/min [2025-09-16T19:14:46.313549] [INFO ] [MAIN ] LPG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:14:46.343092] [DEBUG] [LOCAL ] LPG: Loaded 29546 existing records from primary storage. [2025-09-16T19:14:46.343238] [INFO ] [MAIN ] LPG: Loaded 29546 existing records. [2025-09-16T19:14:46.533238] [WARN ] [MAIN ] LPG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:14:46.533503] [INFO ] [MAIN ] LPG: Wiping existing primary data before full refresh. [2025-09-16T19:14:46.533549] [INFO ] [LOCAL ] LPG: Starting deletion of primary data. [2025-09-16T19:14:46.534248] [DEBUG] [LOCAL ] LPG: Deleted primary data file. [2025-09-16T19:14:46.534296] [INFO ] [MAIN ] LPG: Starting full primary refresh. [2025-09-16T19:14:46.780372] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-05. [2025-09-16T19:14:46.897591] [INFO ] [REMOTE ] LPG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:14:47.117341] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T19:14:47.120807] [INFO ] [REMOTE ] LOB: Full fetch complete (29390 records). [2025-09-16T19:14:47.120906] [DEBUG] [VALIDATOR ] LOB: Validating primary data. [2025-09-16T19:14:47.490781] [DEBUG] [VALIDATOR ] LOB: All expected trading days are present. [2025-09-16T19:14:47.490866] [DEBUG] [VALIDATOR ] LOB: Primary data validation passed. [2025-09-16T19:14:47.491494] [INFO ] [MAIN ] LOB: Fetched primary data is valid. [2025-09-16T19:14:47.491535] [DEBUG] [LOCAL ] LOB: Saving primary data. [2025-09-16T19:14:47.492374] [DEBUG] [LOCAL ] LOB: Trimmed 29390 records to 19344 (keeping 150 days of data) [2025-09-16T19:14:47.578094] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-02. [2025-09-16T19:14:47.613034] [DEBUG] [LOCAL ] LOB: Primary data saved (19344 records). [2025-09-16T19:14:47.613189] [INFO ] [MAIN ] LOB: Full history saved. [2025-09-16T19:14:47.613278] [INFO ] [MAIN ] Update Progress: 1271/2369 symbols completed. LOB took 24.6s. Estimated remaining: 1h 56m 24s. Rate: 9.4 symbols/min [2025-09-16T19:14:47.783702] [INFO ] [MAIN ] LPL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:14:47.802499] [DEBUG] [LOCAL ] LPL: Loaded 19036 existing records from primary storage. [2025-09-16T19:14:47.802635] [INFO ] [MAIN ] LPL: Loaded 19036 existing records. [2025-09-16T19:14:47.976372] [WARN ] [MAIN ] LPL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:14:47.976607] [INFO ] [MAIN ] LPL: Wiping existing primary data before full refresh. [2025-09-16T19:14:47.976672] [INFO ] [LOCAL ] LPL: Starting deletion of primary data. [2025-09-16T19:14:47.977283] [DEBUG] [LOCAL ] LPL: Deleted primary data file. [2025-09-16T19:14:47.977332] [INFO ] [MAIN ] LPL: Starting full primary refresh. [2025-09-16T19:14:48.327524] [INFO ] [REMOTE ] LPL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:14:48.347481] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-09-16T19:14:48.374849] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-02. [2025-09-16T19:14:49.064770] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T19:14:49.174915] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-03. [2025-09-16T19:14:49.365906] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-09-16T19:14:49.974935] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-03. [2025-09-16T19:14:50.729846] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-09-16T19:14:50.774910] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-04. [2025-09-16T19:14:51.000276] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-09-16T19:14:51.575007] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-04. [2025-09-16T19:14:52.222232] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-09-16T19:14:52.374922] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-06. [2025-09-16T19:14:52.797414] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-09-16T19:14:53.174882] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-05. [2025-09-16T19:14:53.567177] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-09-16T19:14:53.826796] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-09-16T19:14:53.974902] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-07. [2025-09-16T19:14:54.774882] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-06. [2025-09-16T19:14:55.213634] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-09-16T19:14:55.447167] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-09-16T19:14:55.574905] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-08. [2025-09-16T19:14:56.374867] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-07. [2025-09-16T19:14:56.832523] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-09-16T19:14:57.174901] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-07. [2025-09-16T19:14:57.974878] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-09. [2025-09-16T19:14:58.243064] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T19:14:58.326422] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T19:14:58.774895] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-05. [2025-09-16T19:14:59.518672] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T19:14:59.524075] [INFO ] [REMOTE ] LOMA: Full fetch complete (34261 records). [2025-09-16T19:14:59.524167] [DEBUG] [VALIDATOR ] LOMA: Validating primary data. [2025-09-16T19:14:59.579999] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-08. [2025-09-16T19:14:59.909426] [DEBUG] [VALIDATOR ] LOMA: All expected trading days are present. [2025-09-16T19:14:59.909518] [DEBUG] [VALIDATOR ] LOMA: Primary data validation passed. [2025-09-16T19:14:59.910158] [INFO ] [MAIN ] LOMA: Fetched primary data is valid. [2025-09-16T19:14:59.910202] [DEBUG] [LOCAL ] LOMA: Saving primary data. [2025-09-16T19:14:59.910951] [DEBUG] [LOCAL ] LOMA: Trimmed 34261 records to 23279 (keeping 150 days of data) [2025-09-16T19:15:00.045921] [DEBUG] [LOCAL ] LOMA: Primary data saved (23279 records). [2025-09-16T19:15:00.046056] [INFO ] [MAIN ] LOMA: Full history saved. [2025-09-16T19:15:00.046140] [INFO ] [MAIN ] Update Progress: 1272/2369 symbols completed. LOMA took 30.4s. Estimated remaining: 1h 56m 23s. Rate: 9.4 symbols/min [2025-09-16T19:15:00.214926] [INFO ] [MAIN ] LPX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:00.249719] [DEBUG] [LOCAL ] LPX: Loaded 29378 existing records from primary storage. [2025-09-16T19:15:00.250101] [INFO ] [MAIN ] LPX: Loaded 29378 existing records. [2025-09-16T19:15:00.379312] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-08. [2025-09-16T19:15:00.436433] [WARN ] [MAIN ] LPX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:00.437240] [INFO ] [MAIN ] LPX: Wiping existing primary data before full refresh. [2025-09-16T19:15:00.437365] [INFO ] [LOCAL ] LPX: Starting deletion of primary data. [2025-09-16T19:15:00.438174] [DEBUG] [LOCAL ] LPX: Deleted primary data file. [2025-09-16T19:15:00.438228] [INFO ] [MAIN ] LPX: Starting full primary refresh. [2025-09-16T19:15:00.642942] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-09-16T19:15:00.829464] [INFO ] [REMOTE ] LPX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:01.174906] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-02. [2025-09-16T19:15:01.541576] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-09-16T19:15:01.969340] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T19:15:01.977161] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-09. [2025-09-16T19:15:02.279012] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T19:15:02.774934] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-06. [2025-09-16T19:15:03.574949] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-03. [2025-09-16T19:15:03.677383] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T19:15:03.683840] [INFO ] [REMOTE ] LOW: Full fetch complete (57572 records). [2025-09-16T19:15:03.683999] [DEBUG] [VALIDATOR ] LOW: Validating primary data. [2025-09-16T19:15:04.074427] [DEBUG] [VALIDATOR ] LOW: All expected trading days are present. [2025-09-16T19:15:04.074509] [DEBUG] [VALIDATOR ] LOW: Primary data validation passed. [2025-09-16T19:15:04.075335] [INFO ] [MAIN ] LOW: Fetched primary data is valid. [2025-09-16T19:15:04.075379] [DEBUG] [LOCAL ] LOW: Saving primary data. [2025-09-16T19:15:04.076521] [DEBUG] [LOCAL ] LOW: Trimmed 57572 records to 39480 (keeping 150 days of data) [2025-09-16T19:15:04.330464] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T19:15:04.363371] [DEBUG] [LOCAL ] LOW: Primary data saved (39480 records). [2025-09-16T19:15:04.363527] [INFO ] [MAIN ] LOW: Full history saved. [2025-09-16T19:15:04.364214] [INFO ] [MAIN ] Update Progress: 1273/2369 symbols completed. LOW took 33.7s. Estimated remaining: 1h 56m 15s. Rate: 9.4 symbols/min [2025-09-16T19:15:04.385450] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-09. [2025-09-16T19:15:04.544165] [INFO ] [MAIN ] LRN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:04.583863] [DEBUG] [LOCAL ] LRN: Loaded 29198 existing records from primary storage. [2025-09-16T19:15:04.584045] [INFO ] [MAIN ] LRN: Loaded 29198 existing records. [2025-09-16T19:15:04.862222] [WARN ] [MAIN ] LRN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:04.862504] [INFO ] [MAIN ] LRN: Wiping existing primary data before full refresh. [2025-09-16T19:15:04.862559] [INFO ] [LOCAL ] LRN: Starting deletion of primary data. [2025-09-16T19:15:04.863514] [DEBUG] [LOCAL ] LRN: Deleted primary data file. [2025-09-16T19:15:04.863587] [INFO ] [MAIN ] LRN: Starting full primary refresh. [2025-09-16T19:15:05.194824] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-07. [2025-09-16T19:15:05.291676] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-09-16T19:15:05.485095] [INFO ] [REMOTE ] LRN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:05.711153] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-09-16T19:15:05.718378] [INFO ] [REMOTE ] LPL: Full fetch complete (26271 records). [2025-09-16T19:15:05.718520] [DEBUG] [VALIDATOR ] LPL: Validating primary data. [2025-09-16T19:15:05.980045] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-02. [2025-09-16T19:15:06.110417] [DEBUG] [VALIDATOR ] LPL: All expected trading days are present. [2025-09-16T19:15:06.110487] [DEBUG] [VALIDATOR ] LPL: Primary data validation passed. [2025-09-16T19:15:06.111020] [INFO ] [MAIN ] LPL: Fetched primary data is valid. [2025-09-16T19:15:06.111070] [DEBUG] [LOCAL ] LPL: Saving primary data. [2025-09-16T19:15:06.111742] [DEBUG] [LOCAL ] LPL: Trimmed 26271 records to 19329 (keeping 150 days of data) [2025-09-16T19:15:06.221018] [DEBUG] [LOCAL ] LPL: Primary data saved (19329 records). [2025-09-16T19:15:06.221154] [INFO ] [MAIN ] LPL: Full history saved. [2025-09-16T19:15:06.221257] [INFO ] [MAIN ] Update Progress: 1274/2369 symbols completed. LPL took 18.6s. Estimated remaining: 1h 56m 5s. Rate: 9.4 symbols/min [2025-09-16T19:15:06.391017] [INFO ] [MAIN ] LSPD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:06.421018] [DEBUG] [LOCAL ] LSPD: Loaded 30931 existing records from primary storage. [2025-09-16T19:15:06.421152] [INFO ] [MAIN ] LSPD: Loaded 30931 existing records. [2025-09-16T19:15:06.598307] [WARN ] [MAIN ] LSPD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:06.598598] [INFO ] [MAIN ] LSPD: Wiping existing primary data before full refresh. [2025-09-16T19:15:06.598672] [INFO ] [LOCAL ] LSPD: Starting deletion of primary data. [2025-09-16T19:15:06.599319] [DEBUG] [LOCAL ] LSPD: Deleted primary data file. [2025-09-16T19:15:06.599370] [INFO ] [MAIN ] LSPD: Starting full primary refresh. [2025-09-16T19:15:06.779031] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-04. [2025-09-16T19:15:06.965556] [INFO ] [REMOTE ] LSPD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:06.988976] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-09-16T19:15:07.469228] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T19:15:07.574955] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-02. [2025-09-16T19:15:08.316274] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-09-16T19:15:08.374957] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-08. [2025-09-16T19:15:09.174940] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-05. [2025-09-16T19:15:09.430904] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-09-16T19:15:09.513923] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T19:15:09.974902] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-09. [2025-09-16T19:15:10.494234] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T19:15:10.774892] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-06. [2025-09-16T19:15:10.993318] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T19:15:10.998207] [INFO ] [REMOTE ] LPG: Full fetch complete (44435 records). [2025-09-16T19:15:10.998317] [DEBUG] [VALIDATOR ] LPG: Validating primary data. [2025-09-16T19:15:11.369517] [DEBUG] [VALIDATOR ] LPG: All expected trading days are present. [2025-09-16T19:15:11.369593] [DEBUG] [VALIDATOR ] LPG: Primary data validation passed. [2025-09-16T19:15:11.370313] [INFO ] [MAIN ] LPG: Fetched primary data is valid. [2025-09-16T19:15:11.370377] [DEBUG] [LOCAL ] LPG: Saving primary data. [2025-09-16T19:15:11.371305] [DEBUG] [LOCAL ] LPG: Trimmed 44435 records to 29760 (keeping 150 days of data) [2025-09-16T19:15:11.559743] [DEBUG] [LOCAL ] LPG: Primary data saved (29760 records). [2025-09-16T19:15:11.559911] [INFO ] [MAIN ] LPG: Full history saved. [2025-09-16T19:15:11.560016] [INFO ] [MAIN ] Update Progress: 1275/2369 symbols completed. LPG took 25.4s. Estimated remaining: 1h 55m 57s. Rate: 9.4 symbols/min [2025-09-16T19:15:11.581347] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-03. [2025-09-16T19:15:11.736219] [INFO ] [MAIN ] LTC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:11.757272] [DEBUG] [LOCAL ] LTC: Loaded 22112 existing records from primary storage. [2025-09-16T19:15:11.757436] [INFO ] [MAIN ] LTC: Loaded 22112 existing records. [2025-09-16T19:15:11.938503] [WARN ] [MAIN ] LTC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:11.938801] [INFO ] [MAIN ] LTC: Wiping existing primary data before full refresh. [2025-09-16T19:15:11.938848] [INFO ] [LOCAL ] LTC: Starting deletion of primary data. [2025-09-16T19:15:11.939465] [DEBUG] [LOCAL ] LTC: Deleted primary data file. [2025-09-16T19:15:11.939516] [INFO ] [MAIN ] LTC: Starting full primary refresh. [2025-09-16T19:15:12.331237] [INFO ] [REMOTE ] LTC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:12.360289] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T19:15:12.374864] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-07. [2025-09-16T19:15:13.064803] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-09-16T19:15:13.174929] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-04. [2025-09-16T19:15:13.288251] [DEBUG] [REMOTE ] LRN: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-09-16T19:15:13.288575] [WARN ] [REMOTE ] LRN: No data returned for month 2025-04 [2025-09-16T19:15:13.850678] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T19:15:13.974857] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-08. [2025-09-16T19:15:14.774940] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-05. [2025-09-16T19:15:15.537178] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T19:15:15.574875] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-09. [2025-09-16T19:15:16.379975] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-02. [2025-09-16T19:15:16.394414] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T19:15:17.080673] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-09-16T19:15:17.085400] [INFO ] [REMOTE ] LPX: Full fetch complete (43576 records). [2025-09-16T19:15:17.085519] [DEBUG] [VALIDATOR ] LPX: Validating primary data. [2025-09-16T19:15:17.182682] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-06. [2025-09-16T19:15:17.459773] [DEBUG] [VALIDATOR ] LPX: All expected trading days are present. [2025-09-16T19:15:17.459848] [DEBUG] [VALIDATOR ] LPX: Primary data validation passed. [2025-09-16T19:15:17.460539] [INFO ] [MAIN ] LPX: Fetched primary data is valid. [2025-09-16T19:15:17.460587] [DEBUG] [LOCAL ] LPX: Saving primary data. [2025-09-16T19:15:17.461500] [DEBUG] [LOCAL ] LPX: Trimmed 43576 records to 29644 (keeping 150 days of data) [2025-09-16T19:15:17.594559] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-09-16T19:15:17.667549] [DEBUG] [LOCAL ] LPX: Primary data saved (29644 records). [2025-09-16T19:15:17.667700] [INFO ] [MAIN ] LPX: Full history saved. [2025-09-16T19:15:17.668371] [INFO ] [MAIN ] Update Progress: 1276/2369 symbols completed. LPX took 17.6s. Estimated remaining: 1h 55m 51s. Rate: 9.4 symbols/min [2025-09-16T19:15:17.844173] [INFO ] [MAIN ] LTH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:17.883293] [DEBUG] [LOCAL ] LTH: Loaded 38227 existing records from primary storage. [2025-09-16T19:15:17.883426] [INFO ] [MAIN ] LTH: Loaded 38227 existing records. [2025-09-16T19:15:17.982215] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-03. [2025-09-16T19:15:18.062198] [WARN ] [MAIN ] LTH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:18.062416] [INFO ] [MAIN ] LTH: Wiping existing primary data before full refresh. [2025-09-16T19:15:18.062463] [INFO ] [LOCAL ] LTH: Starting deletion of primary data. [2025-09-16T19:15:18.063189] [DEBUG] [LOCAL ] LTH: Deleted primary data file. [2025-09-16T19:15:18.063241] [INFO ] [MAIN ] LTH: Starting full primary refresh. [2025-09-16T19:15:18.419277] [INFO ] [REMOTE ] LTH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:18.781459] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-03. [2025-09-16T19:15:18.796869] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-09-16T19:15:19.250088] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T19:15:19.574903] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-02. [2025-09-16T19:15:20.374871] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-04. [2025-09-16T19:15:20.452394] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-09-16T19:15:21.174870] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-04. [2025-09-16T19:15:21.203962] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T19:15:21.974898] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-03. [2025-09-16T19:15:22.100868] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-09-16T19:15:22.351069] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T19:15:22.774932] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-05. [2025-09-16T19:15:23.574919] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-07. [2025-09-16T19:15:23.771459] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T19:15:23.840517] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-09-16T19:15:24.374904] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-04. [2025-09-16T19:15:25.018730] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T19:15:25.174899] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-06. [2025-09-16T19:15:25.881623] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T19:15:25.974870] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-05. [2025-09-16T19:15:26.732792] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T19:15:26.774866] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-07. [2025-09-16T19:15:27.574931] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-08. [2025-09-16T19:15:27.655014] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-09-16T19:15:28.025949] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-09-16T19:15:28.374908] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-06. [2025-09-16T19:15:29.174910] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-05. [2025-09-16T19:15:29.398250] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T19:15:29.791171] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T19:15:29.974900] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-08. [2025-09-16T19:15:30.485775] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T19:15:30.774900] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-07. [2025-09-16T19:15:31.416397] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T19:15:31.574903] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-09. [2025-09-16T19:15:32.238305] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T19:15:32.374893] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-06. [2025-09-16T19:15:33.065104] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T19:15:33.072376] [INFO ] [REMOTE ] LRN: Full fetch complete (36831 records). [2025-09-16T19:15:33.072536] [DEBUG] [VALIDATOR ] LRN: Validating primary data. [2025-09-16T19:15:33.179251] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-08. [2025-09-16T19:15:33.266602] [WARN ] [VALIDATOR ] LRN: Validation failed - insufficient historical data. Have 129 unique days, need 150 days [2025-09-16T19:15:33.267078] [FATAL] [MAIN ] LRN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:15:33.267131] [WARN ] [MAIN ] LRN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:15:33.267176] [INFO ] [LOCAL ] LRN: Starting deletion of primary data. [2025-09-16T19:15:33.267231] [DEBUG] [LOCAL ] LRN: No primary data file found to delete. [2025-09-16T19:15:33.267278] [INFO ] [LOCAL ] LRN: Starting deletion of derivative data. [2025-09-16T19:15:33.267586] [DEBUG] [LOCAL ] LRN: Deleted derivative data file. [2025-09-16T19:15:33.267701] [INFO ] [MAIN ] Update Progress: 1277/2369 symbols completed. LRN took 28.9s. Estimated remaining: 1h 55m 52s. Rate: 9.4 symbols/min [2025-09-16T19:15:33.329374] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-09-16T19:15:33.974904] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-09. [2025-09-16T19:15:34.774916] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-07. [2025-09-16T19:15:34.838072] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T19:15:35.399693] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-09-16T19:15:35.407817] [INFO ] [REMOTE ] LSPD: Full fetch complete (47915 records). [2025-09-16T19:15:35.408061] [DEBUG] [VALIDATOR ] LSPD: Validating primary data. [2025-09-16T19:15:35.581109] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-09. [2025-09-16T19:15:35.847869] [DEBUG] [VALIDATOR ] LSPD: All expected trading days are present. [2025-09-16T19:15:35.848049] [DEBUG] [VALIDATOR ] LSPD: Primary data validation passed. [2025-09-16T19:15:35.849253] [INFO ] [MAIN ] LSPD: Fetched primary data is valid. [2025-09-16T19:15:35.849328] [DEBUG] [LOCAL ] LSPD: Saving primary data. [2025-09-16T19:15:35.851185] [DEBUG] [LOCAL ] LSPD: Trimmed 47915 records to 31234 (keeping 150 days of data) [2025-09-16T19:15:36.055389] [DEBUG] [LOCAL ] LSPD: Primary data saved (31234 records). [2025-09-16T19:15:36.055612] [INFO ] [MAIN ] LSPD: Full history saved. [2025-09-16T19:15:36.055760] [INFO ] [MAIN ] Update Progress: 1278/2369 symbols completed. LSPD took 29.8s. Estimated remaining: 1h 55m 43s. Rate: 9.4 symbols/min [2025-09-16T19:15:36.311140] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-09-16T19:15:36.374878] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-08. [2025-09-16T19:15:36.918524] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-09-16T19:15:36.925339] [INFO ] [REMOTE ] LTH: Full fetch complete (56378 records). [2025-09-16T19:15:36.925506] [DEBUG] [VALIDATOR ] LTH: Validating primary data. [2025-09-16T19:15:37.332337] [DEBUG] [VALIDATOR ] LTH: All expected trading days are present. [2025-09-16T19:15:37.332477] [DEBUG] [VALIDATOR ] LTH: Primary data validation passed. [2025-09-16T19:15:37.333971] [INFO ] [MAIN ] LTH: Fetched primary data is valid. [2025-09-16T19:15:37.334175] [DEBUG] [LOCAL ] LTH: Saving primary data. [2025-09-16T19:15:37.336313] [DEBUG] [LOCAL ] LTH: Trimmed 56378 records to 38610 (keeping 150 days of data) [2025-09-16T19:15:37.590808] [DEBUG] [LOCAL ] LTH: Primary data saved (38610 records). [2025-09-16T19:15:37.609092] [INFO ] [MAIN ] LTH: Full history saved. [2025-09-16T19:15:37.610358] [INFO ] [MAIN ] Update Progress: 1279/2369 symbols completed. LTH took 19.9s. Estimated remaining: 1h 55m 32s. Rate: 9.4 symbols/min [2025-09-16T19:15:37.611123] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T19:15:37.618202] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-09. [2025-09-16T19:15:38.799066] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-09-16T19:15:38.802625] [INFO ] [REMOTE ] LTC: Full fetch complete (32479 records). [2025-09-16T19:15:38.802726] [DEBUG] [VALIDATOR ] LTC: Validating primary data. [2025-09-16T19:15:39.173321] [DEBUG] [VALIDATOR ] LTC: All expected trading days are present. [2025-09-16T19:15:39.173405] [DEBUG] [VALIDATOR ] LTC: Primary data validation passed. [2025-09-16T19:15:39.174174] [INFO ] [MAIN ] LTC: Fetched primary data is valid. [2025-09-16T19:15:39.174226] [DEBUG] [LOCAL ] LTC: Saving primary data. [2025-09-16T19:15:39.175038] [DEBUG] [LOCAL ] LTC: Trimmed 32479 records to 22342 (keeping 150 days of data) [2025-09-16T19:15:39.313063] [DEBUG] [LOCAL ] LTC: Primary data saved (22342 records). [2025-09-16T19:15:39.313216] [INFO ] [MAIN ] LTC: Full history saved. [2025-09-16T19:15:39.313347] [INFO ] [MAIN ] Update Progress: 1280/2369 symbols completed. LTC took 27.8s. Estimated remaining: 1h 55m 22s. Rate: 9.4 symbols/min [2025-09-16T19:15:39.874492] [INFO ] [MAIN ] LUMN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:39.987394] [INFO ] [MAIN ] LUCK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:39.985237] [INFO ] [MAIN ] LTM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:40.006075] [INFO ] [MAIN ] LU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:40.057242] [DEBUG] [LOCAL ] LUCK: Loaded 17494 existing records from primary storage. [2025-09-16T19:15:40.057579] [INFO ] [MAIN ] LUCK: Loaded 17494 existing records. [2025-09-16T19:15:40.141871] [DEBUG] [LOCAL ] LTM: Loaded 27982 existing records from primary storage. [2025-09-16T19:15:40.168250] [INFO ] [MAIN ] LTM: Loaded 27982 existing records. [2025-09-16T19:15:40.174107] [DEBUG] [LOCAL ] LUMN: Loaded 39620 existing records from primary storage. [2025-09-16T19:15:40.200377] [INFO ] [MAIN ] LUMN: Loaded 39620 existing records. [2025-09-16T19:15:40.250138] [DEBUG] [LOCAL ] LU: Loaded 36227 existing records from primary storage. [2025-09-16T19:15:40.296253] [INFO ] [MAIN ] LU: Loaded 36227 existing records. [2025-09-16T19:15:40.577196] [WARN ] [MAIN ] LUCK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:40.670942] [INFO ] [MAIN ] LUCK: Wiping existing primary data before full refresh. [2025-09-16T19:15:40.712842] [WARN ] [MAIN ] LTM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:40.743875] [INFO ] [LOCAL ] LUCK: Starting deletion of primary data. [2025-09-16T19:15:40.834408] [WARN ] [MAIN ] LUMN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:40.834627] [INFO ] [MAIN ] LUMN: Wiping existing primary data before full refresh. [2025-09-16T19:15:40.834292] [WARN ] [MAIN ] LU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:40.834699] [INFO ] [LOCAL ] LUMN: Starting deletion of primary data. [2025-09-16T19:15:40.834907] [INFO ] [MAIN ] LU: Wiping existing primary data before full refresh. [2025-09-16T19:15:40.834956] [INFO ] [LOCAL ] LU: Starting deletion of primary data. [2025-09-16T19:15:40.835009] [DEBUG] [LOCAL ] LUCK: Deleted primary data file. [2025-09-16T19:15:40.835070] [INFO ] [MAIN ] LUCK: Starting full primary refresh. [2025-09-16T19:15:40.841010] [INFO ] [MAIN ] LTM: Wiping existing primary data before full refresh. [2025-09-16T19:15:40.840839] [DEBUG] [LOCAL ] LU: Deleted primary data file. [2025-09-16T19:15:40.840784] [DEBUG] [LOCAL ] LUMN: Deleted primary data file. [2025-09-16T19:15:40.853823] [INFO ] [MAIN ] LU: Starting full primary refresh. [2025-09-16T19:15:40.885190] [INFO ] [LOCAL ] LTM: Starting deletion of primary data. [2025-09-16T19:15:40.885116] [INFO ] [MAIN ] LUMN: Starting full primary refresh. [2025-09-16T19:15:40.946988] [DEBUG] [LOCAL ] LTM: Deleted primary data file. [2025-09-16T19:15:40.978089] [INFO ] [MAIN ] LTM: Starting full primary refresh. [2025-09-16T19:15:42.166824] [INFO ] [REMOTE ] LUMN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:42.291208] [INFO ] [REMOTE ] LUCK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:42.337768] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-02. [2025-09-16T19:15:42.326625] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-02. [2025-09-16T19:15:42.369054] [INFO ] [REMOTE ] LU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:42.373903] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-02. [2025-09-16T19:15:42.462346] [INFO ] [REMOTE ] LTM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:42.462779] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-02. [2025-09-16T19:15:43.630765] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-09-16T19:15:43.636570] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-03. [2025-09-16T19:15:43.765208] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-09-16T19:15:43.772767] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-03. [2025-09-16T19:15:43.934536] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T19:15:43.944683] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-03. [2025-09-16T19:15:44.002579] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T19:15:44.012163] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-03. [2025-09-16T19:15:44.577366] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T19:15:44.583751] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-04. [2025-09-16T19:15:45.093288] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-09-16T19:15:45.174940] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-04. [2025-09-16T19:15:45.749933] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T19:15:45.836933] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-09-16T19:15:45.975575] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-05. [2025-09-16T19:15:45.978763] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-09-16T19:15:46.221158] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-09-16T19:15:46.774896] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-04. [2025-09-16T19:15:47.422556] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-09-16T19:15:47.574871] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-06. [2025-09-16T19:15:48.374978] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-04. [2025-09-16T19:15:48.478294] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-09-16T19:15:48.938473] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-09-16T19:15:49.174917] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-05. [2025-09-16T19:15:49.974886] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-07. [2025-09-16T19:15:50.083418] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-09-16T19:15:50.774897] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-05. [2025-09-16T19:15:50.860576] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-09-16T19:15:51.430581] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T19:15:51.574907] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-08. [2025-09-16T19:15:52.374934] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-05. [2025-09-16T19:15:52.412080] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T19:15:53.133260] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-09-16T19:15:53.174866] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-09. [2025-09-16T19:15:53.566085] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-09-16T19:15:53.974905] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-06. [2025-09-16T19:15:54.687702] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-09-16T19:15:54.693409] [INFO ] [REMOTE ] LTM: Full fetch complete (37654 records). [2025-09-16T19:15:54.693531] [DEBUG] [VALIDATOR ] LTM: Validating primary data. [2025-09-16T19:15:54.784735] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-06. [2025-09-16T19:15:55.082221] [DEBUG] [VALIDATOR ] LTM: All expected trading days are present. [2025-09-16T19:15:55.082346] [DEBUG] [VALIDATOR ] LTM: Primary data validation passed. [2025-09-16T19:15:55.083095] [INFO ] [MAIN ] LTM: Fetched primary data is valid. [2025-09-16T19:15:55.083170] [DEBUG] [LOCAL ] LTM: Saving primary data. [2025-09-16T19:15:55.098895] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-09-16T19:15:55.104517] [DEBUG] [LOCAL ] LTM: Trimmed 37654 records to 28294 (keeping 150 days of data) [2025-09-16T19:15:55.275097] [DEBUG] [LOCAL ] LTM: Primary data saved (28294 records). [2025-09-16T19:15:55.275229] [INFO ] [MAIN ] LTM: Full history saved. [2025-09-16T19:15:55.275363] [INFO ] [MAIN ] Update Progress: 1281/2369 symbols completed. LTM took 16.0s. Estimated remaining: 1h 55m 24s. Rate: 9.4 symbols/min [2025-09-16T19:15:55.449141] [INFO ] [MAIN ] LUV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:15:55.490299] [DEBUG] [LOCAL ] LUV: Loaded 39649 existing records from primary storage. [2025-09-16T19:15:55.490439] [INFO ] [MAIN ] LUV: Loaded 39649 existing records. [2025-09-16T19:15:55.580084] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-06. [2025-09-16T19:15:55.677194] [WARN ] [MAIN ] LUV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:15:55.677438] [INFO ] [MAIN ] LUV: Wiping existing primary data before full refresh. [2025-09-16T19:15:55.677484] [INFO ] [LOCAL ] LUV: Starting deletion of primary data. [2025-09-16T19:15:55.678485] [DEBUG] [LOCAL ] LUV: Deleted primary data file. [2025-09-16T19:15:55.678557] [INFO ] [MAIN ] LUV: Starting full primary refresh. [2025-09-16T19:15:56.050823] [INFO ] [REMOTE ] LUV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:15:56.374909] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-02. [2025-09-16T19:15:56.481610] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T19:15:57.174964] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-07. [2025-09-16T19:15:57.566871] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-09-16T19:15:57.906903] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T19:15:57.974987] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-07. [2025-09-16T19:15:58.774915] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-03. [2025-09-16T19:15:58.891123] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-09-16T19:15:59.193818] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-09-16T19:15:59.574948] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-08. [2025-09-16T19:16:00.374911] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-07. [2025-09-16T19:16:00.616716] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T19:16:00.809992] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-09-16T19:16:01.174941] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-04. [2025-09-16T19:16:01.974909] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-09. [2025-09-16T19:16:02.331704] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-09-16T19:16:02.774984] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-08. [2025-09-16T19:16:03.287924] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-04 (2.11 seconds). [2025-09-16T19:16:03.574967] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-05. [2025-09-16T19:16:03.673554] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T19:16:03.683524] [INFO ] [REMOTE ] LU: Full fetch complete (53134 records). [2025-09-16T19:16:03.683741] [DEBUG] [VALIDATOR ] LU: Validating primary data. [2025-09-16T19:16:04.072566] [DEBUG] [VALIDATOR ] LU: All expected trading days are present. [2025-09-16T19:16:04.072667] [DEBUG] [VALIDATOR ] LU: Primary data validation passed. [2025-09-16T19:16:04.073484] [INFO ] [MAIN ] LU: Fetched primary data is valid. [2025-09-16T19:16:04.073530] [DEBUG] [LOCAL ] LU: Saving primary data. [2025-09-16T19:16:04.074704] [DEBUG] [LOCAL ] LU: Trimmed 53134 records to 36575 (keeping 150 days of data) [2025-09-16T19:16:04.282840] [DEBUG] [LOCAL ] LU: Primary data saved (36575 records). [2025-09-16T19:16:04.282991] [INFO ] [MAIN ] LU: Full history saved. [2025-09-16T19:16:04.283092] [INFO ] [MAIN ] Update Progress: 1282/2369 symbols completed. LU took 25.0s. Estimated remaining: 1h 55m 20s. Rate: 9.4 symbols/min [2025-09-16T19:16:04.381513] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-08. [2025-09-16T19:16:04.467481] [INFO ] [MAIN ] LUXE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:04.467721] [DEBUG] [LOCAL ] LUXE: No existing data file found in primary storage. [2025-09-16T19:16:04.467762] [WARN ] [MAIN ] LUXE: No existing data found, performing full refresh instead. [2025-09-16T19:16:04.467887] [INFO ] [MAIN ] LUXE: Wiping existing primary data before full refresh. [2025-09-16T19:16:04.467934] [INFO ] [LOCAL ] LUXE: Starting deletion of primary data. [2025-09-16T19:16:04.467977] [DEBUG] [LOCAL ] LUXE: No primary data file found to delete. [2025-09-16T19:16:04.468009] [INFO ] [MAIN ] LUXE: Starting full primary refresh. [2025-09-16T19:16:04.848460] [INFO ] [REMOTE ] LUXE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:04.888171] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-09-16T19:16:05.174939] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-02. [2025-09-16T19:16:05.390450] [DEBUG] [REMOTE ] LUXE: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:16:05.391179] [WARN ] [REMOTE ] LUXE: No data returned for month 2025-02 [2025-09-16T19:16:05.421202] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-09-16T19:16:05.458465] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-09-16T19:16:05.974908] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-09. [2025-09-16T19:16:06.774888] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-06. [2025-09-16T19:16:06.881116] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-09-16T19:16:06.884267] [INFO ] [REMOTE ] LUCK: Full fetch complete (28776 records). [2025-09-16T19:16:06.884356] [DEBUG] [VALIDATOR ] LUCK: Validating primary data. [2025-09-16T19:16:07.283874] [DEBUG] [VALIDATOR ] LUCK: All expected trading days are present. [2025-09-16T19:16:07.284005] [DEBUG] [VALIDATOR ] LUCK: Primary data validation passed. [2025-09-16T19:16:07.284672] [INFO ] [MAIN ] LUCK: Fetched primary data is valid. [2025-09-16T19:16:07.284720] [DEBUG] [LOCAL ] LUCK: Saving primary data. [2025-09-16T19:16:07.285728] [DEBUG] [LOCAL ] LUCK: Trimmed 28776 records to 17666 (keeping 150 days of data) [2025-09-16T19:16:07.396873] [DEBUG] [LOCAL ] LUCK: Primary data saved (17666 records). [2025-09-16T19:16:07.397033] [INFO ] [MAIN ] LUCK: Full history saved. [2025-09-16T19:16:07.397139] [INFO ] [MAIN ] Update Progress: 1283/2369 symbols completed. LUCK took 28.1s. Estimated remaining: 1h 55m 11s. Rate: 9.4 symbols/min [2025-09-16T19:16:07.569519] [INFO ] [MAIN ] LVS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:07.574915] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-09. [2025-09-16T19:16:07.614700] [DEBUG] [LOCAL ] LVS: Loaded 39539 existing records from primary storage. [2025-09-16T19:16:07.614864] [INFO ] [MAIN ] LVS: Loaded 39539 existing records. [2025-09-16T19:16:07.815848] [WARN ] [MAIN ] LVS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:07.816143] [INFO ] [MAIN ] LVS: Wiping existing primary data before full refresh. [2025-09-16T19:16:07.816192] [INFO ] [LOCAL ] LVS: Starting deletion of primary data. [2025-09-16T19:16:07.817469] [DEBUG] [LOCAL ] LVS: Deleted primary data file. [2025-09-16T19:16:07.817529] [INFO ] [MAIN ] LVS: Starting full primary refresh. [2025-09-16T19:16:08.193638] [INFO ] [REMOTE ] LVS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:08.342653] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-09-16T19:16:08.374847] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-07. [2025-09-16T19:16:09.174953] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-03. [2025-09-16T19:16:09.396615] [DEBUG] [REMOTE ] LUXE: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:16:09.397122] [WARN ] [REMOTE ] LUXE: No data returned for month 2025-03 [2025-09-16T19:16:09.733188] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-09-16T19:16:09.739613] [INFO ] [REMOTE ] LUMN: Full fetch complete (58306 records). [2025-09-16T19:16:09.739753] [DEBUG] [VALIDATOR ] LUMN: Validating primary data. [2025-09-16T19:16:09.979707] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-02. [2025-09-16T19:16:10.124458] [DEBUG] [VALIDATOR ] LUMN: All expected trading days are present. [2025-09-16T19:16:10.124556] [DEBUG] [VALIDATOR ] LUMN: Primary data validation passed. [2025-09-16T19:16:10.125436] [INFO ] [MAIN ] LUMN: Fetched primary data is valid. [2025-09-16T19:16:10.125486] [DEBUG] [LOCAL ] LUMN: Saving primary data. [2025-09-16T19:16:10.126773] [DEBUG] [LOCAL ] LUMN: Trimmed 58306 records to 40009 (keeping 150 days of data) [2025-09-16T19:16:10.392234] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-09-16T19:16:10.424688] [DEBUG] [LOCAL ] LUMN: Primary data saved (40009 records). [2025-09-16T19:16:10.424873] [INFO ] [MAIN ] LUMN: Full history saved. [2025-09-16T19:16:10.424991] [INFO ] [MAIN ] Update Progress: 1284/2369 symbols completed. LUMN took 31.1s. Estimated remaining: 1h 55m 2s. Rate: 9.4 symbols/min [2025-09-16T19:16:10.615556] [INFO ] [MAIN ] LVWR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:10.631133] [DEBUG] [LOCAL ] LVWR: Loaded 15660 existing records from primary storage. [2025-09-16T19:16:10.631261] [INFO ] [MAIN ] LVWR: Loaded 15660 existing records. [2025-09-16T19:16:10.776902] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-04. [2025-09-16T19:16:10.810359] [WARN ] [MAIN ] LVWR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:10.810604] [INFO ] [MAIN ] LVWR: Wiping existing primary data before full refresh. [2025-09-16T19:16:10.810672] [INFO ] [LOCAL ] LVWR: Starting deletion of primary data. [2025-09-16T19:16:10.811179] [DEBUG] [LOCAL ] LVWR: Deleted primary data file. [2025-09-16T19:16:10.811224] [INFO ] [MAIN ] LVWR: Starting full primary refresh. [2025-09-16T19:16:11.062804] [DEBUG] [REMOTE ] LUXE: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:16:11.090288] [WARN ] [REMOTE ] LUXE: No data returned for month 2025-04 [2025-09-16T19:16:11.176337] [INFO ] [REMOTE ] LVWR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:11.571201] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-09-16T19:16:11.580513] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-05. [2025-09-16T19:16:12.374867] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-03. [2025-09-16T19:16:12.729307] [INFO ] [REMOTE ] LUXE: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-09-16T19:16:13.174933] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-08. [2025-09-16T19:16:13.974913] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-02. [2025-09-16T19:16:14.085442] [DEBUG] [REMOTE ] LVWR: Error fetching primary data for 2025-02: 'API call returned an empty response.' [2025-09-16T19:16:14.085792] [WARN ] [REMOTE ] LVWR: No data returned for month 2025-02 [2025-09-16T19:16:14.222547] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-09-16T19:16:14.774892] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-03. [2025-09-16T19:16:14.911237] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-09-16T19:16:15.146113] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-09-16T19:16:15.574880] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-04. [2025-09-16T19:16:16.208801] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-09-16T19:16:16.374908] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-04. [2025-09-16T19:16:17.174912] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-06. [2025-09-16T19:16:17.974904] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-09. [2025-09-16T19:16:18.073349] [INFO ] [REMOTE ] LUXE: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-09-16T19:16:18.391178] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-09-16T19:16:18.774938] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-05. [2025-09-16T19:16:19.574903] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-07. [2025-09-16T19:16:19.972975] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-09-16T19:16:19.979840] [INFO ] [REMOTE ] LUV: Full fetch complete (58368 records). [2025-09-16T19:16:19.989206] [DEBUG] [VALIDATOR ] LUV: Validating primary data. [2025-09-16T19:16:19.990138] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T19:16:20.381569] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-05. [2025-09-16T19:16:20.382299] [DEBUG] [VALIDATOR ] LUV: All expected trading days are present. [2025-09-16T19:16:20.383275] [DEBUG] [VALIDATOR ] LUV: Primary data validation passed. [2025-09-16T19:16:20.384210] [INFO ] [MAIN ] LUV: Fetched primary data is valid. [2025-09-16T19:16:20.384353] [DEBUG] [LOCAL ] LUV: Saving primary data. [2025-09-16T19:16:20.385662] [DEBUG] [LOCAL ] LUV: Trimmed 58368 records to 40039 (keeping 150 days of data) [2025-09-16T19:16:20.586909] [INFO ] [REMOTE ] LUXE: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-09-16T19:16:20.655000] [DEBUG] [LOCAL ] LUV: Primary data saved (40039 records). [2025-09-16T19:16:20.655134] [INFO ] [MAIN ] LUV: Full history saved. [2025-09-16T19:16:20.656128] [INFO ] [MAIN ] Update Progress: 1285/2369 symbols completed. LUV took 25.4s. Estimated remaining: 1h 54m 58s. Rate: 9.4 symbols/min [2025-09-16T19:16:20.823553] [INFO ] [MAIN ] LW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:20.860720] [DEBUG] [LOCAL ] LW: Loaded 37616 existing records from primary storage. [2025-09-16T19:16:20.860857] [INFO ] [MAIN ] LW: Loaded 37616 existing records. [2025-09-16T19:16:21.044453] [WARN ] [MAIN ] LW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:21.044716] [INFO ] [MAIN ] LW: Wiping existing primary data before full refresh. [2025-09-16T19:16:21.044764] [INFO ] [LOCAL ] LW: Starting deletion of primary data. [2025-09-16T19:16:21.045754] [DEBUG] [LOCAL ] LW: Deleted primary data file. [2025-09-16T19:16:21.045803] [INFO ] [MAIN ] LW: Starting full primary refresh. [2025-09-16T19:16:21.186911] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-06. [2025-09-16T19:16:21.402473] [INFO ] [REMOTE ] LW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:21.974914] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-02. [2025-09-16T19:16:22.311773] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-09-16T19:16:22.774908] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-06. [2025-09-16T19:16:22.992947] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-09-16T19:16:23.574927] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-08. [2025-09-16T19:16:23.621148] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-09-16T19:16:24.374933] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-03. [2025-09-16T19:16:24.483474] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T19:16:24.506360] [INFO ] [REMOTE ] LUXE: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T19:16:25.174910] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-09. [2025-09-16T19:16:25.936276] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-09-16T19:16:25.974886] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-04. [2025-09-16T19:16:26.128472] [INFO ] [REMOTE ] LUXE: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T19:16:26.131521] [INFO ] [REMOTE ] LUXE: Full fetch complete (11831 records). [2025-09-16T19:16:26.131603] [DEBUG] [VALIDATOR ] LUXE: Validating primary data. [2025-09-16T19:16:26.315013] [WARN ] [VALIDATOR ] LUXE: Validation failed - insufficient historical data. Have 95 unique days, need 150 days [2025-09-16T19:16:26.315267] [FATAL] [MAIN ] LUXE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:16:26.315315] [WARN ] [MAIN ] LUXE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:16:26.315359] [INFO ] [LOCAL ] LUXE: Starting deletion of primary data. [2025-09-16T19:16:26.315445] [DEBUG] [LOCAL ] LUXE: No primary data file found to delete. [2025-09-16T19:16:26.315483] [INFO ] [LOCAL ] LUXE: Starting deletion of derivative data. [2025-09-16T19:16:26.315533] [DEBUG] [LOCAL ] LUXE: No derivative data file found to delete. [2025-09-16T19:16:26.315616] [INFO ] [MAIN ] Update Progress: 1286/2369 symbols completed. LUXE took 22.0s. Estimated remaining: 1h 54m 51s. Rate: 9.4 symbols/min [2025-09-16T19:16:26.485847] [INFO ] [MAIN ] LXFR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:26.499774] [DEBUG] [LOCAL ] LXFR: Loaded 12308 existing records from primary storage. [2025-09-16T19:16:26.499910] [INFO ] [MAIN ] LXFR: Loaded 12308 existing records. [2025-09-16T19:16:26.678478] [WARN ] [MAIN ] LXFR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:26.678760] [INFO ] [MAIN ] LXFR: Wiping existing primary data before full refresh. [2025-09-16T19:16:26.678810] [INFO ] [LOCAL ] LXFR: Starting deletion of primary data. [2025-09-16T19:16:26.679226] [DEBUG] [LOCAL ] LXFR: Deleted primary data file. [2025-09-16T19:16:26.679273] [INFO ] [MAIN ] LXFR: Starting full primary refresh. [2025-09-16T19:16:26.782037] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-07. [2025-09-16T19:16:27.049629] [INFO ] [REMOTE ] LXFR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:27.574877] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-02. [2025-09-16T19:16:27.693498] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-09-16T19:16:27.819104] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T19:16:28.374895] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-08. [2025-09-16T19:16:28.585617] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-09-16T19:16:29.174900] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-07. [2025-09-16T19:16:29.238076] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-09-16T19:16:29.974946] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-09. [2025-09-16T19:16:30.774867] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-05. [2025-09-16T19:16:30.984212] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T19:16:30.987042] [INFO ] [REMOTE ] LVWR: Full fetch complete (16869 records). [2025-09-16T19:16:30.987129] [DEBUG] [VALIDATOR ] LVWR: Validating primary data. [2025-09-16T19:16:31.168950] [WARN ] [VALIDATOR ] LVWR: Validation failed - insufficient historical data. Have 137 unique days, need 150 days [2025-09-16T19:16:31.169352] [FATAL] [MAIN ] LVWR: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:16:31.169490] [WARN ] [MAIN ] LVWR: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:16:31.169620] [INFO ] [LOCAL ] LVWR: Starting deletion of primary data. [2025-09-16T19:16:31.169755] [DEBUG] [LOCAL ] LVWR: No primary data file found to delete. [2025-09-16T19:16:31.169825] [INFO ] [LOCAL ] LVWR: Starting deletion of derivative data. [2025-09-16T19:16:31.170143] [DEBUG] [LOCAL ] LVWR: Deleted derivative data file. [2025-09-16T19:16:31.170226] [INFO ] [MAIN ] Update Progress: 1287/2369 symbols completed. LVWR took 20.7s. Estimated remaining: 1h 54m 44s. Rate: 9.4 symbols/min [2025-09-16T19:16:31.341470] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-09-16T19:16:31.380984] [INFO ] [MAIN ] LXP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:31.415623] [DEBUG] [LOCAL ] LXP: Loaded 37393 existing records from primary storage. [2025-09-16T19:16:31.415802] [INFO ] [MAIN ] LXP: Loaded 37393 existing records. [2025-09-16T19:16:31.583662] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-03. [2025-09-16T19:16:31.594811] [WARN ] [MAIN ] LXP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:31.595063] [INFO ] [MAIN ] LXP: Wiping existing primary data before full refresh. [2025-09-16T19:16:31.595125] [INFO ] [LOCAL ] LXP: Starting deletion of primary data. [2025-09-16T19:16:31.595970] [DEBUG] [LOCAL ] LXP: Deleted primary data file. [2025-09-16T19:16:31.596033] [INFO ] [MAIN ] LXP: Starting full primary refresh. [2025-09-16T19:16:31.957557] [INFO ] [REMOTE ] LXP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:32.374894] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-02. [2025-09-16T19:16:32.539699] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T19:16:32.661211] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-09-16T19:16:33.174943] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-04. [2025-09-16T19:16:33.928777] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T19:16:33.974879] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-03. [2025-09-16T19:16:34.215902] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T19:16:34.774899] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-05. [2025-09-16T19:16:35.570154] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-09-16T19:16:35.580029] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-06. [2025-09-16T19:16:35.780597] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T19:16:36.374933] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-08. [2025-09-16T19:16:36.966103] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T19:16:37.174890] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-07. [2025-09-16T19:16:37.851180] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T19:16:37.974876] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-09. [2025-09-16T19:16:38.682754] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-09-16T19:16:38.774865] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-08. [2025-09-16T19:16:39.574891] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-04. [2025-09-16T19:16:39.859437] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-09-16T19:16:39.866201] [INFO ] [REMOTE ] LVS: Full fetch complete (58244 records). [2025-09-16T19:16:39.866331] [DEBUG] [VALIDATOR ] LVS: Validating primary data. [2025-09-16T19:16:40.259465] [DEBUG] [VALIDATOR ] LVS: All expected trading days are present. [2025-09-16T19:16:40.259550] [DEBUG] [VALIDATOR ] LVS: Primary data validation passed. [2025-09-16T19:16:40.260517] [INFO ] [MAIN ] LVS: Fetched primary data is valid. [2025-09-16T19:16:40.260568] [DEBUG] [LOCAL ] LVS: Saving primary data. [2025-09-16T19:16:40.261851] [DEBUG] [LOCAL ] LVS: Trimmed 58244 records to 39929 (keeping 150 days of data) [2025-09-16T19:16:40.375913] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-06. [2025-09-16T19:16:40.422089] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T19:16:40.557660] [DEBUG] [LOCAL ] LVS: Primary data saved (39929 records). [2025-09-16T19:16:40.557804] [INFO ] [MAIN ] LVS: Full history saved. [2025-09-16T19:16:40.559022] [INFO ] [MAIN ] Update Progress: 1288/2369 symbols completed. LVS took 33.2s. Estimated remaining: 1h 54m 40s. Rate: 9.4 symbols/min [2025-09-16T19:16:40.727762] [INFO ] [MAIN ] LXU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:40.750608] [DEBUG] [LOCAL ] LXU: Loaded 23480 existing records from primary storage. [2025-09-16T19:16:40.750762] [INFO ] [MAIN ] LXU: Loaded 23480 existing records. [2025-09-16T19:16:40.925188] [WARN ] [MAIN ] LXU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:40.925441] [INFO ] [MAIN ] LXU: Wiping existing primary data before full refresh. [2025-09-16T19:16:40.925490] [INFO ] [LOCAL ] LXU: Starting deletion of primary data. [2025-09-16T19:16:40.926056] [DEBUG] [LOCAL ] LXU: Deleted primary data file. [2025-09-16T19:16:40.926106] [INFO ] [MAIN ] LXU: Starting full primary refresh. [2025-09-16T19:16:41.178556] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-09. [2025-09-16T19:16:41.292056] [INFO ] [REMOTE ] LXU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:41.323701] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-09-16T19:16:41.344457] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-09-16T19:16:41.974907] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-02. [2025-09-16T19:16:42.582963] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T19:16:42.589176] [INFO ] [REMOTE ] LW: Full fetch complete (55912 records). [2025-09-16T19:16:42.589283] [DEBUG] [VALIDATOR ] LW: Validating primary data. [2025-09-16T19:16:42.784344] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-05. [2025-09-16T19:16:42.991862] [DEBUG] [VALIDATOR ] LW: All expected trading days are present. [2025-09-16T19:16:42.991994] [DEBUG] [VALIDATOR ] LW: Primary data validation passed. [2025-09-16T19:16:42.992859] [INFO ] [MAIN ] LW: Fetched primary data is valid. [2025-09-16T19:16:42.992906] [DEBUG] [LOCAL ] LW: Saving primary data. [2025-09-16T19:16:42.994262] [DEBUG] [LOCAL ] LW: Trimmed 55912 records to 37983 (keeping 150 days of data) [2025-09-16T19:16:43.220023] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T19:16:43.255006] [DEBUG] [LOCAL ] LW: Primary data saved (37983 records). [2025-09-16T19:16:43.255195] [INFO ] [MAIN ] LW: Full history saved. [2025-09-16T19:16:43.256094] [INFO ] [MAIN ] Update Progress: 1289/2369 symbols completed. LW took 22.6s. Estimated remaining: 1h 54m 31s. Rate: 9.4 symbols/min [2025-09-16T19:16:43.423894] [INFO ] [MAIN ] LYB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:43.462954] [DEBUG] [LOCAL ] LYB: Loaded 39585 existing records from primary storage. [2025-09-16T19:16:43.463097] [INFO ] [MAIN ] LYB: Loaded 39585 existing records. [2025-09-16T19:16:43.588042] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-03. [2025-09-16T19:16:43.671248] [WARN ] [MAIN ] LYB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:43.671506] [INFO ] [MAIN ] LYB: Wiping existing primary data before full refresh. [2025-09-16T19:16:43.671568] [INFO ] [LOCAL ] LYB: Starting deletion of primary data. [2025-09-16T19:16:43.672552] [DEBUG] [LOCAL ] LYB: Deleted primary data file. [2025-09-16T19:16:43.672771] [INFO ] [MAIN ] LYB: Starting full primary refresh. [2025-09-16T19:16:44.031054] [INFO ] [REMOTE ] LYB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:44.374934] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-07. [2025-09-16T19:16:44.529268] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T19:16:45.078151] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-09-16T19:16:45.174949] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-04. [2025-09-16T19:16:45.349360] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T19:16:45.974902] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-06. [2025-09-16T19:16:46.592808] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-09-16T19:16:46.774851] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-05. [2025-09-16T19:16:47.530406] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T19:16:47.574845] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-07. [2025-09-16T19:16:47.908320] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-09-16T19:16:48.374957] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-06. [2025-09-16T19:16:49.174971] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-02. [2025-09-16T19:16:49.309990] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T19:16:49.376908] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T19:16:49.974952] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-08. [2025-09-16T19:16:50.774878] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-07. [2025-09-16T19:16:50.861292] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T19:16:51.023961] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T19:16:51.574870] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-03. [2025-09-16T19:16:52.035807] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-09-16T19:16:52.374911] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-08. [2025-09-16T19:16:53.174871] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-08. [2025-09-16T19:16:53.437134] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-09-16T19:16:53.672054] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-09-16T19:16:53.974920] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-04. [2025-09-16T19:16:54.425941] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-09-16T19:16:54.774906] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-09. [2025-09-16T19:16:55.574938] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-09. [2025-09-16T19:16:55.800113] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T19:16:56.366231] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-09-16T19:16:56.372029] [INFO ] [REMOTE ] LXP: Full fetch complete (54783 records). [2025-09-16T19:16:56.372124] [DEBUG] [VALIDATOR ] LXP: Validating primary data. [2025-09-16T19:16:56.383901] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-09. [2025-09-16T19:16:56.492907] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T19:16:56.544851] [INFO ] [REMOTE ] LXFR: Full fetch complete (18853 records). [2025-09-16T19:16:56.568932] [DEBUG] [VALIDATOR ] LXFR: Validating primary data. [2025-09-16T19:16:57.031868] [DEBUG] [VALIDATOR ] LXP: All expected trading days are present. [2025-09-16T19:16:57.043469] [DEBUG] [VALIDATOR ] LXP: Primary data validation passed. [2025-09-16T19:16:57.065093] [INFO ] [MAIN ] LXP: Fetched primary data is valid. [2025-09-16T19:16:57.065174] [DEBUG] [LOCAL ] LXP: Saving primary data. [2025-09-16T19:16:57.094694] [DEBUG] [LOCAL ] LXP: Trimmed 54783 records to 37755 (keeping 150 days of data) [2025-09-16T19:16:57.151368] [DEBUG] [VALIDATOR ] LXFR: All expected trading days are present. [2025-09-16T19:16:57.151446] [DEBUG] [VALIDATOR ] LXFR: Primary data validation passed. [2025-09-16T19:16:57.151670] [INFO ] [MAIN ] LXFR: Fetched primary data is valid. [2025-09-16T19:16:57.151716] [DEBUG] [LOCAL ] LXFR: Saving primary data. [2025-09-16T19:16:57.152372] [DEBUG] [LOCAL ] LXFR: Trimmed 18853 records to 12410 (keeping 150 days of data) [2025-09-16T19:16:57.204235] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-05. [2025-09-16T19:16:57.321245] [DEBUG] [LOCAL ] LXFR: Primary data saved (12410 records). [2025-09-16T19:16:57.321873] [INFO ] [MAIN ] LXFR: Full history saved. [2025-09-16T19:16:57.322469] [INFO ] [MAIN ] Update Progress: 1290/2369 symbols completed. LXFR took 31.0s. Estimated remaining: 1h 54m 31s. Rate: 9.4 symbols/min [2025-09-16T19:16:57.504901] [INFO ] [MAIN ] LYG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:57.643716] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T19:16:57.648822] [INFO ] [REMOTE ] LXU: Full fetch complete (34714 records). [2025-09-16T19:16:57.650266] [DEBUG] [VALIDATOR ] LXU: Validating primary data. [2025-09-16T19:16:57.667535] [DEBUG] [LOCAL ] LYG: Loaded 38981 existing records from primary storage. [2025-09-16T19:16:57.709700] [DEBUG] [LOCAL ] LXP: Primary data saved (37755 records). [2025-09-16T19:16:57.709573] [INFO ] [MAIN ] LYG: Loaded 38981 existing records. [2025-09-16T19:16:57.726352] [INFO ] [MAIN ] LXP: Full history saved. [2025-09-16T19:16:57.771167] [INFO ] [MAIN ] Update Progress: 1291/2369 symbols completed. LXP took 26.6s. Estimated remaining: 1h 54m 19s. Rate: 9.4 symbols/min [2025-09-16T19:16:58.239849] [WARN ] [MAIN ] LYG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:58.313270] [INFO ] [MAIN ] LYV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:58.328131] [INFO ] [MAIN ] LYG: Wiping existing primary data before full refresh. [2025-09-16T19:16:58.331946] [INFO ] [LOCAL ] LYG: Starting deletion of primary data. [2025-09-16T19:16:58.338460] [DEBUG] [LOCAL ] LYG: Deleted primary data file. [2025-09-16T19:16:58.359386] [INFO ] [MAIN ] LYG: Starting full primary refresh. [2025-09-16T19:16:58.490606] [DEBUG] [VALIDATOR ] LXU: All expected trading days are present. [2025-09-16T19:16:58.507808] [DEBUG] [VALIDATOR ] LXU: Primary data validation passed. [2025-09-16T19:16:58.520202] [INFO ] [MAIN ] LXU: Fetched primary data is valid. [2025-09-16T19:16:58.539891] [DEBUG] [LOCAL ] LXU: Saving primary data. [2025-09-16T19:16:58.626806] [DEBUG] [LOCAL ] LXU: Trimmed 34714 records to 23663 (keeping 150 days of data) [2025-09-16T19:16:58.635784] [DEBUG] [LOCAL ] LYV: Loaded 37576 existing records from primary storage. [2025-09-16T19:16:58.663426] [INFO ] [MAIN ] LYV: Loaded 37576 existing records. [2025-09-16T19:16:59.103320] [INFO ] [REMOTE ] LYG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:59.103873] [WARN ] [MAIN ] LYV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:59.125158] [INFO ] [MAIN ] LYV: Wiping existing primary data before full refresh. [2025-09-16T19:16:59.120134] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-02. [2025-09-16T19:16:59.126232] [INFO ] [LOCAL ] LYV: Starting deletion of primary data. [2025-09-16T19:16:59.132143] [DEBUG] [LOCAL ] LXU: Primary data saved (23663 records). [2025-09-16T19:16:59.132191] [DEBUG] [LOCAL ] LYV: Deleted primary data file. [2025-09-16T19:16:59.132489] [INFO ] [MAIN ] LXU: Full history saved. [2025-09-16T19:16:59.133204] [INFO ] [MAIN ] Update Progress: 1292/2369 symbols completed. LXU took 18.6s. Estimated remaining: 1h 54m 9s. Rate: 9.4 symbols/min [2025-09-16T19:16:59.133271] [INFO ] [MAIN ] LYV: Starting full primary refresh. [2025-09-16T19:16:59.472775] [INFO ] [MAIN ] LZB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:16:59.684784] [INFO ] [REMOTE ] LYV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:16:59.685162] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-02. [2025-09-16T19:16:59.695480] [DEBUG] [LOCAL ] LZB: Loaded 26036 existing records from primary storage. [2025-09-16T19:16:59.695756] [INFO ] [MAIN ] LZB: Loaded 26036 existing records. [2025-09-16T19:16:59.845412] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-05 (2.64 seconds). [2025-09-16T19:16:59.922360] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-06. [2025-09-16T19:16:59.922485] [WARN ] [MAIN ] LZB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:16:59.923473] [INFO ] [MAIN ] LZB: Wiping existing primary data before full refresh. [2025-09-16T19:16:59.923530] [INFO ] [LOCAL ] LZB: Starting deletion of primary data. [2025-09-16T19:16:59.924271] [DEBUG] [LOCAL ] LZB: Deleted primary data file. [2025-09-16T19:16:59.924322] [INFO ] [MAIN ] LZB: Starting full primary refresh. [2025-09-16T19:17:00.274046] [INFO ] [REMOTE ] LZB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:00.374848] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-02. [2025-09-16T19:17:00.965238] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-09-16T19:17:01.174952] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-03. [2025-09-16T19:17:01.440910] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-09-16T19:17:01.478306] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-09-16T19:17:01.974912] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-03. [2025-09-16T19:17:02.144197] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-06 (2.22 seconds). [2025-09-16T19:17:02.629354] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-09-16T19:17:02.774888] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-04. [2025-09-16T19:17:03.574928] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-07. [2025-09-16T19:17:03.931425] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-09-16T19:17:04.374895] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-03. [2025-09-16T19:17:04.667291] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-09-16T19:17:05.174900] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-05. [2025-09-16T19:17:05.445799] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-09-16T19:17:05.591803] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-09-16T19:17:05.974909] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-08. [2025-09-16T19:17:06.763521] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T19:17:06.775083] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-04. [2025-09-16T19:17:07.574867] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-06. [2025-09-16T19:17:08.089186] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-09-16T19:17:08.331018] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T19:17:08.374917] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-09. [2025-09-16T19:17:09.174968] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-05. [2025-09-16T19:17:09.288470] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T19:17:09.932435] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T19:17:09.940913] [INFO ] [REMOTE ] LYB: Full fetch complete (58194 records). [2025-09-16T19:17:09.941123] [DEBUG] [VALIDATOR ] LYB: Validating primary data. [2025-09-16T19:17:09.988251] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-07. [2025-09-16T19:17:10.490430] [DEBUG] [VALIDATOR ] LYB: All expected trading days are present. [2025-09-16T19:17:10.490540] [DEBUG] [VALIDATOR ] LYB: Primary data validation passed. [2025-09-16T19:17:10.491426] [INFO ] [MAIN ] LYB: Fetched primary data is valid. [2025-09-16T19:17:10.491511] [DEBUG] [LOCAL ] LYB: Saving primary data. [2025-09-16T19:17:10.492966] [DEBUG] [LOCAL ] LYB: Trimmed 58194 records to 39975 (keeping 150 days of data) [2025-09-16T19:17:10.752037] [DEBUG] [LOCAL ] LYB: Primary data saved (39975 records). [2025-09-16T19:17:10.752298] [INFO ] [MAIN ] LYB: Full history saved. [2025-09-16T19:17:10.753224] [INFO ] [MAIN ] Update Progress: 1293/2369 symbols completed. LYB took 27.5s. Estimated remaining: 1h 54m 7s. Rate: 9.4 symbols/min [2025-09-16T19:17:10.786169] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-04. [2025-09-16T19:17:10.965668] [INFO ] [MAIN ] LZM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:17:10.967505] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T19:17:10.989738] [DEBUG] [LOCAL ] LZM: Loaded 8818 existing records from primary storage. [2025-09-16T19:17:10.989887] [INFO ] [MAIN ] LZM: Loaded 8818 existing records. [2025-09-16T19:17:11.175303] [WARN ] [MAIN ] LZM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:17:11.175570] [INFO ] [MAIN ] LZM: Wiping existing primary data before full refresh. [2025-09-16T19:17:11.175618] [INFO ] [LOCAL ] LZM: Starting deletion of primary data. [2025-09-16T19:17:11.176046] [DEBUG] [LOCAL ] LZM: Deleted primary data file. [2025-09-16T19:17:11.176096] [INFO ] [MAIN ] LZM: Starting full primary refresh. [2025-09-16T19:17:11.547436] [INFO ] [REMOTE ] LZM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:11.574891] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-02. [2025-09-16T19:17:11.608297] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-09-16T19:17:12.099779] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-09-16T19:17:12.300983] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T19:17:12.374934] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-05. [2025-09-16T19:17:13.174983] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-08. [2025-09-16T19:17:13.386329] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-09-16T19:17:13.974920] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-03. [2025-09-16T19:17:14.774893] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-06. [2025-09-16T19:17:14.883532] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-09-16T19:17:14.926835] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T19:17:15.574955] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-04. [2025-09-16T19:17:16.374913] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-09. [2025-09-16T19:17:16.542072] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-09-16T19:17:16.591691] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-09-16T19:17:17.174908] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-06. [2025-09-16T19:17:17.827128] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-09-16T19:17:17.833429] [INFO ] [REMOTE ] LYG: Full fetch complete (57507 records). [2025-09-16T19:17:17.833591] [DEBUG] [VALIDATOR ] LYG: Validating primary data. [2025-09-16T19:17:17.979992] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-07. [2025-09-16T19:17:18.233810] [DEBUG] [VALIDATOR ] LYG: All expected trading days are present. [2025-09-16T19:17:18.233924] [DEBUG] [VALIDATOR ] LYG: Primary data validation passed. [2025-09-16T19:17:18.234976] [INFO ] [MAIN ] LYG: Fetched primary data is valid. [2025-09-16T19:17:18.235054] [DEBUG] [LOCAL ] LYG: Saving primary data. [2025-09-16T19:17:18.236489] [DEBUG] [LOCAL ] LYG: Trimmed 57507 records to 39363 (keeping 150 days of data) [2025-09-16T19:17:18.484172] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T19:17:18.518301] [DEBUG] [LOCAL ] LYG: Primary data saved (39363 records). [2025-09-16T19:17:18.518492] [INFO ] [MAIN ] LYG: Full history saved. [2025-09-16T19:17:18.518635] [INFO ] [MAIN ] Update Progress: 1294/2369 symbols completed. LYG took 21.2s. Estimated remaining: 1h 54m 2s. Rate: 9.4 symbols/min [2025-09-16T19:17:18.715219] [INFO ] [MAIN ] M: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:17:18.754868] [DEBUG] [LOCAL ] M: Loaded 39519 existing records from primary storage. [2025-09-16T19:17:18.755026] [INFO ] [MAIN ] M: Loaded 39519 existing records. [2025-09-16T19:17:18.785517] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-07. [2025-09-16T19:17:18.939825] [WARN ] [MAIN ] M: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:17:18.940098] [INFO ] [MAIN ] M: Wiping existing primary data before full refresh. [2025-09-16T19:17:18.940148] [INFO ] [LOCAL ] M: Starting deletion of primary data. [2025-09-16T19:17:18.941069] [DEBUG] [LOCAL ] M: Deleted primary data file. [2025-09-16T19:17:18.941127] [INFO ] [MAIN ] M: Starting full primary refresh. [2025-09-16T19:17:19.313430] [INFO ] [REMOTE ] M: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:19.534847] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-09-16T19:17:19.574926] [INFO ] [REMOTE ] M: Fetching primary data for 2025-02. [2025-09-16T19:17:20.222561] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-09-16T19:17:20.374906] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-05. [2025-09-16T19:17:21.174995] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-08. [2025-09-16T19:17:21.354498] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-09-16T19:17:21.357361] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-09-16T19:17:21.974973] [INFO ] [REMOTE ] M: Fetching primary data for 2025-03. [2025-09-16T19:17:22.257303] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T19:17:22.774932] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-09. [2025-09-16T19:17:23.574960] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-06. [2025-09-16T19:17:23.915944] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-09-16T19:17:24.049891] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-09-16T19:17:24.291527] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T19:17:24.296396] [INFO ] [REMOTE ] LZB: Full fetch complete (38846 records). [2025-09-16T19:17:24.296496] [DEBUG] [VALIDATOR ] LZB: Validating primary data. [2025-09-16T19:17:24.379990] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-07. [2025-09-16T19:17:24.678546] [DEBUG] [VALIDATOR ] LZB: All expected trading days are present. [2025-09-16T19:17:24.678633] [DEBUG] [VALIDATOR ] LZB: Primary data validation passed. [2025-09-16T19:17:24.679316] [INFO ] [MAIN ] LZB: Fetched primary data is valid. [2025-09-16T19:17:24.679358] [DEBUG] [LOCAL ] LZB: Saving primary data. [2025-09-16T19:17:24.680205] [DEBUG] [LOCAL ] LZB: Trimmed 38846 records to 26279 (keeping 150 days of data) [2025-09-16T19:17:24.841920] [DEBUG] [LOCAL ] LZB: Primary data saved (26279 records). [2025-09-16T19:17:24.842058] [INFO ] [MAIN ] LZB: Full history saved. [2025-09-16T19:17:24.842162] [INFO ] [MAIN ] Update Progress: 1295/2369 symbols completed. LZB took 25.7s. Estimated remaining: 1h 53m 55s. Rate: 9.4 symbols/min [2025-09-16T19:17:25.019373] [INFO ] [MAIN ] MA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:17:25.058393] [DEBUG] [LOCAL ] MA: Loaded 38662 existing records from primary storage. [2025-09-16T19:17:25.058530] [INFO ] [MAIN ] MA: Loaded 38662 existing records. [2025-09-16T19:17:25.179996] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-08. [2025-09-16T19:17:25.239726] [WARN ] [MAIN ] MA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:17:25.239958] [INFO ] [MAIN ] MA: Wiping existing primary data before full refresh. [2025-09-16T19:17:25.240007] [INFO ] [LOCAL ] MA: Starting deletion of primary data. [2025-09-16T19:17:25.240983] [DEBUG] [LOCAL ] MA: Deleted primary data file. [2025-09-16T19:17:25.241043] [INFO ] [MAIN ] MA: Starting full primary refresh. [2025-09-16T19:17:25.472688] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-09-16T19:17:25.608456] [INFO ] [REMOTE ] MA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:25.974897] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-08. [2025-09-16T19:17:26.774891] [INFO ] [REMOTE ] M: Fetching primary data for 2025-04. [2025-09-16T19:17:26.821779] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-09-16T19:17:27.003245] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-09-16T19:17:27.574936] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-09. [2025-09-16T19:17:28.241233] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-09-16T19:17:28.243951] [INFO ] [REMOTE ] LZM: Full fetch complete (14141 records). [2025-09-16T19:17:28.244082] [DEBUG] [VALIDATOR ] LZM: Validating primary data. [2025-09-16T19:17:28.391456] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-09. [2025-09-16T19:17:28.622233] [DEBUG] [VALIDATOR ] LZM: All expected trading days are present. [2025-09-16T19:17:28.622376] [DEBUG] [VALIDATOR ] LZM: Primary data validation passed. [2025-09-16T19:17:28.622797] [INFO ] [MAIN ] LZM: Fetched primary data is valid. [2025-09-16T19:17:28.622854] [DEBUG] [LOCAL ] LZM: Saving primary data. [2025-09-16T19:17:28.623360] [DEBUG] [LOCAL ] LZM: Trimmed 14141 records to 8881 (keeping 150 days of data) [2025-09-16T19:17:28.674477] [DEBUG] [LOCAL ] LZM: Primary data saved (8881 records). [2025-09-16T19:17:28.674673] [INFO ] [MAIN ] LZM: Full history saved. [2025-09-16T19:17:28.674811] [INFO ] [MAIN ] Update Progress: 1296/2369 symbols completed. LZM took 17.9s. Estimated remaining: 1h 53m 47s. Rate: 9.4 symbols/min [2025-09-16T19:17:28.913521] [INFO ] [MAIN ] MAA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:17:28.916014] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-09-16T19:17:28.961043] [DEBUG] [LOCAL ] MAA: Loaded 31623 existing records from primary storage. [2025-09-16T19:17:28.961182] [INFO ] [MAIN ] MAA: Loaded 31623 existing records. [2025-09-16T19:17:29.142036] [WARN ] [MAIN ] MAA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:17:29.142266] [INFO ] [MAIN ] MAA: Wiping existing primary data before full refresh. [2025-09-16T19:17:29.142314] [INFO ] [LOCAL ] MAA: Starting deletion of primary data. [2025-09-16T19:17:29.143704] [DEBUG] [LOCAL ] MAA: Deleted primary data file. [2025-09-16T19:17:29.143757] [INFO ] [MAIN ] MAA: Starting full primary refresh. [2025-09-16T19:17:29.180086] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-02. [2025-09-16T19:17:29.510057] [INFO ] [REMOTE ] MAA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:29.974916] [INFO ] [REMOTE ] M: Fetching primary data for 2025-05. [2025-09-16T19:17:30.096281] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T19:17:30.103917] [INFO ] [REMOTE ] LYV: Full fetch complete (55901 records). [2025-09-16T19:17:30.104048] [DEBUG] [VALIDATOR ] LYV: Validating primary data. [2025-09-16T19:17:30.490132] [DEBUG] [VALIDATOR ] LYV: All expected trading days are present. [2025-09-16T19:17:30.490220] [DEBUG] [VALIDATOR ] LYV: Primary data validation passed. [2025-09-16T19:17:30.491121] [INFO ] [MAIN ] LYV: Fetched primary data is valid. [2025-09-16T19:17:30.491168] [DEBUG] [LOCAL ] LYV: Saving primary data. [2025-09-16T19:17:30.492283] [DEBUG] [LOCAL ] LYV: Trimmed 55901 records to 37955 (keeping 150 days of data) [2025-09-16T19:17:30.725069] [DEBUG] [LOCAL ] LYV: Primary data saved (37955 records). [2025-09-16T19:17:30.725218] [INFO ] [MAIN ] LYV: Full history saved. [2025-09-16T19:17:30.725305] [INFO ] [MAIN ] Update Progress: 1297/2369 symbols completed. LYV took 32.9s. Estimated remaining: 1h 53m 37s. Rate: 9.4 symbols/min [2025-09-16T19:17:30.789399] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-02. [2025-09-16T19:17:30.897360] [INFO ] [MAIN ] MAC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:17:30.933533] [DEBUG] [LOCAL ] MAC: Loaded 36424 existing records from primary storage. [2025-09-16T19:17:30.933694] [INFO ] [MAIN ] MAC: Loaded 36424 existing records. [2025-09-16T19:17:31.112522] [WARN ] [MAIN ] MAC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:17:31.112885] [INFO ] [MAIN ] MAC: Wiping existing primary data before full refresh. [2025-09-16T19:17:31.113017] [INFO ] [LOCAL ] MAC: Starting deletion of primary data. [2025-09-16T19:17:31.113888] [DEBUG] [LOCAL ] MAC: Deleted primary data file. [2025-09-16T19:17:31.113939] [INFO ] [MAIN ] MAC: Starting full primary refresh. [2025-09-16T19:17:31.252628] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-09-16T19:17:31.537081] [INFO ] [REMOTE ] MAC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:31.570716] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-09-16T19:17:31.581120] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-03. [2025-09-16T19:17:32.374906] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-02. [2025-09-16T19:17:32.680810] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-09-16T19:17:33.174933] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-03. [2025-09-16T19:17:33.564072] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-09-16T19:17:33.682582] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T19:17:33.974901] [INFO ] [REMOTE ] M: Fetching primary data for 2025-06. [2025-09-16T19:17:34.785550] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-03. [2025-09-16T19:17:34.788300] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T19:17:35.574910] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-04. [2025-09-16T19:17:35.838530] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-09-16T19:17:36.374909] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-04. [2025-09-16T19:17:36.520995] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-09-16T19:17:37.174884] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-04. [2025-09-16T19:17:37.654864] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-09-16T19:17:37.974905] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-05. [2025-09-16T19:17:38.161618] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-09-16T19:17:38.774873] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-05. [2025-09-16T19:17:38.829087] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T19:17:39.574872] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-05. [2025-09-16T19:17:40.028504] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-05 (2.05 seconds). [2025-09-16T19:17:40.302350] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T19:17:40.374905] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-06. [2025-09-16T19:17:41.174884] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-06. [2025-09-16T19:17:41.278503] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T19:17:41.974948] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-06. [2025-09-16T19:17:42.414910] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-06 (2.04 seconds). [2025-09-16T19:17:42.537762] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-09-16T19:17:42.774891] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-07. [2025-09-16T19:17:43.308798] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-09-16T19:17:43.574947] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-07. [2025-09-16T19:17:44.374905] [INFO ] [REMOTE ] M: Fetching primary data for 2025-07. [2025-09-16T19:17:44.621502] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-09-16T19:17:45.161224] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T19:17:45.174847] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-08. [2025-09-16T19:17:45.974903] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-08. [2025-09-16T19:17:46.406794] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-09-16T19:17:46.713484] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T19:17:46.774887] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-09. [2025-09-16T19:17:47.574920] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-07. [2025-09-16T19:17:47.736107] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-09-16T19:17:48.304310] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-09-16T19:17:48.310094] [INFO ] [REMOTE ] MAA: Full fetch complete (46726 records). [2025-09-16T19:17:48.310259] [DEBUG] [VALIDATOR ] MAA: Validating primary data. [2025-09-16T19:17:48.382126] [INFO ] [REMOTE ] M: Fetching primary data for 2025-08. [2025-09-16T19:17:48.715175] [DEBUG] [VALIDATOR ] MAA: All expected trading days are present. [2025-09-16T19:17:48.715254] [DEBUG] [VALIDATOR ] MAA: Primary data validation passed. [2025-09-16T19:17:48.716007] [INFO ] [MAIN ] MAA: Fetched primary data is valid. [2025-09-16T19:17:48.716062] [DEBUG] [LOCAL ] MAA: Saving primary data. [2025-09-16T19:17:48.717054] [DEBUG] [LOCAL ] MAA: Trimmed 46726 records to 31954 (keeping 150 days of data) [2025-09-16T19:17:48.917969] [DEBUG] [LOCAL ] MAA: Primary data saved (31954 records). [2025-09-16T19:17:48.918106] [INFO ] [MAIN ] MAA: Full history saved. [2025-09-16T19:17:48.918205] [INFO ] [MAIN ] Update Progress: 1298/2369 symbols completed. MAA took 20.2s. Estimated remaining: 1h 53m 40s. Rate: 9.4 symbols/min [2025-09-16T19:17:49.095830] [INFO ] [MAIN ] MAGN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:17:49.123260] [DEBUG] [LOCAL ] MAGN: Loaded 26043 existing records from primary storage. [2025-09-16T19:17:49.123405] [INFO ] [MAIN ] MAGN: Loaded 26043 existing records. [2025-09-16T19:17:49.189104] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-09. [2025-09-16T19:17:49.306563] [WARN ] [MAIN ] MAGN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:17:49.306827] [INFO ] [MAIN ] MAGN: Wiping existing primary data before full refresh. [2025-09-16T19:17:49.306882] [INFO ] [LOCAL ] MAGN: Starting deletion of primary data. [2025-09-16T19:17:49.307451] [DEBUG] [LOCAL ] MAGN: Deleted primary data file. [2025-09-16T19:17:49.307513] [INFO ] [MAIN ] MAGN: Starting full primary refresh. [2025-09-16T19:17:49.667690] [INFO ] [REMOTE ] MAGN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:49.695983] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-09-16T19:17:49.974870] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-08. [2025-09-16T19:17:50.296674] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-09-16T19:17:50.774886] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-02. [2025-09-16T19:17:50.918044] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-09-16T19:17:50.926956] [INFO ] [REMOTE ] MA: Full fetch complete (57188 records). [2025-09-16T19:17:50.927060] [DEBUG] [VALIDATOR ] MA: Validating primary data. [2025-09-16T19:17:51.335135] [DEBUG] [VALIDATOR ] MA: All expected trading days are present. [2025-09-16T19:17:51.335258] [DEBUG] [VALIDATOR ] MA: Primary data validation passed. [2025-09-16T19:17:51.336202] [INFO ] [MAIN ] MA: Fetched primary data is valid. [2025-09-16T19:17:51.336271] [DEBUG] [LOCAL ] MA: Saving primary data. [2025-09-16T19:17:51.338024] [DEBUG] [LOCAL ] MA: Trimmed 57188 records to 39040 (keeping 150 days of data) [2025-09-16T19:17:51.476474] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-09-16T19:17:51.598775] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-09. [2025-09-16T19:17:51.632126] [DEBUG] [LOCAL ] MA: Primary data saved (39040 records). [2025-09-16T19:17:51.632237] [INFO ] [MAIN ] MA: Full history saved. [2025-09-16T19:17:51.633023] [INFO ] [MAIN ] Update Progress: 1299/2369 symbols completed. MA took 26.8s. Estimated remaining: 1h 53m 31s. Rate: 9.4 symbols/min [2025-09-16T19:17:51.809223] [INFO ] [MAIN ] MAIN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:17:51.846892] [DEBUG] [LOCAL ] MAIN: Loaded 30333 existing records from primary storage. [2025-09-16T19:17:51.847058] [INFO ] [MAIN ] MAIN: Loaded 30333 existing records. [2025-09-16T19:17:52.020781] [WARN ] [MAIN ] MAIN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:17:52.021026] [INFO ] [MAIN ] MAIN: Wiping existing primary data before full refresh. [2025-09-16T19:17:52.021073] [INFO ] [LOCAL ] MAIN: Starting deletion of primary data. [2025-09-16T19:17:52.021769] [DEBUG] [LOCAL ] MAIN: Deleted primary data file. [2025-09-16T19:17:52.021819] [INFO ] [MAIN ] MAIN: Starting full primary refresh. [2025-09-16T19:17:52.392507] [INFO ] [REMOTE ] M: Fetching primary data for 2025-09. [2025-09-16T19:17:52.399078] [INFO ] [REMOTE ] MAIN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:52.400765] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T19:17:52.953118] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T19:17:52.963183] [INFO ] [REMOTE ] MAC: Full fetch complete (53971 records). [2025-09-16T19:17:52.963365] [DEBUG] [VALIDATOR ] MAC: Validating primary data. [2025-09-16T19:17:53.182249] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-02. [2025-09-16T19:17:53.342481] [DEBUG] [VALIDATOR ] MAC: All expected trading days are present. [2025-09-16T19:17:53.342573] [DEBUG] [VALIDATOR ] MAC: Primary data validation passed. [2025-09-16T19:17:53.343396] [INFO ] [MAIN ] MAC: Fetched primary data is valid. [2025-09-16T19:17:53.343444] [DEBUG] [LOCAL ] MAC: Saving primary data. [2025-09-16T19:17:53.344624] [DEBUG] [LOCAL ] MAC: Trimmed 53971 records to 36799 (keeping 150 days of data) [2025-09-16T19:17:53.577078] [DEBUG] [LOCAL ] MAC: Primary data saved (36799 records). [2025-09-16T19:17:53.577218] [INFO ] [MAIN ] MAC: Full history saved. [2025-09-16T19:17:53.577301] [INFO ] [MAIN ] Update Progress: 1300/2369 symbols completed. MAC took 22.9s. Estimated remaining: 1h 53m 21s. Rate: 9.4 symbols/min [2025-09-16T19:17:53.751892] [INFO ] [MAIN ] MAN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:17:53.784665] [DEBUG] [LOCAL ] MAN: Loaded 32327 existing records from primary storage. [2025-09-16T19:17:53.784806] [INFO ] [MAIN ] MAN: Loaded 32327 existing records. [2025-09-16T19:17:53.960785] [WARN ] [MAIN ] MAN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:17:53.961008] [INFO ] [MAIN ] MAN: Wiping existing primary data before full refresh. [2025-09-16T19:17:53.961088] [INFO ] [LOCAL ] MAN: Starting deletion of primary data. [2025-09-16T19:17:53.961759] [DEBUG] [LOCAL ] MAN: Deleted primary data file. [2025-09-16T19:17:53.961808] [INFO ] [MAIN ] MAN: Starting full primary refresh. [2025-09-16T19:17:53.993111] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-03. [2025-09-16T19:17:54.341870] [INFO ] [REMOTE ] MAN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:54.360025] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-09-16T19:17:54.367419] [INFO ] [REMOTE ] M: Full fetch complete (58188 records). [2025-09-16T19:17:54.367529] [DEBUG] [VALIDATOR ] M: Validating primary data. [2025-09-16T19:17:54.756993] [DEBUG] [VALIDATOR ] M: All expected trading days are present. [2025-09-16T19:17:54.757096] [DEBUG] [VALIDATOR ] M: Primary data validation passed. [2025-09-16T19:17:54.758087] [INFO ] [MAIN ] M: Fetched primary data is valid. [2025-09-16T19:17:54.758160] [DEBUG] [LOCAL ] M: Saving primary data. [2025-09-16T19:17:54.759768] [DEBUG] [LOCAL ] M: Trimmed 58188 records to 39907 (keeping 150 days of data) [2025-09-16T19:17:54.819905] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-02. [2025-09-16T19:17:54.923996] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T19:17:55.056202] [DEBUG] [LOCAL ] M: Primary data saved (39907 records). [2025-09-16T19:17:55.056362] [INFO ] [MAIN ] M: Full history saved. [2025-09-16T19:17:55.056478] [INFO ] [MAIN ] Update Progress: 1301/2369 symbols completed. M took 36.5s. Estimated remaining: 1h 53m 11s. Rate: 9.4 symbols/min [2025-09-16T19:17:55.244509] [INFO ] [MAIN ] MANU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:17:55.264167] [DEBUG] [LOCAL ] MANU: Loaded 19151 existing records from primary storage. [2025-09-16T19:17:55.264286] [INFO ] [MAIN ] MANU: Loaded 19151 existing records. [2025-09-16T19:17:55.444909] [WARN ] [MAIN ] MANU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:17:55.445407] [INFO ] [MAIN ] MANU: Wiping existing primary data before full refresh. [2025-09-16T19:17:55.445488] [INFO ] [LOCAL ] MANU: Starting deletion of primary data. [2025-09-16T19:17:55.446120] [DEBUG] [LOCAL ] MANU: Deleted primary data file. [2025-09-16T19:17:55.446173] [INFO ] [MAIN ] MANU: Starting full primary refresh. [2025-09-16T19:17:55.577211] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-03. [2025-09-16T19:17:55.583733] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-09-16T19:17:55.846086] [INFO ] [REMOTE ] MANU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:17:55.871554] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-09-16T19:17:56.374918] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-02. [2025-09-16T19:17:57.174982] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-03. [2025-09-16T19:17:57.390374] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T19:17:57.545782] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-09-16T19:17:57.974959] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-04. [2025-09-16T19:17:58.613004] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T19:17:58.774909] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-04. [2025-09-16T19:17:59.574939] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-04. [2025-09-16T19:17:59.808314] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-09-16T19:18:00.121802] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-09-16T19:18:00.374930] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-05. [2025-09-16T19:18:00.665467] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-09-16T19:18:01.174895] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-03. [2025-09-16T19:18:01.974980] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-05. [2025-09-16T19:18:02.078474] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T19:18:02.285716] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-09-16T19:18:02.774896] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-04. [2025-09-16T19:18:03.568801] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T19:18:03.576525] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-06. [2025-09-16T19:18:03.688820] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-09-16T19:18:04.374946] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-05. [2025-09-16T19:18:05.016192] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-09-16T19:18:05.174939] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-05. [2025-09-16T19:18:05.534120] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-09-16T19:18:05.974952] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-07. [2025-09-16T19:18:06.774890] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-06. [2025-09-16T19:18:07.059828] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-09-16T19:18:07.433117] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T19:18:07.574918] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-08. [2025-09-16T19:18:08.169245] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-09-16T19:18:08.374930] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-06. [2025-09-16T19:18:09.086213] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T19:18:09.174889] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-09. [2025-09-16T19:18:09.740916] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-09-16T19:18:09.974937] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-06. [2025-09-16T19:18:10.295121] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-09-16T19:18:10.300316] [INFO ] [REMOTE ] MAN: Full fetch complete (47027 records). [2025-09-16T19:18:10.300396] [DEBUG] [VALIDATOR ] MAN: Validating primary data. [2025-09-16T19:18:10.671821] [DEBUG] [VALIDATOR ] MAN: All expected trading days are present. [2025-09-16T19:18:10.671915] [DEBUG] [VALIDATOR ] MAN: Primary data validation passed. [2025-09-16T19:18:10.672684] [INFO ] [MAIN ] MAN: Fetched primary data is valid. [2025-09-16T19:18:10.672733] [DEBUG] [LOCAL ] MAN: Saving primary data. [2025-09-16T19:18:10.673922] [DEBUG] [LOCAL ] MAN: Trimmed 47027 records to 32640 (keeping 150 days of data) [2025-09-16T19:18:10.776213] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-07. [2025-09-16T19:18:10.872622] [DEBUG] [LOCAL ] MAN: Primary data saved (32640 records). [2025-09-16T19:18:10.872779] [INFO ] [MAIN ] MAN: Full history saved. [2025-09-16T19:18:10.872874] [INFO ] [MAIN ] Update Progress: 1302/2369 symbols completed. MAN took 17.3s. Estimated remaining: 1h 53m 12s. Rate: 9.4 symbols/min [2025-09-16T19:18:11.041002] [INFO ] [MAIN ] MAS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:18:11.075659] [DEBUG] [LOCAL ] MAS: Loaded 38615 existing records from primary storage. [2025-09-16T19:18:11.075804] [INFO ] [MAIN ] MAS: Loaded 38615 existing records. [2025-09-16T19:18:11.252662] [WARN ] [MAIN ] MAS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:18:11.252954] [INFO ] [MAIN ] MAS: Wiping existing primary data before full refresh. [2025-09-16T19:18:11.253002] [INFO ] [LOCAL ] MAS: Starting deletion of primary data. [2025-09-16T19:18:11.253837] [DEBUG] [LOCAL ] MAS: Deleted primary data file. [2025-09-16T19:18:11.253887] [INFO ] [MAIN ] MAS: Starting full primary refresh. [2025-09-16T19:18:11.575300] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-07. [2025-09-16T19:18:11.604177] [INFO ] [REMOTE ] MAS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:18:11.685430] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T19:18:12.066955] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-09-16T19:18:12.374908] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-02. [2025-09-16T19:18:13.035422] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-09-16T19:18:13.174935] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-08. [2025-09-16T19:18:13.528846] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-09-16T19:18:13.974887] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-08. [2025-09-16T19:18:14.284511] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-09-16T19:18:14.774890] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-03. [2025-09-16T19:18:15.032078] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-09-16T19:18:15.574941] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-07. [2025-09-16T19:18:16.206702] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-09-16T19:18:16.374919] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-04. [2025-09-16T19:18:17.174900] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-09. [2025-09-16T19:18:17.279934] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-09-16T19:18:17.974961] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-08. [2025-09-16T19:18:18.152227] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-09-16T19:18:18.312596] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T19:18:18.316614] [INFO ] [REMOTE ] MANU: Full fetch complete (29456 records). [2025-09-16T19:18:18.316762] [DEBUG] [VALIDATOR ] MANU: Validating primary data. [2025-09-16T19:18:18.695204] [DEBUG] [VALIDATOR ] MANU: All expected trading days are present. [2025-09-16T19:18:18.695315] [DEBUG] [VALIDATOR ] MANU: Primary data validation passed. [2025-09-16T19:18:18.695941] [INFO ] [MAIN ] MANU: Fetched primary data is valid. [2025-09-16T19:18:18.695988] [DEBUG] [LOCAL ] MANU: Saving primary data. [2025-09-16T19:18:18.696847] [DEBUG] [LOCAL ] MANU: Trimmed 29456 records to 19379 (keeping 150 days of data) [2025-09-16T19:18:18.780392] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-09. [2025-09-16T19:18:18.815558] [DEBUG] [LOCAL ] MANU: Primary data saved (19379 records). [2025-09-16T19:18:18.815774] [INFO ] [MAIN ] MANU: Full history saved. [2025-09-16T19:18:18.815879] [INFO ] [MAIN ] Update Progress: 1303/2369 symbols completed. MANU took 23.8s. Estimated remaining: 1h 53m 7s. Rate: 9.4 symbols/min [2025-09-16T19:18:19.034330] [INFO ] [MAIN ] MATV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:18:19.061997] [DEBUG] [LOCAL ] MATV: Loaded 26656 existing records from primary storage. [2025-09-16T19:18:19.062157] [INFO ] [MAIN ] MATV: Loaded 26656 existing records. [2025-09-16T19:18:19.249932] [WARN ] [MAIN ] MATV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:18:19.250206] [INFO ] [MAIN ] MATV: Wiping existing primary data before full refresh. [2025-09-16T19:18:19.250253] [INFO ] [LOCAL ] MATV: Starting deletion of primary data. [2025-09-16T19:18:19.250943] [DEBUG] [LOCAL ] MATV: Deleted primary data file. [2025-09-16T19:18:19.250995] [INFO ] [MAIN ] MATV: Starting full primary refresh. [2025-09-16T19:18:19.577825] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-05. [2025-09-16T19:18:19.613174] [INFO ] [REMOTE ] MATV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:18:19.727174] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-09-16T19:18:20.070080] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T19:18:20.074247] [INFO ] [REMOTE ] MAGN: Full fetch complete (38316 records). [2025-09-16T19:18:20.074334] [DEBUG] [VALIDATOR ] MAGN: Validating primary data. [2025-09-16T19:18:20.380019] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-09. [2025-09-16T19:18:20.449963] [DEBUG] [VALIDATOR ] MAGN: All expected trading days are present. [2025-09-16T19:18:20.450039] [DEBUG] [VALIDATOR ] MAGN: Primary data validation passed. [2025-09-16T19:18:20.450789] [INFO ] [MAIN ] MAGN: Fetched primary data is valid. [2025-09-16T19:18:20.450835] [DEBUG] [LOCAL ] MAGN: Saving primary data. [2025-09-16T19:18:20.451733] [DEBUG] [LOCAL ] MAGN: Trimmed 38316 records to 26255 (keeping 150 days of data) [2025-09-16T19:18:20.604666] [DEBUG] [LOCAL ] MAGN: Primary data saved (26255 records). [2025-09-16T19:18:20.604799] [INFO ] [MAIN ] MAGN: Full history saved. [2025-09-16T19:18:20.604904] [INFO ] [MAIN ] Update Progress: 1304/2369 symbols completed. MAGN took 31.7s. Estimated remaining: 1h 52m 57s. Rate: 9.4 symbols/min [2025-09-16T19:18:20.774713] [INFO ] [MAIN ] MATX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:18:20.798824] [DEBUG] [LOCAL ] MATX: Loaded 23548 existing records from primary storage. [2025-09-16T19:18:20.798951] [INFO ] [MAIN ] MATX: Loaded 23548 existing records. [2025-09-16T19:18:20.976076] [WARN ] [MAIN ] MATX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:18:20.976318] [INFO ] [MAIN ] MATX: Wiping existing primary data before full refresh. [2025-09-16T19:18:20.976381] [INFO ] [LOCAL ] MATX: Starting deletion of primary data. [2025-09-16T19:18:20.977152] [DEBUG] [LOCAL ] MATX: Deleted primary data file. [2025-09-16T19:18:20.977213] [INFO ] [MAIN ] MATX: Starting full primary refresh. [2025-09-16T19:18:21.180017] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-02. [2025-09-16T19:18:21.345589] [INFO ] [REMOTE ] MATX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:18:21.399017] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-09-16T19:18:21.974946] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-06. [2025-09-16T19:18:22.013505] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-09-16T19:18:22.019109] [INFO ] [REMOTE ] MAIN: Full fetch complete (45061 records). [2025-09-16T19:18:22.019247] [DEBUG] [VALIDATOR ] MAIN: Validating primary data. [2025-09-16T19:18:22.425410] [DEBUG] [VALIDATOR ] MAIN: All expected trading days are present. [2025-09-16T19:18:22.425549] [DEBUG] [VALIDATOR ] MAIN: Primary data validation passed. [2025-09-16T19:18:22.426474] [INFO ] [MAIN ] MAIN: Fetched primary data is valid. [2025-09-16T19:18:22.426571] [DEBUG] [LOCAL ] MAIN: Saving primary data. [2025-09-16T19:18:22.450990] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-09-16T19:18:22.449852] [DEBUG] [LOCAL ] MAIN: Trimmed 45061 records to 30668 (keeping 150 days of data) [2025-09-16T19:18:22.647022] [DEBUG] [LOCAL ] MAIN: Primary data saved (30668 records). [2025-09-16T19:18:22.647156] [INFO ] [MAIN ] MAIN: Full history saved. [2025-09-16T19:18:22.648378] [INFO ] [MAIN ] Update Progress: 1305/2369 symbols completed. MAIN took 31.0s. Estimated remaining: 1h 52m 47s. Rate: 9.4 symbols/min [2025-09-16T19:18:22.787994] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-03. [2025-09-16T19:18:22.816974] [INFO ] [MAIN ] MAX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:18:22.843394] [DEBUG] [LOCAL ] MAX: Loaded 22023 existing records from primary storage. [2025-09-16T19:18:22.843539] [INFO ] [MAIN ] MAX: Loaded 22023 existing records. [2025-09-16T19:18:23.029795] [WARN ] [MAIN ] MAX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:18:23.030086] [INFO ] [MAIN ] MAX: Wiping existing primary data before full refresh. [2025-09-16T19:18:23.030145] [INFO ] [LOCAL ] MAX: Starting deletion of primary data. [2025-09-16T19:18:23.030768] [DEBUG] [LOCAL ] MAX: Deleted primary data file. [2025-09-16T19:18:23.030833] [INFO ] [MAIN ] MAX: Starting full primary refresh. [2025-09-16T19:18:23.397786] [INFO ] [REMOTE ] MAX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:18:23.574942] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-02. [2025-09-16T19:18:23.736387] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-09-16T19:18:24.290013] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-09-16T19:18:24.374902] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-04. [2025-09-16T19:18:24.992121] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-09-16T19:18:25.174926] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-07. [2025-09-16T19:18:25.856825] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T19:18:25.974901] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-05. [2025-09-16T19:18:26.774956] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-02. [2025-09-16T19:18:26.840839] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-09-16T19:18:26.939978] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-09-16T19:18:27.574914] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-06. [2025-09-16T19:18:27.740630] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-09-16T19:18:28.374908] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-03. [2025-09-16T19:18:29.049995] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T19:18:29.174867] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-07. [2025-09-16T19:18:29.545015] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-09-16T19:18:29.974910] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-03. [2025-09-16T19:18:30.589045] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-09-16T19:18:30.774900] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-08. [2025-09-16T19:18:30.978581] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-09-16T19:18:31.574903] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-08. [2025-09-16T19:18:32.306220] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-09-16T19:18:32.374882] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-09. [2025-09-16T19:18:33.061393] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-09-16T19:18:33.174906] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-04. [2025-09-16T19:18:33.974880] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-09. [2025-09-16T19:18:34.039245] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-09-16T19:18:34.045849] [INFO ] [REMOTE ] MAS: Full fetch complete (56776 records). [2025-09-16T19:18:34.045976] [DEBUG] [VALIDATOR ] MAS: Validating primary data. [2025-09-16T19:18:34.441085] [DEBUG] [VALIDATOR ] MAS: All expected trading days are present. [2025-09-16T19:18:34.441166] [DEBUG] [VALIDATOR ] MAS: Primary data validation passed. [2025-09-16T19:18:34.442354] [INFO ] [MAIN ] MAS: Fetched primary data is valid. [2025-09-16T19:18:34.442399] [DEBUG] [LOCAL ] MAS: Saving primary data. [2025-09-16T19:18:34.443550] [DEBUG] [LOCAL ] MAS: Trimmed 56776 records to 39000 (keeping 150 days of data) [2025-09-16T19:18:34.678333] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T19:18:34.713584] [DEBUG] [LOCAL ] MAS: Primary data saved (39000 records). [2025-09-16T19:18:34.713761] [INFO ] [MAIN ] MAS: Full history saved. [2025-09-16T19:18:34.713866] [INFO ] [MAIN ] Update Progress: 1306/2369 symbols completed. MAS took 23.8s. Estimated remaining: 1h 52m 45s. Rate: 9.4 symbols/min [2025-09-16T19:18:34.780006] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-05. [2025-09-16T19:18:34.887547] [INFO ] [MAIN ] MBC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:18:34.918199] [DEBUG] [LOCAL ] MBC: Loaded 31444 existing records from primary storage. [2025-09-16T19:18:34.918349] [INFO ] [MAIN ] MBC: Loaded 31444 existing records. [2025-09-16T19:18:35.097586] [WARN ] [MAIN ] MBC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:18:35.097848] [INFO ] [MAIN ] MBC: Wiping existing primary data before full refresh. [2025-09-16T19:18:35.097895] [INFO ] [LOCAL ] MBC: Starting deletion of primary data. [2025-09-16T19:18:35.098690] [DEBUG] [LOCAL ] MBC: Deleted primary data file. [2025-09-16T19:18:35.098754] [INFO ] [MAIN ] MBC: Starting full primary refresh. [2025-09-16T19:18:35.268078] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-09-16T19:18:35.369493] [INFO ] [REMOTE ] MATV: Full fetch complete (41301 records). [2025-09-16T19:18:35.386806] [DEBUG] [VALIDATOR ] MATV: Validating primary data. [2025-09-16T19:18:35.587424] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-04. [2025-09-16T19:18:35.592339] [INFO ] [REMOTE ] MBC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:18:35.875263] [DEBUG] [VALIDATOR ] MATV: All expected trading days are present. [2025-09-16T19:18:35.875368] [DEBUG] [VALIDATOR ] MATV: Primary data validation passed. [2025-09-16T19:18:35.876260] [INFO ] [MAIN ] MATV: Fetched primary data is valid. [2025-09-16T19:18:35.876321] [DEBUG] [LOCAL ] MATV: Saving primary data. [2025-09-16T19:18:35.877316] [DEBUG] [LOCAL ] MATV: Trimmed 41301 records to 26810 (keeping 150 days of data) [2025-09-16T19:18:36.039376] [DEBUG] [LOCAL ] MATV: Primary data saved (26810 records). [2025-09-16T19:18:36.039516] [INFO ] [MAIN ] MATV: Full history saved. [2025-09-16T19:18:36.039663] [INFO ] [MAIN ] Update Progress: 1307/2369 symbols completed. MATV took 17.2s. Estimated remaining: 1h 52m 35s. Rate: 9.4 symbols/min [2025-09-16T19:18:36.219412] [INFO ] [MAIN ] MBI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:18:36.236711] [DEBUG] [LOCAL ] MBI: Loaded 17482 existing records from primary storage. [2025-09-16T19:18:36.236838] [INFO ] [MAIN ] MBI: Loaded 17482 existing records. [2025-09-16T19:18:36.389108] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-02. [2025-09-16T19:18:36.428680] [WARN ] [MAIN ] MBI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:18:36.428985] [INFO ] [MAIN ] MBI: Wiping existing primary data before full refresh. [2025-09-16T19:18:36.429062] [INFO ] [LOCAL ] MBI: Starting deletion of primary data. [2025-09-16T19:18:36.437008] [DEBUG] [LOCAL ] MBI: Deleted primary data file. [2025-09-16T19:18:36.449850] [INFO ] [MAIN ] MBI: Starting full primary refresh. [2025-09-16T19:18:36.483047] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T19:18:36.844985] [INFO ] [REMOTE ] MBI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:18:36.928076] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-09-16T19:18:37.174895] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-05. [2025-09-16T19:18:37.605111] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-09-16T19:18:37.974896] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-02. [2025-09-16T19:18:38.481969] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-09-16T19:18:38.767875] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T19:18:38.774831] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-03. [2025-09-16T19:18:39.574880] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-06. [2025-09-16T19:18:39.950275] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-09-16T19:18:40.374897] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-04. [2025-09-16T19:18:40.621301] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-09-16T19:18:41.174917] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-07. [2025-09-16T19:18:41.628445] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-09-16T19:18:41.974869] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-03. [2025-09-16T19:18:42.417561] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-09-16T19:18:42.774923] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-05. [2025-09-16T19:18:43.173285] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-09-16T19:18:43.538635] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-09-16T19:18:43.574874] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-08. [2025-09-16T19:18:44.374925] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-06. [2025-09-16T19:18:44.936289] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-09-16T19:18:45.174889] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-09. [2025-09-16T19:18:45.220564] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-09-16T19:18:45.974950] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-07. [2025-09-16T19:18:46.414074] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-09-16T19:18:46.419072] [INFO ] [REMOTE ] MAX: Full fetch complete (33979 records). [2025-09-16T19:18:46.419529] [DEBUG] [VALIDATOR ] MAX: Validating primary data. [2025-09-16T19:18:46.778669] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-06. [2025-09-16T19:18:46.809393] [DEBUG] [VALIDATOR ] MAX: All expected trading days are present. [2025-09-16T19:18:46.809574] [DEBUG] [VALIDATOR ] MAX: Primary data validation passed. [2025-09-16T19:18:46.810315] [INFO ] [MAIN ] MAX: Fetched primary data is valid. [2025-09-16T19:18:46.810360] [DEBUG] [LOCAL ] MAX: Saving primary data. [2025-09-16T19:18:46.811136] [DEBUG] [LOCAL ] MAX: Trimmed 33979 records to 22245 (keeping 150 days of data) [2025-09-16T19:18:46.940191] [DEBUG] [LOCAL ] MAX: Primary data saved (22245 records). [2025-09-16T19:18:46.940322] [INFO ] [MAIN ] MAX: Full history saved. [2025-09-16T19:18:46.940433] [INFO ] [MAIN ] Update Progress: 1308/2369 symbols completed. MAX took 24.3s. Estimated remaining: 1h 52m 32s. Rate: 9.4 symbols/min [2025-09-16T19:18:47.112488] [INFO ] [MAIN ] MC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:18:47.125861] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T19:18:47.160112] [DEBUG] [LOCAL ] MC: Loaded 32268 existing records from primary storage. [2025-09-16T19:18:47.160261] [INFO ] [MAIN ] MC: Loaded 32268 existing records. [2025-09-16T19:18:47.342819] [WARN ] [MAIN ] MC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:18:47.343094] [INFO ] [MAIN ] MC: Wiping existing primary data before full refresh. [2025-09-16T19:18:47.343142] [INFO ] [LOCAL ] MC: Starting deletion of primary data. [2025-09-16T19:18:47.343798] [DEBUG] [LOCAL ] MC: Deleted primary data file. [2025-09-16T19:18:47.343848] [INFO ] [MAIN ] MC: Starting full primary refresh. [2025-09-16T19:18:47.584121] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-04. [2025-09-16T19:18:47.713507] [INFO ] [REMOTE ] MC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:18:47.906917] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-09-16T19:18:48.374916] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-07. [2025-09-16T19:18:48.786917] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-09-16T19:18:49.174914] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-05. [2025-09-16T19:18:49.724800] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-09-16T19:18:49.974926] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-08. [2025-09-16T19:18:50.680046] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-09-16T19:18:50.774912] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-06. [2025-09-16T19:18:51.139323] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-09-16T19:18:51.574912] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-02. [2025-09-16T19:18:52.254457] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T19:18:52.374978] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-08. [2025-09-16T19:18:53.179969] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-07. [2025-09-16T19:18:53.192286] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T19:18:53.544389] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-09-16T19:18:53.974976] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-09. [2025-09-16T19:18:54.573993] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-09-16T19:18:54.775007] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-09. [2025-09-16T19:18:55.183017] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-09-16T19:18:55.186468] [INFO ] [REMOTE ] MBI: Full fetch complete (27263 records). [2025-09-16T19:18:55.186550] [DEBUG] [VALIDATOR ] MBI: Validating primary data. [2025-09-16T19:18:55.551337] [DEBUG] [VALIDATOR ] MBI: All expected trading days are present. [2025-09-16T19:18:55.551417] [DEBUG] [VALIDATOR ] MBI: Primary data validation passed. [2025-09-16T19:18:55.551985] [INFO ] [MAIN ] MBI: Fetched primary data is valid. [2025-09-16T19:18:55.552029] [DEBUG] [LOCAL ] MBI: Saving primary data. [2025-09-16T19:18:55.552696] [DEBUG] [LOCAL ] MBI: Trimmed 27263 records to 17717 (keeping 150 days of data) [2025-09-16T19:18:55.593885] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-03. [2025-09-16T19:18:55.654522] [DEBUG] [LOCAL ] MBI: Primary data saved (17717 records). [2025-09-16T19:18:55.654664] [INFO ] [MAIN ] MBI: Full history saved. [2025-09-16T19:18:55.654750] [INFO ] [MAIN ] Update Progress: 1309/2369 symbols completed. MBI took 19.6s. Estimated remaining: 1h 52m 28s. Rate: 9.4 symbols/min [2025-09-16T19:18:55.825547] [INFO ] [MAIN ] MCB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:18:55.838085] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-09-16T19:18:55.843808] [INFO ] [REMOTE ] MATX: Full fetch complete (34385 records). [2025-09-16T19:18:55.843928] [DEBUG] [VALIDATOR ] MATX: Validating primary data. [2025-09-16T19:18:55.915497] [DEBUG] [LOCAL ] MCB: Loaded 8093 existing records from primary storage. [2025-09-16T19:18:55.927143] [INFO ] [MAIN ] MCB: Loaded 8093 existing records. [2025-09-16T19:18:56.309089] [WARN ] [MAIN ] MCB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:18:56.330195] [INFO ] [MAIN ] MCB: Wiping existing primary data before full refresh. [2025-09-16T19:18:56.330452] [INFO ] [LOCAL ] MCB: Starting deletion of primary data. [2025-09-16T19:18:56.356488] [DEBUG] [LOCAL ] MCB: Deleted primary data file. [2025-09-16T19:18:56.383677] [INFO ] [MAIN ] MCB: Starting full primary refresh. [2025-09-16T19:18:56.378570] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-08. [2025-09-16T19:18:56.517746] [DEBUG] [VALIDATOR ] MATX: All expected trading days are present. [2025-09-16T19:18:56.520176] [DEBUG] [VALIDATOR ] MATX: Primary data validation passed. [2025-09-16T19:18:56.521206] [INFO ] [MAIN ] MATX: Fetched primary data is valid. [2025-09-16T19:18:56.521573] [DEBUG] [LOCAL ] MATX: Saving primary data. [2025-09-16T19:18:56.581935] [DEBUG] [LOCAL ] MATX: Trimmed 34385 records to 23742 (keeping 150 days of data) [2025-09-16T19:18:56.921019] [INFO ] [REMOTE ] MCB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:18:56.947959] [DEBUG] [LOCAL ] MATX: Primary data saved (23742 records). [2025-09-16T19:18:56.948149] [INFO ] [MAIN ] MATX: Full history saved. [2025-09-16T19:18:56.948259] [INFO ] [MAIN ] Update Progress: 1310/2369 symbols completed. MATX took 36.3s. Estimated remaining: 1h 52m 17s. Rate: 9.4 symbols/min [2025-09-16T19:18:57.128715] [INFO ] [MAIN ] MCD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:18:57.170708] [DEBUG] [LOCAL ] MCD: Loaded 39466 existing records from primary storage. [2025-09-16T19:18:57.170858] [INFO ] [MAIN ] MCD: Loaded 39466 existing records. [2025-09-16T19:18:57.182497] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-02. [2025-09-16T19:18:57.380318] [WARN ] [MAIN ] MCD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:18:57.380762] [INFO ] [MAIN ] MCD: Wiping existing primary data before full refresh. [2025-09-16T19:18:57.380889] [INFO ] [LOCAL ] MCD: Starting deletion of primary data. [2025-09-16T19:18:57.381980] [DEBUG] [LOCAL ] MCD: Deleted primary data file. [2025-09-16T19:18:57.382040] [INFO ] [MAIN ] MCD: Starting full primary refresh. [2025-09-16T19:18:57.693112] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-09-16T19:18:57.776915] [INFO ] [REMOTE ] MCD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:18:57.974911] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-04. [2025-09-16T19:18:58.071562] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-09-16T19:18:58.101323] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-09-16T19:18:58.774903] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-02. [2025-09-16T19:18:59.574915] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-03. [2025-09-16T19:18:59.625223] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T19:19:00.380308] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-09. [2025-09-16T19:19:00.384561] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T19:19:00.454065] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-09-16T19:19:01.174904] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-03. [2025-09-16T19:19:01.843547] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-09-16T19:19:01.849174] [INFO ] [REMOTE ] MBC: Full fetch complete (46898 records). [2025-09-16T19:19:01.849266] [DEBUG] [VALIDATOR ] MBC: Validating primary data. [2025-09-16T19:19:01.989948] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-04. [2025-09-16T19:19:02.226107] [DEBUG] [VALIDATOR ] MBC: All expected trading days are present. [2025-09-16T19:19:02.226190] [DEBUG] [VALIDATOR ] MBC: Primary data validation passed. [2025-09-16T19:19:02.226939] [INFO ] [MAIN ] MBC: Fetched primary data is valid. [2025-09-16T19:19:02.226985] [DEBUG] [LOCAL ] MBC: Saving primary data. [2025-09-16T19:19:02.228052] [DEBUG] [LOCAL ] MBC: Trimmed 46898 records to 31783 (keeping 150 days of data) [2025-09-16T19:19:02.412253] [DEBUG] [LOCAL ] MBC: Primary data saved (31783 records). [2025-09-16T19:19:02.412400] [INFO ] [MAIN ] MBC: Full history saved. [2025-09-16T19:19:02.412491] [INFO ] [MAIN ] Update Progress: 1311/2369 symbols completed. MBC took 27.7s. Estimated remaining: 1h 52m 10s. Rate: 9.4 symbols/min [2025-09-16T19:19:02.587203] [INFO ] [MAIN ] MCI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:02.593891] [DEBUG] [LOCAL ] MCI: Loaded 4602 existing records from primary storage. [2025-09-16T19:19:02.594019] [INFO ] [MAIN ] MCI: Loaded 4602 existing records. [2025-09-16T19:19:02.772389] [WARN ] [MAIN ] MCI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:02.773126] [INFO ] [MAIN ] MCI: Wiping existing primary data before full refresh. [2025-09-16T19:19:02.773187] [INFO ] [LOCAL ] MCI: Starting deletion of primary data. [2025-09-16T19:19:02.773475] [DEBUG] [LOCAL ] MCI: Deleted primary data file. [2025-09-16T19:19:02.773521] [INFO ] [MAIN ] MCI: Starting full primary refresh. [2025-09-16T19:19:02.787709] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-05. [2025-09-16T19:19:03.126862] [INFO ] [REMOTE ] MCI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:03.161957] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-09-16T19:19:03.336437] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-03 (2.16 seconds). [2025-09-16T19:19:03.574909] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-04. [2025-09-16T19:19:04.184795] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T19:19:04.374956] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-05. [2025-09-16T19:19:05.174914] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-06. [2025-09-16T19:19:05.191329] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T19:19:05.455719] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-09-16T19:19:05.974903] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-06. [2025-09-16T19:19:06.524505] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-09-16T19:19:06.607947] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T19:19:06.774936] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-07. [2025-09-16T19:19:07.574979] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-05. [2025-09-16T19:19:08.291363] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-09-16T19:19:08.374856] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-08. [2025-09-16T19:19:09.174910] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-02. [2025-09-16T19:19:09.662218] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-05 (2.09 seconds). [2025-09-16T19:19:09.826453] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-09-16T19:19:09.854089] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T19:19:09.974872] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-03. [2025-09-16T19:19:10.658803] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-09-16T19:19:10.774859] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-04. [2025-09-16T19:19:11.415336] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-09-16T19:19:11.575049] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-05. [2025-09-16T19:19:12.335136] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-09-16T19:19:12.374882] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-06. [2025-09-16T19:19:13.088791] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-09-16T19:19:13.174874] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-07. [2025-09-16T19:19:13.699768] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-09-16T19:19:13.974895] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-08. [2025-09-16T19:19:14.654138] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-09-16T19:19:14.774899] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-09. [2025-09-16T19:19:15.574918] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-06. [2025-09-16T19:19:15.594211] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-09-16T19:19:15.595579] [INFO ] [REMOTE ] MCI: Full fetch complete (7531 records). [2025-09-16T19:19:15.595667] [DEBUG] [VALIDATOR ] MCI: Validating primary data. [2025-09-16T19:19:15.958194] [DEBUG] [VALIDATOR ] MCI: All expected trading days are present. [2025-09-16T19:19:15.958276] [DEBUG] [VALIDATOR ] MCI: Primary data validation passed. [2025-09-16T19:19:15.958489] [INFO ] [MAIN ] MCI: Fetched primary data is valid. [2025-09-16T19:19:15.958554] [DEBUG] [LOCAL ] MCI: Saving primary data. [2025-09-16T19:19:15.958994] [DEBUG] [LOCAL ] MCI: Trimmed 7531 records to 4644 (keeping 150 days of data) [2025-09-16T19:19:15.988149] [DEBUG] [LOCAL ] MCI: Primary data saved (4644 records). [2025-09-16T19:19:15.988290] [INFO ] [MAIN ] MCI: Full history saved. [2025-09-16T19:19:15.988377] [INFO ] [MAIN ] Update Progress: 1312/2369 symbols completed. MCI took 13.6s. Estimated remaining: 1h 52m 9s. Rate: 9.4 symbols/min [2025-09-16T19:19:16.163681] [INFO ] [MAIN ] MCK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:16.192516] [DEBUG] [LOCAL ] MCK: Loaded 28290 existing records from primary storage. [2025-09-16T19:19:16.192669] [INFO ] [MAIN ] MCK: Loaded 28290 existing records. [2025-09-16T19:19:16.371303] [WARN ] [MAIN ] MCK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:16.371578] [INFO ] [MAIN ] MCK: Wiping existing primary data before full refresh. [2025-09-16T19:19:16.371624] [INFO ] [LOCAL ] MCK: Starting deletion of primary data. [2025-09-16T19:19:16.372252] [DEBUG] [LOCAL ] MCK: Deleted primary data file. [2025-09-16T19:19:16.372298] [INFO ] [MAIN ] MCK: Starting full primary refresh. [2025-09-16T19:19:16.383347] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-07. [2025-09-16T19:19:16.729360] [INFO ] [REMOTE ] MCK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:17.174920] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-02. [2025-09-16T19:19:17.352858] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-09-16T19:19:17.564204] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-09-16T19:19:17.974930] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-08. [2025-09-16T19:19:18.714327] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-09-16T19:19:18.748411] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-09-16T19:19:18.774869] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-03. [2025-09-16T19:19:19.574913] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-09. [2025-09-16T19:19:20.366511] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-09-16T19:19:20.375432] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-07. [2025-09-16T19:19:21.071760] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T19:19:21.077255] [INFO ] [REMOTE ] MC: Full fetch complete (48242 records). [2025-09-16T19:19:21.077450] [DEBUG] [VALIDATOR ] MC: Validating primary data. [2025-09-16T19:19:21.183012] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-09. [2025-09-16T19:19:21.468335] [DEBUG] [VALIDATOR ] MC: All expected trading days are present. [2025-09-16T19:19:21.468420] [DEBUG] [VALIDATOR ] MC: Primary data validation passed. [2025-09-16T19:19:21.469305] [INFO ] [MAIN ] MC: Fetched primary data is valid. [2025-09-16T19:19:21.469359] [DEBUG] [LOCAL ] MC: Saving primary data. [2025-09-16T19:19:21.470440] [DEBUG] [LOCAL ] MC: Trimmed 48242 records to 32527 (keeping 150 days of data) [2025-09-16T19:19:21.677627] [DEBUG] [LOCAL ] MC: Primary data saved (32527 records). [2025-09-16T19:19:21.677801] [INFO ] [MAIN ] MC: Full history saved. [2025-09-16T19:19:21.678543] [INFO ] [MAIN ] Update Progress: 1313/2369 symbols completed. MC took 34.7s. Estimated remaining: 1h 52m 3s. Rate: 9.4 symbols/min [2025-09-16T19:19:21.854453] [INFO ] [MAIN ] MCN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:21.863631] [DEBUG] [LOCAL ] MCN: Loaded 5926 existing records from primary storage. [2025-09-16T19:19:21.863826] [INFO ] [MAIN ] MCN: Loaded 5926 existing records. [2025-09-16T19:19:21.985342] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-04. [2025-09-16T19:19:22.048135] [WARN ] [MAIN ] MCN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:22.055297] [INFO ] [MAIN ] MCN: Wiping existing primary data before full refresh. [2025-09-16T19:19:22.055735] [INFO ] [LOCAL ] MCN: Starting deletion of primary data. [2025-09-16T19:19:22.055632] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-09-16T19:19:22.058332] [DEBUG] [LOCAL ] MCN: Deleted primary data file. [2025-09-16T19:19:22.058522] [INFO ] [MAIN ] MCN: Starting full primary refresh. [2025-09-16T19:19:22.058477] [INFO ] [REMOTE ] MCB: Full fetch complete (12560 records). [2025-09-16T19:19:22.127194] [DEBUG] [VALIDATOR ] MCB: Validating primary data. [2025-09-16T19:19:22.701075] [INFO ] [REMOTE ] MCN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:22.787949] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-02. [2025-09-16T19:19:22.894118] [DEBUG] [VALIDATOR ] MCB: All expected trading days are present. [2025-09-16T19:19:22.894213] [DEBUG] [VALIDATOR ] MCB: Primary data validation passed. [2025-09-16T19:19:22.894719] [INFO ] [MAIN ] MCB: Fetched primary data is valid. [2025-09-16T19:19:22.894799] [DEBUG] [LOCAL ] MCB: Saving primary data. [2025-09-16T19:19:22.895510] [DEBUG] [LOCAL ] MCB: Trimmed 12560 records to 8210 (keeping 150 days of data) [2025-09-16T19:19:22.986594] [DEBUG] [LOCAL ] MCB: Primary data saved (8210 records). [2025-09-16T19:19:22.987009] [INFO ] [MAIN ] MCB: Full history saved. [2025-09-16T19:19:22.988353] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-07 (2.61 seconds). [2025-09-16T19:19:22.988550] [INFO ] [MAIN ] Update Progress: 1314/2369 symbols completed. MCB took 27.3s. Estimated remaining: 1h 51m 52s. Rate: 9.4 symbols/min [2025-09-16T19:19:23.169177] [INFO ] [MAIN ] MCO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:23.196417] [DEBUG] [LOCAL ] MCO: Loaded 27717 existing records from primary storage. [2025-09-16T19:19:23.196559] [INFO ] [MAIN ] MCO: Loaded 27717 existing records. [2025-09-16T19:19:23.378439] [WARN ] [MAIN ] MCO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:23.378715] [INFO ] [MAIN ] MCO: Wiping existing primary data before full refresh. [2025-09-16T19:19:23.378764] [INFO ] [LOCAL ] MCO: Starting deletion of primary data. [2025-09-16T19:19:23.379527] [DEBUG] [LOCAL ] MCO: Deleted primary data file. [2025-09-16T19:19:23.379576] [INFO ] [MAIN ] MCO: Starting full primary refresh. [2025-09-16T19:19:23.595562] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-08. [2025-09-16T19:19:23.617175] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-09-16T19:19:23.758775] [INFO ] [REMOTE ] MCO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:23.790422] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-09-16T19:19:24.374908] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-02. [2025-09-16T19:19:25.174909] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-03. [2025-09-16T19:19:25.845946] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-09-16T19:19:25.938943] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-09-16T19:19:25.982588] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-04. [2025-09-16T19:19:25.998500] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-09-16T19:19:26.628104] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-09-16T19:19:26.774896] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-09. [2025-09-16T19:19:27.574905] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-03. [2025-09-16T19:19:28.374906] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-05. [2025-09-16T19:19:28.981238] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-09-16T19:19:28.987919] [INFO ] [REMOTE ] MCD: Full fetch complete (58136 records). [2025-09-16T19:19:28.988046] [DEBUG] [VALIDATOR ] MCD: Validating primary data. [2025-09-16T19:19:29.178599] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-05. [2025-09-16T19:19:29.390349] [DEBUG] [VALIDATOR ] MCD: All expected trading days are present. [2025-09-16T19:19:29.390533] [DEBUG] [VALIDATOR ] MCD: Primary data validation passed. [2025-09-16T19:19:29.391525] [INFO ] [MAIN ] MCD: Fetched primary data is valid. [2025-09-16T19:19:29.391587] [DEBUG] [LOCAL ] MCD: Saving primary data. [2025-09-16T19:19:29.393024] [DEBUG] [LOCAL ] MCD: Trimmed 58136 records to 39855 (keeping 150 days of data) [2025-09-16T19:19:29.492168] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-09-16T19:19:29.693524] [DEBUG] [LOCAL ] MCD: Primary data saved (39855 records). [2025-09-16T19:19:29.693706] [INFO ] [MAIN ] MCD: Full history saved. [2025-09-16T19:19:29.694922] [INFO ] [MAIN ] Update Progress: 1315/2369 symbols completed. MCD took 32.7s. Estimated remaining: 1h 51m 46s. Rate: 9.4 symbols/min [2025-09-16T19:19:29.865482] [INFO ] [MAIN ] MCR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:29.874336] [DEBUG] [LOCAL ] MCR: Loaded 6518 existing records from primary storage. [2025-09-16T19:19:29.874595] [INFO ] [MAIN ] MCR: Loaded 6518 existing records. [2025-09-16T19:19:29.978047] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-04. [2025-09-16T19:19:30.039285] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-09-16T19:19:30.064522] [WARN ] [MAIN ] MCR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:30.067750] [INFO ] [MAIN ] MCR: Wiping existing primary data before full refresh. [2025-09-16T19:19:30.068294] [INFO ] [LOCAL ] MCR: Starting deletion of primary data. [2025-09-16T19:19:30.068688] [DEBUG] [LOCAL ] MCR: Deleted primary data file. [2025-09-16T19:19:30.068740] [INFO ] [MAIN ] MCR: Starting full primary refresh. [2025-09-16T19:19:30.446780] [INFO ] [REMOTE ] MCR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:30.448607] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-09-16T19:19:30.774945] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-06. [2025-09-16T19:19:31.546579] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T19:19:31.574850] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-06. [2025-09-16T19:19:31.874951] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-09-16T19:19:32.374954] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-02. [2025-09-16T19:19:33.021977] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-09-16T19:19:33.098419] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-09-16T19:19:33.174871] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-03. [2025-09-16T19:19:33.817423] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-09-16T19:19:33.974896] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-04. [2025-09-16T19:19:34.751274] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-09-16T19:19:34.774846] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-05. [2025-09-16T19:19:35.528212] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-09-16T19:19:35.574866] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-06. [2025-09-16T19:19:36.373413] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-09-16T19:19:36.379038] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-05. [2025-09-16T19:19:37.174891] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-07. [2025-09-16T19:19:37.887137] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-09-16T19:19:37.912524] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-09-16T19:19:37.974887] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-06. [2025-09-16T19:19:38.774893] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-07. [2025-09-16T19:19:39.406893] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-09-16T19:19:39.574897] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-07. [2025-09-16T19:19:39.911492] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T19:19:39.979594] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-09-16T19:19:40.374895] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-07. [2025-09-16T19:19:41.174914] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-08. [2025-09-16T19:19:41.847100] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-09-16T19:19:41.974892] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-08. [2025-09-16T19:19:42.018622] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T19:19:42.774919] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-08. [2025-09-16T19:19:43.578840] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-09. [2025-09-16T19:19:43.591160] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-09-16T19:19:44.217898] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-09-16T19:19:44.374950] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-09. [2025-09-16T19:19:44.473577] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T19:19:44.474975] [INFO ] [REMOTE ] MCR: Full fetch complete (9809 records). [2025-09-16T19:19:44.475046] [DEBUG] [VALIDATOR ] MCR: Validating primary data. [2025-09-16T19:19:44.847865] [DEBUG] [VALIDATOR ] MCR: All expected trading days are present. [2025-09-16T19:19:44.847941] [DEBUG] [VALIDATOR ] MCR: Primary data validation passed. [2025-09-16T19:19:44.848155] [INFO ] [MAIN ] MCR: Fetched primary data is valid. [2025-09-16T19:19:44.848195] [DEBUG] [LOCAL ] MCR: Saving primary data. [2025-09-16T19:19:44.848633] [DEBUG] [LOCAL ] MCR: Trimmed 9809 records to 6564 (keeping 150 days of data) [2025-09-16T19:19:44.887020] [DEBUG] [LOCAL ] MCR: Primary data saved (6564 records). [2025-09-16T19:19:44.887129] [INFO ] [MAIN ] MCR: Full history saved. [2025-09-16T19:19:44.887210] [INFO ] [MAIN ] Update Progress: 1316/2369 symbols completed. MCR took 15.2s. Estimated remaining: 1h 51m 47s. Rate: 9.4 symbols/min [2025-09-16T19:19:45.062882] [INFO ] [MAIN ] MCS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:45.079230] [DEBUG] [LOCAL ] MCS: Loaded 15437 existing records from primary storage. [2025-09-16T19:19:45.079352] [INFO ] [MAIN ] MCS: Loaded 15437 existing records. [2025-09-16T19:19:45.183820] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-08. [2025-09-16T19:19:45.265951] [WARN ] [MAIN ] MCS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:45.266210] [INFO ] [MAIN ] MCS: Wiping existing primary data before full refresh. [2025-09-16T19:19:45.266266] [INFO ] [LOCAL ] MCS: Starting deletion of primary data. [2025-09-16T19:19:45.266729] [DEBUG] [LOCAL ] MCS: Deleted primary data file. [2025-09-16T19:19:45.266786] [INFO ] [MAIN ] MCS: Starting full primary refresh. [2025-09-16T19:19:45.619666] [INFO ] [REMOTE ] MCS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:45.881856] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-09-16T19:19:45.898574] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T19:19:45.904006] [INFO ] [REMOTE ] MCO: Full fetch complete (43517 records). [2025-09-16T19:19:45.904163] [DEBUG] [VALIDATOR ] MCO: Validating primary data. [2025-09-16T19:19:45.980030] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-09. [2025-09-16T19:19:46.290663] [DEBUG] [VALIDATOR ] MCO: All expected trading days are present. [2025-09-16T19:19:46.290745] [DEBUG] [VALIDATOR ] MCO: Primary data validation passed. [2025-09-16T19:19:46.291457] [INFO ] [MAIN ] MCO: Fetched primary data is valid. [2025-09-16T19:19:46.291498] [DEBUG] [LOCAL ] MCO: Saving primary data. [2025-09-16T19:19:46.292473] [DEBUG] [LOCAL ] MCO: Trimmed 43517 records to 27948 (keeping 150 days of data) [2025-09-16T19:19:46.471608] [DEBUG] [LOCAL ] MCO: Primary data saved (27948 records). [2025-09-16T19:19:46.471754] [INFO ] [MAIN ] MCO: Full history saved. [2025-09-16T19:19:46.471837] [INFO ] [MAIN ] Update Progress: 1317/2369 symbols completed. MCO took 23.5s. Estimated remaining: 1h 51m 37s. Rate: 9.4 symbols/min [2025-09-16T19:19:46.774900] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-09. [2025-09-16T19:19:47.447499] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-09-16T19:19:47.452253] [INFO ] [REMOTE ] MCK: Full fetch complete (43160 records). [2025-09-16T19:19:47.452387] [DEBUG] [VALIDATOR ] MCK: Validating primary data. [2025-09-16T19:19:47.579040] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-02. [2025-09-16T19:19:47.591758] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-09-16T19:19:47.605632] [INFO ] [REMOTE ] MCN: Full fetch complete (8896 records). [2025-09-16T19:19:47.648364] [DEBUG] [VALIDATOR ] MCN: Validating primary data. [2025-09-16T19:19:48.390512] [DEBUG] [VALIDATOR ] MCK: All expected trading days are present. [2025-09-16T19:19:48.422973] [DEBUG] [VALIDATOR ] MCK: Primary data validation passed. [2025-09-16T19:19:48.427209] [INFO ] [MAIN ] MCK: Fetched primary data is valid. [2025-09-16T19:19:48.437961] [DEBUG] [LOCAL ] MCK: Saving primary data. [2025-09-16T19:19:48.508724] [DEBUG] [VALIDATOR ] MCN: All expected trading days are present. [2025-09-16T19:19:48.508877] [DEBUG] [VALIDATOR ] MCN: Primary data validation passed. [2025-09-16T19:19:48.509150] [INFO ] [MAIN ] MCN: Fetched primary data is valid. [2025-09-16T19:19:48.509201] [DEBUG] [LOCAL ] MCN: Saving primary data. [2025-09-16T19:19:48.509302] [DEBUG] [LOCAL ] MCK: Trimmed 43160 records to 28557 (keeping 150 days of data) [2025-09-16T19:19:48.510528] [DEBUG] [LOCAL ] MCN: Trimmed 8896 records to 5967 (keeping 150 days of data) [2025-09-16T19:19:48.689930] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-09-16T19:19:48.697612] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-03. [2025-09-16T19:19:48.709085] [DEBUG] [LOCAL ] MCN: Primary data saved (5967 records). [2025-09-16T19:19:48.709457] [INFO ] [MAIN ] MCN: Full history saved. [2025-09-16T19:19:48.709882] [INFO ] [MAIN ] Update Progress: 1318/2369 symbols completed. MCN took 27.0s. Estimated remaining: 1h 51m 27s. Rate: 9.4 symbols/min [2025-09-16T19:19:48.783152] [DEBUG] [LOCAL ] MCK: Primary data saved (28557 records). [2025-09-16T19:19:48.783287] [INFO ] [MAIN ] MCK: Full history saved. [2025-09-16T19:19:48.783369] [INFO ] [MAIN ] Update Progress: 1319/2369 symbols completed. MCK took 32.8s. Estimated remaining: 1h 51m 15s. Rate: 9.4 symbols/min [2025-09-16T19:19:49.857078] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T19:19:49.863997] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-04. [2025-09-16T19:19:50.897129] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T19:19:50.901795] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-05. [2025-09-16T19:19:52.030917] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-09-16T19:19:52.036768] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-06. [2025-09-16T19:19:52.926380] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T19:19:52.930477] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-07. [2025-09-16T19:19:54.059937] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-09-16T19:19:54.065782] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-08. [2025-09-16T19:19:55.161097] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-09-16T19:19:55.166829] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-09. [2025-09-16T19:19:56.178840] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T19:19:56.182809] [INFO ] [REMOTE ] MCS: Full fetch complete (23386 records). [2025-09-16T19:19:56.182933] [DEBUG] [VALIDATOR ] MCS: Validating primary data. [2025-09-16T19:19:56.637367] [DEBUG] [VALIDATOR ] MCS: All expected trading days are present. [2025-09-16T19:19:56.637449] [DEBUG] [VALIDATOR ] MCS: Primary data validation passed. [2025-09-16T19:19:56.638002] [INFO ] [MAIN ] MCS: Fetched primary data is valid. [2025-09-16T19:19:56.638047] [DEBUG] [LOCAL ] MCS: Saving primary data. [2025-09-16T19:19:56.638724] [DEBUG] [LOCAL ] MCS: Trimmed 23386 records to 15589 (keeping 150 days of data) [2025-09-16T19:19:56.734568] [DEBUG] [LOCAL ] MCS: Primary data saved (15589 records). [2025-09-16T19:19:56.734719] [INFO ] [MAIN ] MCS: Full history saved. [2025-09-16T19:19:56.734804] [INFO ] [MAIN ] Update Progress: 1320/2369 symbols completed. MCS took 11.8s. Estimated remaining: 1h 51m 10s. Rate: 9.4 symbols/min [2025-09-16T19:19:57.341840] [INFO ] [MAIN ] MCY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:57.418484] [INFO ] [MAIN ] MDST: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:57.439186] [INFO ] [MAIN ] MD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:57.438135] [INFO ] [MAIN ] MDT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:19:57.449864] [DEBUG] [LOCAL ] MDST: Loaded 4046 existing records from primary storage. [2025-09-16T19:19:57.450159] [INFO ] [MAIN ] MDST: Loaded 4046 existing records. [2025-09-16T19:19:57.572751] [DEBUG] [LOCAL ] MCY: Loaded 17493 existing records from primary storage. [2025-09-16T19:19:57.578349] [INFO ] [MAIN ] MCY: Loaded 17493 existing records. [2025-09-16T19:19:57.658706] [WARN ] [MAIN ] MDST: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:57.712858] [DEBUG] [LOCAL ] MD: Loaded 28839 existing records from primary storage. [2025-09-16T19:19:57.712995] [INFO ] [MAIN ] MDST: Wiping existing primary data before full refresh. [2025-09-16T19:19:57.718425] [INFO ] [LOCAL ] MDST: Starting deletion of primary data. [2025-09-16T19:19:57.718366] [INFO ] [MAIN ] MD: Loaded 28839 existing records. [2025-09-16T19:19:57.762421] [DEBUG] [LOCAL ] MDST: Deleted primary data file. [2025-09-16T19:19:57.778296] [INFO ] [MAIN ] MDST: Starting full primary refresh. [2025-09-16T19:19:57.980077] [DEBUG] [LOCAL ] MDT: Loaded 39638 existing records from primary storage. [2025-09-16T19:19:58.019257] [INFO ] [MAIN ] MDT: Loaded 39638 existing records. [2025-09-16T19:19:58.068908] [WARN ] [MAIN ] MCY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:58.229261] [WARN ] [MAIN ] MD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:58.229149] [INFO ] [MAIN ] MCY: Wiping existing primary data before full refresh. [2025-09-16T19:19:58.278082] [INFO ] [MAIN ] MD: Wiping existing primary data before full refresh. [2025-09-16T19:19:58.267581] [INFO ] [LOCAL ] MCY: Starting deletion of primary data. [2025-09-16T19:19:58.353286] [DEBUG] [LOCAL ] MCY: Deleted primary data file. [2025-09-16T19:19:58.358484] [INFO ] [LOCAL ] MD: Starting deletion of primary data. [2025-09-16T19:19:58.379113] [INFO ] [MAIN ] MCY: Starting full primary refresh. [2025-09-16T19:19:58.460408] [DEBUG] [LOCAL ] MD: Deleted primary data file. [2025-09-16T19:19:58.606287] [INFO ] [MAIN ] MD: Starting full primary refresh. [2025-09-16T19:19:58.621859] [WARN ] [MAIN ] MDT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:19:58.656120] [INFO ] [MAIN ] MDT: Wiping existing primary data before full refresh. [2025-09-16T19:19:58.695807] [INFO ] [LOCAL ] MDT: Starting deletion of primary data. [2025-09-16T19:19:58.787934] [INFO ] [REMOTE ] MDST: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:58.803403] [DEBUG] [LOCAL ] MDT: Deleted primary data file. [2025-09-16T19:19:58.813892] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-02. [2025-09-16T19:19:58.865941] [INFO ] [MAIN ] MDT: Starting full primary refresh. [2025-09-16T19:19:59.351305] [INFO ] [REMOTE ] MCY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:59.413226] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-02. [2025-09-16T19:19:59.593412] [INFO ] [REMOTE ] MD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:59.618284] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-02. [2025-09-16T19:19:59.696261] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-09-16T19:19:59.700465] [INFO ] [REMOTE ] MDT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:19:59.702151] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-02. [2025-09-16T19:19:59.702563] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-03. [2025-09-16T19:20:00.373066] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-09-16T19:20:00.374982] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-04. [2025-09-16T19:20:01.281012] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T19:20:01.281619] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-09-16T19:20:01.292773] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-03. [2025-09-16T19:20:01.294315] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-05. [2025-09-16T19:20:01.436106] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-09-16T19:20:01.444335] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-03. [2025-09-16T19:20:01.677777] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-09-16T19:20:01.688181] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-03. [2025-09-16T19:20:01.731069] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-09-16T19:20:01.974948] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-06. [2025-09-16T19:20:02.561015] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-09-16T19:20:02.592206] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T19:20:02.779982] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-04. [2025-09-16T19:20:02.793142] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-09-16T19:20:03.416299] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-09-16T19:20:03.574898] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-04. [2025-09-16T19:20:04.003561] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-09-16T19:20:04.374906] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-07. [2025-09-16T19:20:04.839713] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-09-16T19:20:05.077030] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-09-16T19:20:05.174891] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-05. [2025-09-16T19:20:05.974932] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-04. [2025-09-16T19:20:06.518763] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-09-16T19:20:06.774904] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-05. [2025-09-16T19:20:07.574889] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-08. [2025-09-16T19:20:07.646329] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-09-16T19:20:08.187796] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-09-16T19:20:08.213501] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T19:20:08.374929] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-06. [2025-09-16T19:20:09.174889] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-06. [2025-09-16T19:20:09.470976] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-09-16T19:20:09.974900] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-05. [2025-09-16T19:20:10.190220] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-09-16T19:20:10.774906] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-09. [2025-09-16T19:20:11.346475] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-09-16T19:20:11.347966] [INFO ] [REMOTE ] MDST: Full fetch complete (6038 records). [2025-09-16T19:20:11.348035] [DEBUG] [VALIDATOR ] MDST: Validating primary data. [2025-09-16T19:20:11.585256] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-07. [2025-09-16T19:20:11.720491] [DEBUG] [VALIDATOR ] MDST: All expected trading days are present. [2025-09-16T19:20:11.720665] [DEBUG] [VALIDATOR ] MDST: Primary data validation passed. [2025-09-16T19:20:11.720971] [INFO ] [MAIN ] MDST: Fetched primary data is valid. [2025-09-16T19:20:11.721074] [DEBUG] [LOCAL ] MDST: Saving primary data. [2025-09-16T19:20:11.721703] [DEBUG] [LOCAL ] MDST: Trimmed 6038 records to 4121 (keeping 150 days of data) [2025-09-16T19:20:11.748658] [DEBUG] [LOCAL ] MDST: Primary data saved (4121 records). [2025-09-16T19:20:11.748823] [INFO ] [MAIN ] MDST: Full history saved. [2025-09-16T19:20:11.748954] [INFO ] [MAIN ] Update Progress: 1321/2369 symbols completed. MDST took 15.0s. Estimated remaining: 1h 51m 11s. Rate: 9.4 symbols/min [2025-09-16T19:20:11.921678] [INFO ] [MAIN ] MDU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:20:11.961064] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-09-16T19:20:11.998268] [DEBUG] [LOCAL ] MDU: Loaded 34744 existing records from primary storage. [2025-09-16T19:20:11.998415] [INFO ] [MAIN ] MDU: Loaded 34744 existing records. [2025-09-16T19:20:12.170469] [WARN ] [MAIN ] MDU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:20:12.170751] [INFO ] [MAIN ] MDU: Wiping existing primary data before full refresh. [2025-09-16T19:20:12.170820] [INFO ] [LOCAL ] MDU: Starting deletion of primary data. [2025-09-16T19:20:12.171715] [DEBUG] [LOCAL ] MDU: Deleted primary data file. [2025-09-16T19:20:12.171765] [INFO ] [MAIN ] MDU: Starting full primary refresh. [2025-09-16T19:20:12.385113] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-07. [2025-09-16T19:20:12.523191] [INFO ] [REMOTE ] MDU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:20:12.815334] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-09-16T19:20:13.174869] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-02. [2025-09-16T19:20:13.555953] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T19:20:13.974908] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-08. [2025-09-16T19:20:14.578528] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-09-16T19:20:14.774859] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-03. [2025-09-16T19:20:14.937082] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-09-16T19:20:15.574896] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-08. [2025-09-16T19:20:16.374900] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-06. [2025-09-16T19:20:16.539422] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T19:20:17.183688] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-04. [2025-09-16T19:20:17.186792] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-09-16T19:20:17.974929] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-09. [2025-09-16T19:20:18.246470] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-09-16T19:20:18.774913] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-09. [2025-09-16T19:20:18.890927] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-09-16T19:20:19.114956] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T19:20:19.117703] [INFO ] [REMOTE ] MCY: Full fetch complete (29416 records). [2025-09-16T19:20:19.117783] [DEBUG] [VALIDATOR ] MCY: Validating primary data. [2025-09-16T19:20:19.490776] [DEBUG] [VALIDATOR ] MCY: All expected trading days are present. [2025-09-16T19:20:19.490888] [DEBUG] [VALIDATOR ] MCY: Primary data validation passed. [2025-09-16T19:20:19.491483] [INFO ] [MAIN ] MCY: Fetched primary data is valid. [2025-09-16T19:20:19.491525] [DEBUG] [LOCAL ] MCY: Saving primary data. [2025-09-16T19:20:19.492291] [DEBUG] [LOCAL ] MCY: Trimmed 29416 records to 17638 (keeping 150 days of data) [2025-09-16T19:20:19.576012] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-05. [2025-09-16T19:20:19.610680] [DEBUG] [LOCAL ] MCY: Primary data saved (17638 records). [2025-09-16T19:20:19.610834] [INFO ] [MAIN ] MCY: Full history saved. [2025-09-16T19:20:19.610944] [INFO ] [MAIN ] Update Progress: 1322/2369 symbols completed. MCY took 22.9s. Estimated remaining: 1h 51m 6s. Rate: 9.4 symbols/min [2025-09-16T19:20:19.786787] [INFO ] [MAIN ] MDV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:20:19.794774] [DEBUG] [LOCAL ] MDV: Loaded 6491 existing records from primary storage. [2025-09-16T19:20:19.794874] [INFO ] [MAIN ] MDV: Loaded 6491 existing records. [2025-09-16T19:20:19.974288] [WARN ] [MAIN ] MDV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:20:19.974617] [INFO ] [MAIN ] MDV: Wiping existing primary data before full refresh. [2025-09-16T19:20:19.974700] [INFO ] [LOCAL ] MDV: Starting deletion of primary data. [2025-09-16T19:20:19.975060] [DEBUG] [LOCAL ] MDV: Deleted primary data file. [2025-09-16T19:20:19.975132] [INFO ] [MAIN ] MDV: Starting full primary refresh. [2025-09-16T19:20:20.276174] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T19:20:20.336171] [INFO ] [REMOTE ] MD: Full fetch complete (42257 records). [2025-09-16T19:20:20.357791] [DEBUG] [VALIDATOR ] MD: Validating primary data. [2025-09-16T19:20:20.357718] [INFO ] [REMOTE ] MDV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:20:20.379946] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-02. [2025-09-16T19:20:20.733879] [DEBUG] [VALIDATOR ] MD: All expected trading days are present. [2025-09-16T19:20:20.733955] [DEBUG] [VALIDATOR ] MD: Primary data validation passed. [2025-09-16T19:20:20.734660] [INFO ] [MAIN ] MD: Fetched primary data is valid. [2025-09-16T19:20:20.734704] [DEBUG] [LOCAL ] MD: Saving primary data. [2025-09-16T19:20:20.735623] [DEBUG] [LOCAL ] MD: Trimmed 42257 records to 29126 (keeping 150 days of data) [2025-09-16T19:20:20.915352] [DEBUG] [LOCAL ] MD: Primary data saved (29126 records). [2025-09-16T19:20:20.915486] [INFO ] [MAIN ] MD: Full history saved. [2025-09-16T19:20:20.915579] [INFO ] [MAIN ] Update Progress: 1323/2369 symbols completed. MD took 24.2s. Estimated remaining: 1h 50m 55s. Rate: 9.4 symbols/min [2025-09-16T19:20:21.086500] [INFO ] [MAIN ] MEC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:20:21.091443] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-09-16T19:20:21.104608] [DEBUG] [LOCAL ] MEC: Loaded 12261 existing records from primary storage. [2025-09-16T19:20:21.104735] [INFO ] [MAIN ] MEC: Loaded 12261 existing records. [2025-09-16T19:20:21.176215] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-03. [2025-09-16T19:20:21.291671] [WARN ] [MAIN ] MEC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:20:21.292079] [INFO ] [MAIN ] MEC: Wiping existing primary data before full refresh. [2025-09-16T19:20:21.292265] [INFO ] [LOCAL ] MEC: Starting deletion of primary data. [2025-09-16T19:20:21.292811] [DEBUG] [LOCAL ] MEC: Deleted primary data file. [2025-09-16T19:20:21.292963] [INFO ] [MAIN ] MEC: Starting full primary refresh. [2025-09-16T19:20:21.500087] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-09-16T19:20:21.695233] [INFO ] [REMOTE ] MEC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:20:21.756326] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T19:20:21.974916] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-07. [2025-09-16T19:20:22.774954] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-06. [2025-09-16T19:20:23.574902] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-04. [2025-09-16T19:20:23.951366] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-09-16T19:20:24.258979] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T19:20:24.365713] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-09-16T19:20:24.374850] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-05. [2025-09-16T19:20:25.114224] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-09-16T19:20:25.174920] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-07. [2025-09-16T19:20:25.974927] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-06. [2025-09-16T19:20:26.740536] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T19:20:26.774855] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-07. [2025-09-16T19:20:26.859663] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-09-16T19:20:27.510997] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-09-16T19:20:27.574971] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-02. [2025-09-16T19:20:28.374935] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-08. [2025-09-16T19:20:28.393388] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T19:20:29.174927] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-08. [2025-09-16T19:20:29.918068] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-09-16T19:20:29.975377] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-09-16T19:20:29.975444] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-08. [2025-09-16T19:20:30.774916] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-03. [2025-09-16T19:20:31.574890] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-09. [2025-09-16T19:20:31.905786] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T19:20:31.928093] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-09-16T19:20:32.374907] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-04. [2025-09-16T19:20:33.127387] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-09-16T19:20:33.134410] [INFO ] [REMOTE ] MDU: Full fetch complete (52242 records). [2025-09-16T19:20:33.134684] [DEBUG] [VALIDATOR ] MDU: Validating primary data. [2025-09-16T19:20:33.181373] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-09. [2025-09-16T19:20:33.456678] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-09-16T19:20:33.547343] [DEBUG] [VALIDATOR ] MDU: All expected trading days are present. [2025-09-16T19:20:33.547416] [DEBUG] [VALIDATOR ] MDU: Primary data validation passed. [2025-09-16T19:20:33.548238] [INFO ] [MAIN ] MDU: Fetched primary data is valid. [2025-09-16T19:20:33.548284] [DEBUG] [LOCAL ] MDU: Saving primary data. [2025-09-16T19:20:33.549358] [DEBUG] [LOCAL ] MDU: Trimmed 52242 records to 35119 (keeping 150 days of data) [2025-09-16T19:20:33.770330] [DEBUG] [LOCAL ] MDU: Primary data saved (35119 records). [2025-09-16T19:20:33.770466] [INFO ] [MAIN ] MDU: Full history saved. [2025-09-16T19:20:33.770548] [INFO ] [MAIN ] Update Progress: 1324/2369 symbols completed. MDU took 22.0s. Estimated remaining: 1h 50m 54s. Rate: 9.4 symbols/min [2025-09-16T19:20:33.947252] [INFO ] [MAIN ] MED: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:20:33.928714] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-09-16T19:20:33.948849] [INFO ] [REMOTE ] MDV: Full fetch complete (9397 records). [2025-09-16T19:20:33.948921] [DEBUG] [VALIDATOR ] MDV: Validating primary data. [2025-09-16T19:20:33.980020] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-05. [2025-09-16T19:20:34.120221] [DEBUG] [LOCAL ] MED: Loaded 15589 existing records from primary storage. [2025-09-16T19:20:34.129329] [INFO ] [MAIN ] MED: Loaded 15589 existing records. [2025-09-16T19:20:34.543098] [WARN ] [MAIN ] MED: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:20:34.551480] [DEBUG] [VALIDATOR ] MDV: All expected trading days are present. [2025-09-16T19:20:34.551770] [INFO ] [MAIN ] MED: Wiping existing primary data before full refresh. [2025-09-16T19:20:34.551868] [DEBUG] [VALIDATOR ] MDV: Primary data validation passed. [2025-09-16T19:20:34.551929] [INFO ] [LOCAL ] MED: Starting deletion of primary data. [2025-09-16T19:20:34.552501] [INFO ] [MAIN ] MDV: Fetched primary data is valid. [2025-09-16T19:20:34.552671] [DEBUG] [LOCAL ] MDV: Saving primary data. [2025-09-16T19:20:34.553111] [DEBUG] [LOCAL ] MED: Deleted primary data file. [2025-09-16T19:20:34.553305] [INFO ] [MAIN ] MED: Starting full primary refresh. [2025-09-16T19:20:34.587840] [DEBUG] [LOCAL ] MDV: Trimmed 9397 records to 6546 (keeping 150 days of data) [2025-09-16T19:20:34.785030] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-09. [2025-09-16T19:20:34.915791] [DEBUG] [LOCAL ] MDV: Primary data saved (6546 records). [2025-09-16T19:20:34.921536] [INFO ] [MAIN ] MDV: Full history saved. [2025-09-16T19:20:34.936949] [INFO ] [MAIN ] Update Progress: 1325/2369 symbols completed. MDV took 15.3s. Estimated remaining: 1h 50m 44s. Rate: 9.4 symbols/min [2025-09-16T19:20:35.029231] [INFO ] [REMOTE ] MED: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:20:35.163575] [INFO ] [MAIN ] MEG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:20:35.174356] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-09-16T19:20:35.198137] [DEBUG] [LOCAL ] MEG: Loaded 22278 existing records from primary storage. [2025-09-16T19:20:35.198276] [INFO ] [MAIN ] MEG: Loaded 22278 existing records. [2025-09-16T19:20:35.372718] [WARN ] [MAIN ] MEG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:20:35.372953] [INFO ] [MAIN ] MEG: Wiping existing primary data before full refresh. [2025-09-16T19:20:35.373006] [INFO ] [LOCAL ] MEG: Starting deletion of primary data. [2025-09-16T19:20:35.373607] [DEBUG] [LOCAL ] MEG: Deleted primary data file. [2025-09-16T19:20:35.373684] [INFO ] [MAIN ] MEG: Starting full primary refresh. [2025-09-16T19:20:35.580014] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-02. [2025-09-16T19:20:35.743074] [INFO ] [REMOTE ] MEG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:20:36.374915] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-02. [2025-09-16T19:20:36.540255] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-09-16T19:20:36.547099] [INFO ] [REMOTE ] MDT: Full fetch complete (58358 records). [2025-09-16T19:20:36.547209] [DEBUG] [VALIDATOR ] MDT: Validating primary data. [2025-09-16T19:20:36.930244] [DEBUG] [VALIDATOR ] MDT: All expected trading days are present. [2025-09-16T19:20:36.930354] [DEBUG] [VALIDATOR ] MDT: Primary data validation passed. [2025-09-16T19:20:36.931209] [INFO ] [MAIN ] MDT: Fetched primary data is valid. [2025-09-16T19:20:36.931267] [DEBUG] [LOCAL ] MDT: Saving primary data. [2025-09-16T19:20:36.946196] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T19:20:36.951267] [DEBUG] [LOCAL ] MDT: Trimmed 58358 records to 40028 (keeping 150 days of data) [2025-09-16T19:20:37.174922] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-06. [2025-09-16T19:20:37.204765] [DEBUG] [LOCAL ] MDT: Primary data saved (40028 records). [2025-09-16T19:20:37.204877] [INFO ] [MAIN ] MDT: Full history saved. [2025-09-16T19:20:37.204961] [INFO ] [MAIN ] Update Progress: 1326/2369 symbols completed. MDT took 40.5s. Estimated remaining: 1h 50m 34s. Rate: 9.4 symbols/min [2025-09-16T19:20:37.404348] [INFO ] [MAIN ] MEGI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:20:37.419728] [DEBUG] [LOCAL ] MEGI: Loaded 14795 existing records from primary storage. [2025-09-16T19:20:37.419855] [INFO ] [MAIN ] MEGI: Loaded 14795 existing records. [2025-09-16T19:20:37.599946] [WARN ] [MAIN ] MEGI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:20:37.600301] [INFO ] [MAIN ] MEGI: Wiping existing primary data before full refresh. [2025-09-16T19:20:37.600401] [INFO ] [LOCAL ] MEGI: Starting deletion of primary data. [2025-09-16T19:20:37.600969] [DEBUG] [LOCAL ] MEGI: Deleted primary data file. [2025-09-16T19:20:37.601109] [INFO ] [MAIN ] MEGI: Starting full primary refresh. [2025-09-16T19:20:37.747214] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-09-16T19:20:37.980031] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-03. [2025-09-16T19:20:37.983793] [INFO ] [REMOTE ] MEGI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:20:38.172737] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T19:20:38.774949] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-02. [2025-09-16T19:20:39.041124] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-09-16T19:20:39.574900] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-03. [2025-09-16T19:20:39.752771] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-09-16T19:20:40.374923] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-04. [2025-09-16T19:20:40.806607] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-09-16T19:20:41.174966] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-07. [2025-09-16T19:20:41.571778] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-09-16T19:20:41.974918] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-04. [2025-09-16T19:20:42.066541] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-09-16T19:20:42.774908] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-03. [2025-09-16T19:20:43.249536] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-09-16T19:20:43.574890] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-05. [2025-09-16T19:20:43.619670] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-09-16T19:20:44.374898] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-04. [2025-09-16T19:20:44.636979] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-09-16T19:20:45.174928] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-05. [2025-09-16T19:20:45.208608] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-09-16T19:20:45.974932] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-06. [2025-09-16T19:20:46.493858] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T19:20:46.774888] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-08. [2025-09-16T19:20:47.046990] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-09-16T19:20:47.531961] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-09-16T19:20:47.574862] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-06. [2025-09-16T19:20:48.374940] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-05. [2025-09-16T19:20:48.567855] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-09-16T19:20:49.174939] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-09. [2025-09-16T19:20:49.361942] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T19:20:49.974947] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-06. [2025-09-16T19:20:50.093249] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T19:20:50.096595] [INFO ] [REMOTE ] MEC: Full fetch complete (17508 records). [2025-09-16T19:20:50.096737] [DEBUG] [VALIDATOR ] MEC: Validating primary data. [2025-09-16T19:20:50.461941] [DEBUG] [VALIDATOR ] MEC: All expected trading days are present. [2025-09-16T19:20:50.462019] [DEBUG] [VALIDATOR ] MEC: Primary data validation passed. [2025-09-16T19:20:50.462388] [INFO ] [MAIN ] MEC: Fetched primary data is valid. [2025-09-16T19:20:50.462469] [DEBUG] [LOCAL ] MEC: Saving primary data. [2025-09-16T19:20:50.463033] [DEBUG] [LOCAL ] MEC: Trimmed 17508 records to 12359 (keeping 150 days of data) [2025-09-16T19:20:50.535832] [DEBUG] [LOCAL ] MEC: Primary data saved (12359 records). [2025-09-16T19:20:50.535984] [INFO ] [MAIN ] MEC: Full history saved. [2025-09-16T19:20:50.536076] [INFO ] [MAIN ] Update Progress: 1327/2369 symbols completed. MEC took 29.6s. Estimated remaining: 1h 50m 33s. Rate: 9.4 symbols/min [2025-09-16T19:20:50.710102] [INFO ] [MAIN ] MEI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:20:50.731809] [DEBUG] [LOCAL ] MEI: Loaded 23106 existing records from primary storage. [2025-09-16T19:20:50.731953] [INFO ] [MAIN ] MEI: Loaded 23106 existing records. [2025-09-16T19:20:50.787090] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-07. [2025-09-16T19:20:50.912664] [WARN ] [MAIN ] MEI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:20:50.921666] [INFO ] [MAIN ] MEI: Wiping existing primary data before full refresh. [2025-09-16T19:20:50.921822] [INFO ] [LOCAL ] MEI: Starting deletion of primary data. [2025-09-16T19:20:50.922611] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-09-16T19:20:50.923898] [DEBUG] [LOCAL ] MEI: Deleted primary data file. [2025-09-16T19:20:50.926326] [INFO ] [MAIN ] MEI: Starting full primary refresh. [2025-09-16T19:20:51.276612] [INFO ] [REMOTE ] MEI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:20:51.574867] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-07. [2025-09-16T19:20:51.893186] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-09-16T19:20:52.374875] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-07. [2025-09-16T19:20:52.695335] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-09-16T19:20:53.174895] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-08. [2025-09-16T19:20:53.517083] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T19:20:53.974960] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-08. [2025-09-16T19:20:54.270313] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-09-16T19:20:54.774937] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-02. [2025-09-16T19:20:55.128984] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-09-16T19:20:55.574884] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-08. [2025-09-16T19:20:55.921173] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-09-16T19:20:56.374915] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-09. [2025-09-16T19:20:56.715472] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-09-16T19:20:57.174885] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-09. [2025-09-16T19:20:57.367310] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-09-16T19:20:57.369796] [INFO ] [REMOTE ] MED: Full fetch complete (24383 records). [2025-09-16T19:20:57.369869] [DEBUG] [VALIDATOR ] MED: Validating primary data. [2025-09-16T19:20:57.751608] [DEBUG] [VALIDATOR ] MED: All expected trading days are present. [2025-09-16T19:20:57.751701] [DEBUG] [VALIDATOR ] MED: Primary data validation passed. [2025-09-16T19:20:57.752194] [INFO ] [MAIN ] MED: Fetched primary data is valid. [2025-09-16T19:20:57.752235] [DEBUG] [LOCAL ] MED: Saving primary data. [2025-09-16T19:20:57.752985] [DEBUG] [LOCAL ] MED: Trimmed 24383 records to 15705 (keeping 150 days of data) [2025-09-16T19:20:57.849884] [DEBUG] [LOCAL ] MED: Primary data saved (15705 records). [2025-09-16T19:20:57.850038] [INFO ] [MAIN ] MED: Full history saved. [2025-09-16T19:20:57.850146] [INFO ] [MAIN ] Update Progress: 1328/2369 symbols completed. MED took 24.1s. Estimated remaining: 1h 50m 28s. Rate: 9.4 symbols/min [2025-09-16T19:20:57.979935] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-09. [2025-09-16T19:20:58.015095] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T19:20:58.042348] [INFO ] [MAIN ] MET: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:20:58.042671] [INFO ] [REMOTE ] MEGI: Full fetch complete (22190 records). [2025-09-16T19:20:58.043103] [DEBUG] [VALIDATOR ] MEGI: Validating primary data. [2025-09-16T19:20:58.357857] [DEBUG] [LOCAL ] MET: Loaded 39323 existing records from primary storage. [2025-09-16T19:20:58.363614] [INFO ] [MAIN ] MET: Loaded 39323 existing records. [2025-09-16T19:20:58.558389] [DEBUG] [VALIDATOR ] MEGI: All expected trading days are present. [2025-09-16T19:20:58.558796] [DEBUG] [VALIDATOR ] MEGI: Primary data validation passed. [2025-09-16T19:20:58.569853] [INFO ] [MAIN ] MEGI: Fetched primary data is valid. [2025-09-16T19:20:58.575019] [DEBUG] [LOCAL ] MEGI: Saving primary data. [2025-09-16T19:20:58.641958] [DEBUG] [LOCAL ] MEGI: Trimmed 22190 records to 15098 (keeping 150 days of data) [2025-09-16T19:20:58.642092] [WARN ] [MAIN ] MET: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:20:58.642717] [INFO ] [MAIN ] MET: Wiping existing primary data before full refresh. [2025-09-16T19:20:58.642821] [INFO ] [LOCAL ] MET: Starting deletion of primary data. [2025-09-16T19:20:58.643808] [DEBUG] [LOCAL ] MET: Deleted primary data file. [2025-09-16T19:20:58.691184] [INFO ] [MAIN ] MET: Starting full primary refresh. [2025-09-16T19:20:58.784620] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-03. [2025-09-16T19:20:58.951601] [DEBUG] [LOCAL ] MEGI: Primary data saved (15098 records). [2025-09-16T19:20:58.962399] [INFO ] [MAIN ] MEGI: Full history saved. [2025-09-16T19:20:59.025143] [INFO ] [MAIN ] Update Progress: 1329/2369 symbols completed. MEGI took 21.8s. Estimated remaining: 1h 50m 17s. Rate: 9.4 symbols/min [2025-09-16T19:20:59.387360] [INFO ] [REMOTE ] MET: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:20:59.514152] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-09-16T19:20:59.526143] [INFO ] [MAIN ] MFA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:20:59.523702] [INFO ] [REMOTE ] MEG: Full fetch complete (34914 records). [2025-09-16T19:20:59.527428] [DEBUG] [VALIDATOR ] MEG: Validating primary data. [2025-09-16T19:20:59.577947] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-02. [2025-09-16T19:20:59.729355] [DEBUG] [LOCAL ] MFA: Loaded 32905 existing records from primary storage. [2025-09-16T19:20:59.729567] [INFO ] [MAIN ] MFA: Loaded 32905 existing records. [2025-09-16T19:21:00.191332] [WARN ] [MAIN ] MFA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:21:00.201113] [INFO ] [MAIN ] MFA: Wiping existing primary data before full refresh. [2025-09-16T19:21:00.216708] [INFO ] [LOCAL ] MFA: Starting deletion of primary data. [2025-09-16T19:21:00.256356] [DEBUG] [LOCAL ] MFA: Deleted primary data file. [2025-09-16T19:21:00.275930] [INFO ] [MAIN ] MFA: Starting full primary refresh. [2025-09-16T19:21:00.284664] [DEBUG] [VALIDATOR ] MEG: All expected trading days are present. [2025-09-16T19:21:00.284804] [DEBUG] [VALIDATOR ] MEG: Primary data validation passed. [2025-09-16T19:21:00.316797] [INFO ] [MAIN ] MEG: Fetched primary data is valid. [2025-09-16T19:21:00.341098] [DEBUG] [LOCAL ] MEG: Saving primary data. [2025-09-16T19:21:00.394594] [DEBUG] [LOCAL ] MEG: Trimmed 34914 records to 22538 (keeping 150 days of data) [2025-09-16T19:21:00.734397] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-09-16T19:21:00.798065] [INFO ] [REMOTE ] MFA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:21:00.799171] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-04. [2025-09-16T19:21:00.855504] [DEBUG] [LOCAL ] MEG: Primary data saved (22538 records). [2025-09-16T19:21:00.855661] [INFO ] [MAIN ] MEG: Full history saved. [2025-09-16T19:21:00.855754] [INFO ] [MAIN ] Update Progress: 1330/2369 symbols completed. MEG took 25.9s. Estimated remaining: 1h 50m 7s. Rate: 9.4 symbols/min [2025-09-16T19:21:01.029142] [INFO ] [MAIN ] MFAN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:21:01.033571] [DEBUG] [LOCAL ] MFAN: Loaded 3011 existing records from primary storage. [2025-09-16T19:21:01.033670] [INFO ] [MAIN ] MFAN: Loaded 3011 existing records. [2025-09-16T19:21:01.184448] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-02. [2025-09-16T19:21:01.211540] [WARN ] [MAIN ] MFAN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:21:01.212658] [INFO ] [MAIN ] MFAN: Wiping existing primary data before full refresh. [2025-09-16T19:21:01.212708] [INFO ] [LOCAL ] MFAN: Starting deletion of primary data. [2025-09-16T19:21:01.212922] [DEBUG] [LOCAL ] MFAN: Deleted primary data file. [2025-09-16T19:21:01.212966] [INFO ] [MAIN ] MFAN: Starting full primary refresh. [2025-09-16T19:21:01.579873] [INFO ] [REMOTE ] MFAN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:21:01.611393] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-02 (2.03 seconds). [2025-09-16T19:21:01.974955] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-02. [2025-09-16T19:21:02.272491] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T19:21:02.448346] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-09-16T19:21:02.460170] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-09-16T19:21:02.774949] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-03. [2025-09-16T19:21:03.574904] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-05. [2025-09-16T19:21:04.349228] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-09-16T19:21:04.374864] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-06. [2025-09-16T19:21:04.579112] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T19:21:05.174902] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-04. [2025-09-16T19:21:05.625580] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T19:21:05.974909] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-03. [2025-09-16T19:21:06.774929] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-03. [2025-09-16T19:21:06.997699] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T19:21:07.341943] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-09-16T19:21:07.582815] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-04. [2025-09-16T19:21:07.600098] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T19:21:08.272990] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-09-16T19:21:08.374909] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-05. [2025-09-16T19:21:09.174909] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-05. [2025-09-16T19:21:09.757604] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-09-16T19:21:09.974910] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-04. [2025-09-16T19:21:10.227041] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-09-16T19:21:10.774906] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-06. [2025-09-16T19:21:11.339006] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-09-16T19:21:11.497474] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T19:21:11.574882] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-07. [2025-09-16T19:21:12.192017] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-09-16T19:21:12.374935] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-05. [2025-09-16T19:21:13.174955] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-06. [2025-09-16T19:21:13.983328] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-08. [2025-09-16T19:21:14.016412] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T19:21:14.514277] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-09-16T19:21:14.774900] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-09. [2025-09-16T19:21:14.876273] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T19:21:15.326266] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-09-16T19:21:15.327405] [INFO ] [REMOTE ] MFAN: Full fetch complete (4540 records). [2025-09-16T19:21:15.327500] [DEBUG] [VALIDATOR ] MFAN: Validating primary data. [2025-09-16T19:21:15.580051] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-07. [2025-09-16T19:21:15.707508] [DEBUG] [VALIDATOR ] MFAN: All expected trading days are present. [2025-09-16T19:21:15.707591] [DEBUG] [VALIDATOR ] MFAN: Primary data validation passed. [2025-09-16T19:21:15.707746] [INFO ] [MAIN ] MFAN: Fetched primary data is valid. [2025-09-16T19:21:15.707821] [DEBUG] [LOCAL ] MFAN: Saving primary data. [2025-09-16T19:21:15.708293] [DEBUG] [LOCAL ] MFAN: Trimmed 4540 records to 3025 (keeping 150 days of data) [2025-09-16T19:21:15.728188] [DEBUG] [LOCAL ] MFAN: Primary data saved (3025 records). [2025-09-16T19:21:15.728315] [INFO ] [MAIN ] MFAN: Full history saved. [2025-09-16T19:21:15.728412] [INFO ] [MAIN ] Update Progress: 1331/2369 symbols completed. MFAN took 14.9s. Estimated remaining: 1h 50m 8s. Rate: 9.4 symbols/min [2025-09-16T19:21:15.899384] [INFO ] [MAIN ] MFAO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:21:15.903921] [DEBUG] [LOCAL ] MFAO: Loaded 2818 existing records from primary storage. [2025-09-16T19:21:15.904008] [INFO ] [MAIN ] MFAO: Loaded 2818 existing records. [2025-09-16T19:21:16.082728] [WARN ] [MAIN ] MFAO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:21:16.082970] [INFO ] [MAIN ] MFAO: Wiping existing primary data before full refresh. [2025-09-16T19:21:16.083018] [INFO ] [LOCAL ] MFAO: Starting deletion of primary data. [2025-09-16T19:21:16.083241] [DEBUG] [LOCAL ] MFAO: Deleted primary data file. [2025-09-16T19:21:16.083287] [INFO ] [MAIN ] MFAO: Starting full primary refresh. [2025-09-16T19:21:16.375035] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-06. [2025-09-16T19:21:16.454486] [INFO ] [REMOTE ] MFAO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:21:17.174940] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-02. [2025-09-16T19:21:17.518663] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-09-16T19:21:17.864225] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-09-16T19:21:17.974865] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-03. [2025-09-16T19:21:18.002848] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T19:21:18.353447] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-09-16T19:21:18.774906] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-07. [2025-09-16T19:21:19.574921] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-04. [2025-09-16T19:21:20.208770] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-09-16T19:21:20.342934] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T19:21:20.374871] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-08. [2025-09-16T19:21:21.174915] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-08. [2025-09-16T19:21:21.894256] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-09-16T19:21:21.974946] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-07. [2025-09-16T19:21:22.628343] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T19:21:22.774925] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-05. [2025-09-16T19:21:23.348598] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-09-16T19:21:23.394509] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-09-16T19:21:23.574933] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-09. [2025-09-16T19:21:24.374905] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-06. [2025-09-16T19:21:25.052993] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-09-16T19:21:25.174905] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-07. [2025-09-16T19:21:25.407461] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-09-16T19:21:25.413771] [INFO ] [REMOTE ] MET: Full fetch complete (57960 records). [2025-09-16T19:21:25.413892] [DEBUG] [VALIDATOR ] MET: Validating primary data. [2025-09-16T19:21:25.528450] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-09-16T19:21:25.810227] [DEBUG] [VALIDATOR ] MET: All expected trading days are present. [2025-09-16T19:21:25.810319] [DEBUG] [VALIDATOR ] MET: Primary data validation passed. [2025-09-16T19:21:25.811162] [INFO ] [MAIN ] MET: Fetched primary data is valid. [2025-09-16T19:21:25.811211] [DEBUG] [LOCAL ] MET: Saving primary data. [2025-09-16T19:21:25.812502] [DEBUG] [LOCAL ] MET: Trimmed 57960 records to 39713 (keeping 150 days of data) [2025-09-16T19:21:26.023073] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-09. [2025-09-16T19:21:26.100067] [DEBUG] [LOCAL ] MET: Primary data saved (39713 records). [2025-09-16T19:21:26.100224] [INFO ] [MAIN ] MET: Full history saved. [2025-09-16T19:21:26.100311] [INFO ] [MAIN ] Update Progress: 1332/2369 symbols completed. MET took 28.3s. Estimated remaining: 1h 50m 4s. Rate: 9.4 symbols/min [2025-09-16T19:21:26.275782] [INFO ] [MAIN ] MFC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:21:26.314183] [DEBUG] [LOCAL ] MFC: Loaded 37624 existing records from primary storage. [2025-09-16T19:21:26.314352] [INFO ] [MAIN ] MFC: Loaded 37624 existing records. [2025-09-16T19:21:26.490941] [WARN ] [MAIN ] MFC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:21:26.491204] [INFO ] [MAIN ] MFC: Wiping existing primary data before full refresh. [2025-09-16T19:21:26.491257] [INFO ] [LOCAL ] MFC: Starting deletion of primary data. [2025-09-16T19:21:26.492136] [DEBUG] [LOCAL ] MFC: Deleted primary data file. [2025-09-16T19:21:26.492185] [INFO ] [MAIN ] MFC: Starting full primary refresh. [2025-09-16T19:21:26.777030] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-08. [2025-09-16T19:21:26.856116] [INFO ] [REMOTE ] MFC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:21:27.574909] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-08. [2025-09-16T19:21:27.633074] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T19:21:27.638520] [INFO ] [REMOTE ] MFA: Full fetch complete (48976 records). [2025-09-16T19:21:27.638613] [DEBUG] [VALIDATOR ] MFA: Validating primary data. [2025-09-16T19:21:28.023652] [DEBUG] [VALIDATOR ] MFA: All expected trading days are present. [2025-09-16T19:21:28.023788] [DEBUG] [VALIDATOR ] MFA: Primary data validation passed. [2025-09-16T19:21:28.024618] [INFO ] [MAIN ] MFA: Fetched primary data is valid. [2025-09-16T19:21:28.024706] [DEBUG] [LOCAL ] MFA: Saving primary data. [2025-09-16T19:21:28.044120] [DEBUG] [LOCAL ] MFA: Trimmed 48976 records to 33246 (keeping 150 days of data) [2025-09-16T19:21:28.044850] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-09-16T19:21:28.148565] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-09-16T19:21:28.260767] [DEBUG] [LOCAL ] MFA: Primary data saved (33246 records). [2025-09-16T19:21:28.260924] [INFO ] [MAIN ] MFA: Full history saved. [2025-09-16T19:21:28.261012] [INFO ] [MAIN ] Update Progress: 1333/2369 symbols completed. MFA took 29.2s. Estimated remaining: 1h 49m 55s. Rate: 9.4 symbols/min [2025-09-16T19:21:28.380024] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-09. [2025-09-16T19:21:28.434991] [INFO ] [MAIN ] MFG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:21:28.473728] [DEBUG] [LOCAL ] MFG: Loaded 36559 existing records from primary storage. [2025-09-16T19:21:28.473941] [INFO ] [MAIN ] MFG: Loaded 36559 existing records. [2025-09-16T19:21:28.666087] [WARN ] [MAIN ] MFG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:21:28.666363] [INFO ] [MAIN ] MFG: Wiping existing primary data before full refresh. [2025-09-16T19:21:28.666411] [INFO ] [LOCAL ] MFG: Starting deletion of primary data. [2025-09-16T19:21:28.667124] [DEBUG] [LOCAL ] MFG: Deleted primary data file. [2025-09-16T19:21:28.667172] [INFO ] [MAIN ] MFG: Starting full primary refresh. [2025-09-16T19:21:29.018635] [INFO ] [REMOTE ] MFG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:21:29.174906] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-09. [2025-09-16T19:21:29.739057] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T19:21:29.740058] [INFO ] [REMOTE ] MFAO: Full fetch complete (4185 records). [2025-09-16T19:21:29.740127] [DEBUG] [VALIDATOR ] MFAO: Validating primary data. [2025-09-16T19:21:29.978449] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-02. [2025-09-16T19:21:29.999953] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T19:21:30.033208] [INFO ] [REMOTE ] MEI: Full fetch complete (34604 records). [2025-09-16T19:21:30.045990] [DEBUG] [VALIDATOR ] MEI: Validating primary data. [2025-09-16T19:21:30.272499] [DEBUG] [VALIDATOR ] MFAO: All expected trading days are present. [2025-09-16T19:21:30.286725] [DEBUG] [VALIDATOR ] MFAO: Primary data validation passed. [2025-09-16T19:21:30.304157] [INFO ] [MAIN ] MFAO: Fetched primary data is valid. [2025-09-16T19:21:30.304849] [DEBUG] [LOCAL ] MFAO: Saving primary data. [2025-09-16T19:21:30.337764] [DEBUG] [LOCAL ] MFAO: Trimmed 4185 records to 2842 (keeping 150 days of data) [2025-09-16T19:21:30.452474] [DEBUG] [LOCAL ] MFAO: Primary data saved (2842 records). [2025-09-16T19:21:30.452616] [INFO ] [MAIN ] MFAO: Full history saved. [2025-09-16T19:21:30.453020] [INFO ] [MAIN ] Update Progress: 1334/2369 symbols completed. MFAO took 14.7s. Estimated remaining: 1h 49m 45s. Rate: 9.4 symbols/min [2025-09-16T19:21:30.650294] [DEBUG] [VALIDATOR ] MEI: All expected trading days are present. [2025-09-16T19:21:30.655485] [DEBUG] [VALIDATOR ] MEI: Primary data validation passed. [2025-09-16T19:21:30.680706] [INFO ] [MAIN ] MEI: Fetched primary data is valid. [2025-09-16T19:21:30.687329] [DEBUG] [LOCAL ] MEI: Saving primary data. [2025-09-16T19:21:30.711876] [INFO ] [MAIN ] MFM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:21:30.712760] [DEBUG] [LOCAL ] MEI: Trimmed 34604 records to 23284 (keeping 150 days of data) [2025-09-16T19:21:30.775300] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-02. [2025-09-16T19:21:30.810000] [DEBUG] [LOCAL ] MFM: Loaded 9588 existing records from primary storage. [2025-09-16T19:21:30.811451] [INFO ] [MAIN ] MFM: Loaded 9588 existing records. [2025-09-16T19:21:31.018711] [WARN ] [MAIN ] MFM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:21:31.021818] [INFO ] [MAIN ] MFM: Wiping existing primary data before full refresh. [2025-09-16T19:21:31.025891] [INFO ] [LOCAL ] MFM: Starting deletion of primary data. [2025-09-16T19:21:31.040959] [DEBUG] [LOCAL ] MFM: Deleted primary data file. [2025-09-16T19:21:31.042178] [INFO ] [MAIN ] MFM: Starting full primary refresh. [2025-09-16T19:21:31.119977] [DEBUG] [LOCAL ] MEI: Primary data saved (23284 records). [2025-09-16T19:21:31.142754] [INFO ] [MAIN ] MEI: Full history saved. [2025-09-16T19:21:31.169466] [INFO ] [MAIN ] Update Progress: 1335/2369 symbols completed. MEI took 40.6s. Estimated remaining: 1h 49m 34s. Rate: 9.4 symbols/min [2025-09-16T19:21:31.571172] [INFO ] [MAIN ] MFSB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:21:31.597504] [DEBUG] [LOCAL ] MFSB: No existing data file found in primary storage. [2025-09-16T19:21:31.636068] [INFO ] [REMOTE ] MFM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:21:31.636409] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-02. [2025-09-16T19:21:31.631427] [WARN ] [MAIN ] MFSB: No existing data found, performing full refresh instead. [2025-09-16T19:21:31.636668] [INFO ] [MAIN ] MFSB: Wiping existing primary data before full refresh. [2025-09-16T19:21:31.637570] [INFO ] [LOCAL ] MFSB: Starting deletion of primary data. [2025-09-16T19:21:31.637682] [DEBUG] [LOCAL ] MFSB: No primary data file found to delete. [2025-09-16T19:21:31.637729] [INFO ] [MAIN ] MFSB: Starting full primary refresh. [2025-09-16T19:21:32.057571] [INFO ] [REMOTE ] MFSB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:21:32.060842] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-02 (2.08 seconds). [2025-09-16T19:21:32.373362] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-09-16T19:21:32.385089] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-03. [2025-09-16T19:21:32.478878] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T19:21:33.174905] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-02. [2025-09-16T19:21:33.403081] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-02 (0.23 seconds). [2025-09-16T19:21:33.974899] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-03. [2025-09-16T19:21:34.195306] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T19:21:34.353589] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-09-16T19:21:34.774900] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-04. [2025-09-16T19:21:35.574900] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-03. [2025-09-16T19:21:36.374922] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-04. [2025-09-16T19:21:36.694260] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-04 (0.32 seconds). [2025-09-16T19:21:37.174910] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-03. [2025-09-16T19:21:37.969573] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-09-16T19:21:37.974831] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-04. [2025-09-16T19:21:38.774933] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-05. [2025-09-16T19:21:39.018979] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-09-16T19:21:39.125112] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T19:21:39.574934] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-05. [2025-09-16T19:21:40.102345] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-04 (5.33 seconds). [2025-09-16T19:21:40.374945] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-06. [2025-09-16T19:21:40.707788] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-09-16T19:21:41.174958] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-07. [2025-09-16T19:21:41.490416] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-09-16T19:21:41.876996] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-03 (6.30 seconds). [2025-09-16T19:21:41.974974] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-05. [2025-09-16T19:21:42.232232] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-05 (2.66 seconds). [2025-09-16T19:21:42.774904] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-04. [2025-09-16T19:21:43.574917] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-06. [2025-09-16T19:21:43.739835] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T19:21:44.374902] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-08. [2025-09-16T19:21:44.482228] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-09-16T19:21:44.519792] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-09-16T19:21:44.824236] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-09-16T19:21:45.175046] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-07. [2025-09-16T19:21:45.974939] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-09. [2025-09-16T19:21:46.111609] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-09-16T19:21:46.626482] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-09-16T19:21:46.627458] [INFO ] [REMOTE ] MFSB: Full fetch complete (1132 records). [2025-09-16T19:21:46.627528] [DEBUG] [VALIDATOR ] MFSB: Validating primary data. [2025-09-16T19:21:46.780039] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-06. [2025-09-16T19:21:46.808667] [WARN ] [VALIDATOR ] MFSB: Validation failed - insufficient historical data. Have 132 unique days, need 150 days [2025-09-16T19:21:46.808890] [FATAL] [MAIN ] MFSB: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:21:46.808971] [WARN ] [MAIN ] MFSB: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:21:46.809033] [INFO ] [LOCAL ] MFSB: Starting deletion of primary data. [2025-09-16T19:21:46.809109] [DEBUG] [LOCAL ] MFSB: No primary data file found to delete. [2025-09-16T19:21:46.809170] [INFO ] [LOCAL ] MFSB: Starting deletion of derivative data. [2025-09-16T19:21:46.809235] [DEBUG] [LOCAL ] MFSB: No derivative data file found to delete. [2025-09-16T19:21:46.809333] [INFO ] [MAIN ] Update Progress: 1336/2369 symbols completed. MFSB took 15.6s. Estimated remaining: 1h 49m 35s. Rate: 9.4 symbols/min [2025-09-16T19:21:46.991300] [INFO ] [MAIN ] MFSG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:21:46.997454] [DEBUG] [LOCAL ] MFSG: Loaded 3906 existing records from primary storage. [2025-09-16T19:21:46.997556] [INFO ] [MAIN ] MFSG: Loaded 3906 existing records. [2025-09-16T19:21:47.183867] [WARN ] [MAIN ] MFSG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:21:47.184166] [INFO ] [MAIN ] MFSG: Wiping existing primary data before full refresh. [2025-09-16T19:21:47.184216] [INFO ] [LOCAL ] MFSG: Starting deletion of primary data. [2025-09-16T19:21:47.184452] [DEBUG] [LOCAL ] MFSG: Deleted primary data file. [2025-09-16T19:21:47.184498] [INFO ] [MAIN ] MFSG: Starting full primary refresh. [2025-09-16T19:21:47.543720] [INFO ] [REMOTE ] MFSG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:21:47.574905] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-02. [2025-09-16T19:21:48.000162] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-09-16T19:21:48.374864] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-05. [2025-09-16T19:21:48.488098] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-09-16T19:21:49.174952] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-07. [2025-09-16T19:21:49.974943] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-03. [2025-09-16T19:21:50.184115] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-09-16T19:21:50.412771] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-09-16T19:21:50.716965] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-09-16T19:21:50.774907] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-08. [2025-09-16T19:21:51.574958] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-08. [2025-09-16T19:21:52.374970] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-06. [2025-09-16T19:21:53.174973] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-04. [2025-09-16T19:21:53.519483] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-08 (2.74 seconds). [2025-09-16T19:21:53.656186] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-09-16T19:21:53.754237] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-09-16T19:21:53.974917] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-05. [2025-09-16T19:21:54.670007] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-06 (2.30 seconds). [2025-09-16T19:21:54.688745] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-09-16T19:21:54.774892] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-06. [2025-09-16T19:21:55.375880] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-09-16T19:21:55.574906] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-09. [2025-09-16T19:21:56.374919] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-07. [2025-09-16T19:21:57.175805] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-09. [2025-09-16T19:21:57.176998] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T19:21:57.183560] [INFO ] [REMOTE ] MFC: Full fetch complete (55970 records). [2025-09-16T19:21:57.183674] [DEBUG] [VALIDATOR ] MFC: Validating primary data. [2025-09-16T19:21:57.575677] [DEBUG] [VALIDATOR ] MFC: All expected trading days are present. [2025-09-16T19:21:57.575791] [DEBUG] [VALIDATOR ] MFC: Primary data validation passed. [2025-09-16T19:21:57.576726] [INFO ] [MAIN ] MFC: Fetched primary data is valid. [2025-09-16T19:21:57.576781] [DEBUG] [LOCAL ] MFC: Saving primary data. [2025-09-16T19:21:57.577956] [DEBUG] [LOCAL ] MFC: Trimmed 55970 records to 37998 (keeping 150 days of data) [2025-09-16T19:21:57.814414] [DEBUG] [LOCAL ] MFC: Primary data saved (37998 records). [2025-09-16T19:21:57.814687] [INFO ] [MAIN ] MFC: Full history saved. [2025-09-16T19:21:57.814795] [INFO ] [MAIN ] Update Progress: 1337/2369 symbols completed. MFC took 31.7s. Estimated remaining: 1h 49m 32s. Rate: 9.4 symbols/min [2025-09-16T19:21:57.975940] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-07. [2025-09-16T19:21:57.994023] [INFO ] [MAIN ] MFSI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:21:58.004763] [DEBUG] [LOCAL ] MFSI: Loaded 8872 existing records from primary storage. [2025-09-16T19:21:58.005142] [INFO ] [MAIN ] MFSI: Loaded 8872 existing records. [2025-09-16T19:21:58.184679] [WARN ] [MAIN ] MFSI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:21:58.192159] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-09-16T19:21:58.192240] [INFO ] [MAIN ] MFSI: Wiping existing primary data before full refresh. [2025-09-16T19:21:58.192422] [INFO ] [LOCAL ] MFSI: Starting deletion of primary data. [2025-09-16T19:21:58.195584] [INFO ] [REMOTE ] MFM: Full fetch complete (15255 records). [2025-09-16T19:21:58.195672] [DEBUG] [LOCAL ] MFSI: Deleted primary data file. [2025-09-16T19:21:58.195900] [INFO ] [MAIN ] MFSI: Starting full primary refresh. [2025-09-16T19:21:58.195814] [DEBUG] [VALIDATOR ] MFM: Validating primary data. [2025-09-16T19:21:58.511243] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-09-16T19:21:58.723392] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-09-16T19:21:58.796736] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-08. [2025-09-16T19:21:58.937852] [DEBUG] [VALIDATOR ] MFM: All expected trading days are present. [2025-09-16T19:21:58.958822] [DEBUG] [VALIDATOR ] MFM: Primary data validation passed. [2025-09-16T19:21:58.975915] [INFO ] [MAIN ] MFM: Fetched primary data is valid. [2025-09-16T19:21:58.976008] [DEBUG] [LOCAL ] MFM: Saving primary data. [2025-09-16T19:21:58.999254] [INFO ] [REMOTE ] MFSI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:21:58.999674] [DEBUG] [LOCAL ] MFM: Trimmed 15255 records to 9639 (keeping 150 days of data) [2025-09-16T19:21:59.058166] [DEBUG] [LOCAL ] MFM: Primary data saved (9639 records). [2025-09-16T19:21:59.058337] [INFO ] [MAIN ] MFM: Full history saved. [2025-09-16T19:21:59.058437] [INFO ] [MAIN ] Update Progress: 1338/2369 symbols completed. MFM took 28.6s. Estimated remaining: 1h 49m 22s. Rate: 9.4 symbols/min [2025-09-16T19:21:59.227361] [INFO ] [MAIN ] MFSV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:21:59.230681] [DEBUG] [LOCAL ] MFSV: Loaded 1681 existing records from primary storage. [2025-09-16T19:21:59.230760] [INFO ] [MAIN ] MFSV: Loaded 1681 existing records. [2025-09-16T19:21:59.409037] [WARN ] [MAIN ] MFSV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:21:59.409297] [INFO ] [MAIN ] MFSV: Wiping existing primary data before full refresh. [2025-09-16T19:21:59.409345] [INFO ] [LOCAL ] MFSV: Starting deletion of primary data. [2025-09-16T19:21:59.409529] [DEBUG] [LOCAL ] MFSV: Deleted primary data file. [2025-09-16T19:21:59.409572] [INFO ] [MAIN ] MFSV: Starting full primary refresh. [2025-09-16T19:21:59.581517] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-02. [2025-09-16T19:21:59.682677] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-09-16T19:21:59.770772] [INFO ] [REMOTE ] MFSV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:00.234447] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-09-16T19:22:00.374980] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-08. [2025-09-16T19:22:01.174928] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-02. [2025-09-16T19:22:01.630874] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-09-16T19:22:01.981603] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-09. [2025-09-16T19:22:02.007674] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-09-16T19:22:02.774922] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-03. [2025-09-16T19:22:02.822877] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-09-16T19:22:02.824335] [INFO ] [REMOTE ] MFSG: Full fetch complete (4976 records). [2025-09-16T19:22:02.824404] [DEBUG] [VALIDATOR ] MFSG: Validating primary data. [2025-09-16T19:22:03.188502] [DEBUG] [VALIDATOR ] MFSG: All expected trading days are present. [2025-09-16T19:22:03.188580] [DEBUG] [VALIDATOR ] MFSG: Primary data validation passed. [2025-09-16T19:22:03.188763] [INFO ] [MAIN ] MFSG: Fetched primary data is valid. [2025-09-16T19:22:03.188808] [DEBUG] [LOCAL ] MFSG: Saving primary data. [2025-09-16T19:22:03.189493] [DEBUG] [LOCAL ] MFSG: Trimmed 4976 records to 3995 (keeping 150 days of data) [2025-09-16T19:22:03.211837] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-09-16T19:22:03.219239] [DEBUG] [LOCAL ] MFSG: Primary data saved (3995 records). [2025-09-16T19:22:03.219313] [INFO ] [MAIN ] MFSG: Full history saved. [2025-09-16T19:22:03.219412] [INFO ] [MAIN ] Update Progress: 1339/2369 symbols completed. MFSG took 16.4s. Estimated remaining: 1h 49m 14s. Rate: 9.4 symbols/min [2025-09-16T19:22:03.393583] [INFO ] [MAIN ] MG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:22:03.405331] [DEBUG] [LOCAL ] MG: Loaded 11429 existing records from primary storage. [2025-09-16T19:22:03.405444] [INFO ] [MAIN ] MG: Loaded 11429 existing records. [2025-09-16T19:22:03.581488] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-04. [2025-09-16T19:22:03.587569] [WARN ] [MAIN ] MG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:22:03.588576] [INFO ] [MAIN ] MG: Wiping existing primary data before full refresh. [2025-09-16T19:22:03.588663] [INFO ] [LOCAL ] MG: Starting deletion of primary data. [2025-09-16T19:22:03.589043] [DEBUG] [LOCAL ] MG: Deleted primary data file. [2025-09-16T19:22:03.589107] [INFO ] [MAIN ] MG: Starting full primary refresh. [2025-09-16T19:22:03.951914] [INFO ] [REMOTE ] MG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:04.131710] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T19:22:04.374934] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-05. [2025-09-16T19:22:04.799201] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-09-16T19:22:05.174942] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-02. [2025-09-16T19:22:05.973713] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-09-16T19:22:05.974892] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-09. [2025-09-16T19:22:06.774929] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-03. [2025-09-16T19:22:07.403627] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-09-16T19:22:07.500874] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-09-16T19:22:07.512833] [INFO ] [REMOTE ] MFG: Full fetch complete (52796 records). [2025-09-16T19:22:07.513006] [DEBUG] [VALIDATOR ] MFG: Validating primary data. [2025-09-16T19:22:07.579997] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-06. [2025-09-16T19:22:07.893722] [DEBUG] [VALIDATOR ] MFG: All expected trading days are present. [2025-09-16T19:22:07.893805] [DEBUG] [VALIDATOR ] MFG: Primary data validation passed. [2025-09-16T19:22:07.894610] [INFO ] [MAIN ] MFG: Fetched primary data is valid. [2025-09-16T19:22:07.894676] [DEBUG] [LOCAL ] MFG: Saving primary data. [2025-09-16T19:22:07.896297] [DEBUG] [LOCAL ] MFG: Trimmed 52796 records to 36915 (keeping 150 days of data) [2025-09-16T19:22:08.106578] [DEBUG] [LOCAL ] MFG: Primary data saved (36915 records). [2025-09-16T19:22:08.106745] [INFO ] [MAIN ] MFG: Full history saved. [2025-09-16T19:22:08.106845] [INFO ] [MAIN ] Update Progress: 1340/2369 symbols completed. MFG took 39.8s. Estimated remaining: 1h 49m 6s. Rate: 9.4 symbols/min [2025-09-16T19:22:08.189904] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-09-16T19:22:08.286267] [INFO ] [MAIN ] MGA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:22:08.286464] [DEBUG] [LOCAL ] MGA: No existing data file found in primary storage. [2025-09-16T19:22:08.286508] [WARN ] [MAIN ] MGA: No existing data found, performing full refresh instead. [2025-09-16T19:22:08.286677] [INFO ] [MAIN ] MGA: Wiping existing primary data before full refresh. [2025-09-16T19:22:08.286757] [INFO ] [LOCAL ] MGA: Starting deletion of primary data. [2025-09-16T19:22:08.286833] [DEBUG] [LOCAL ] MGA: No primary data file found to delete. [2025-09-16T19:22:08.286897] [INFO ] [MAIN ] MGA: Starting full primary refresh. [2025-09-16T19:22:08.383969] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-07. [2025-09-16T19:22:08.641409] [INFO ] [REMOTE ] MGA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:08.845344] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-09-16T19:22:09.174930] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-02. [2025-09-16T19:22:09.974896] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-03. [2025-09-16T19:22:10.774872] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-04. [2025-09-16T19:22:10.915871] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T19:22:11.129934] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T19:22:11.169231] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-04 (0.39 seconds). [2025-09-16T19:22:11.574918] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-03. [2025-09-16T19:22:12.374959] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-04. [2025-09-16T19:22:13.174929] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-05. [2025-09-16T19:22:13.357394] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-09-16T19:22:13.392617] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-09-16T19:22:13.943281] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-09-16T19:22:13.974916] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-08. [2025-09-16T19:22:14.592061] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-09-16T19:22:14.774908] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-04. [2025-09-16T19:22:15.574926] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-06. [2025-09-16T19:22:16.374902] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-09. [2025-09-16T19:22:16.475111] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-09-16T19:22:16.518369] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-09-16T19:22:17.019972] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-09-16T19:22:17.021552] [INFO ] [REMOTE ] MFSV: Full fetch complete (2394 records). [2025-09-16T19:22:17.021721] [DEBUG] [VALIDATOR ] MFSV: Validating primary data. [2025-09-16T19:22:17.180345] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-05. [2025-09-16T19:22:17.378327] [DEBUG] [VALIDATOR ] MFSV: All expected trading days are present. [2025-09-16T19:22:17.378402] [DEBUG] [VALIDATOR ] MFSV: Primary data validation passed. [2025-09-16T19:22:17.378493] [INFO ] [MAIN ] MFSV: Fetched primary data is valid. [2025-09-16T19:22:17.378542] [DEBUG] [LOCAL ] MFSV: Saving primary data. [2025-09-16T19:22:17.378891] [DEBUG] [LOCAL ] MFSV: Trimmed 2394 records to 1713 (keeping 150 days of data) [2025-09-16T19:22:17.390286] [DEBUG] [LOCAL ] MFSV: Primary data saved (1713 records). [2025-09-16T19:22:17.390345] [INFO ] [MAIN ] MFSV: Full history saved. [2025-09-16T19:22:17.390461] [INFO ] [MAIN ] Update Progress: 1341/2369 symbols completed. MFSV took 18.3s. Estimated remaining: 1h 49m 2s. Rate: 9.4 symbols/min [2025-09-16T19:22:17.556749] [INFO ] [MAIN ] MGF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:22:17.562064] [DEBUG] [LOCAL ] MGF: Loaded 3170 existing records from primary storage. [2025-09-16T19:22:17.562139] [INFO ] [MAIN ] MGF: Loaded 3170 existing records. [2025-09-16T19:22:17.733708] [WARN ] [MAIN ] MGF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:22:17.733922] [INFO ] [MAIN ] MGF: Wiping existing primary data before full refresh. [2025-09-16T19:22:17.733967] [INFO ] [LOCAL ] MGF: Starting deletion of primary data. [2025-09-16T19:22:17.734246] [DEBUG] [LOCAL ] MGF: Deleted primary data file. [2025-09-16T19:22:17.734292] [INFO ] [MAIN ] MGF: Starting full primary refresh. [2025-09-16T19:22:17.976059] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-05. [2025-09-16T19:22:18.090852] [INFO ] [REMOTE ] MGF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:18.286063] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-09-16T19:22:18.774901] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-02. [2025-09-16T19:22:19.333395] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-09-16T19:22:19.574884] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-06. [2025-09-16T19:22:19.687526] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-09-16T19:22:20.313882] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-09-16T19:22:20.374971] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-03. [2025-09-16T19:22:20.885664] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-09-16T19:22:21.174960] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-04. [2025-09-16T19:22:21.740195] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-09-16T19:22:21.974950] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-06. [2025-09-16T19:22:22.774911] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-07. [2025-09-16T19:22:23.580215] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-07. [2025-09-16T19:22:23.598966] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-09-16T19:22:23.644758] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T19:22:24.374965] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-05. [2025-09-16T19:22:24.490892] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-09-16T19:22:24.907522] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-09-16T19:22:25.174929] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-06. [2025-09-16T19:22:25.622886] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-09-16T19:22:25.974933] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-07. [2025-09-16T19:22:26.513693] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-09-16T19:22:26.774906] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-08. [2025-09-16T19:22:27.574962] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-08. [2025-09-16T19:22:27.706069] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T19:22:28.159579] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-09-16T19:22:28.374897] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-08. [2025-09-16T19:22:29.174958] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-09. [2025-09-16T19:22:29.249509] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-09-16T19:22:29.694020] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-09-16T19:22:29.695341] [INFO ] [REMOTE ] MGF: Full fetch complete (4458 records). [2025-09-16T19:22:29.695431] [DEBUG] [VALIDATOR ] MGF: Validating primary data. [2025-09-16T19:22:29.980166] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-07. [2025-09-16T19:22:30.094833] [DEBUG] [VALIDATOR ] MGF: All expected trading days are present. [2025-09-16T19:22:30.094915] [DEBUG] [VALIDATOR ] MGF: Primary data validation passed. [2025-09-16T19:22:30.095072] [INFO ] [MAIN ] MGF: Fetched primary data is valid. [2025-09-16T19:22:30.095113] [DEBUG] [LOCAL ] MGF: Saving primary data. [2025-09-16T19:22:30.095543] [DEBUG] [LOCAL ] MGF: Trimmed 4458 records to 3182 (keeping 150 days of data) [2025-09-16T19:22:30.116150] [DEBUG] [LOCAL ] MGF: Primary data saved (3182 records). [2025-09-16T19:22:30.116258] [INFO ] [MAIN ] MGF: Full history saved. [2025-09-16T19:22:30.116369] [INFO ] [MAIN ] Update Progress: 1342/2369 symbols completed. MGF took 12.7s. Estimated remaining: 1h 49m 1s. Rate: 9.4 symbols/min [2025-09-16T19:22:30.289978] [INFO ] [MAIN ] MGM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:22:30.336429] [DEBUG] [LOCAL ] MGM: Loaded 39574 existing records from primary storage. [2025-09-16T19:22:30.336577] [INFO ] [MAIN ] MGM: Loaded 39574 existing records. [2025-09-16T19:22:30.512417] [WARN ] [MAIN ] MGM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:22:30.512707] [INFO ] [MAIN ] MGM: Wiping existing primary data before full refresh. [2025-09-16T19:22:30.512766] [INFO ] [LOCAL ] MGM: Starting deletion of primary data. [2025-09-16T19:22:30.513745] [DEBUG] [LOCAL ] MGM: Deleted primary data file. [2025-09-16T19:22:30.513795] [INFO ] [MAIN ] MGM: Starting full primary refresh. [2025-09-16T19:22:30.780029] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-09. [2025-09-16T19:22:30.868717] [INFO ] [REMOTE ] MGM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:31.574900] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-09. [2025-09-16T19:22:31.722093] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-09-16T19:22:31.746146] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T19:22:31.748588] [INFO ] [REMOTE ] MFSI: Full fetch complete (10984 records). [2025-09-16T19:22:31.748688] [DEBUG] [VALIDATOR ] MFSI: Validating primary data. [2025-09-16T19:22:32.111728] [DEBUG] [VALIDATOR ] MFSI: All expected trading days are present. [2025-09-16T19:22:32.111807] [DEBUG] [VALIDATOR ] MFSI: Primary data validation passed. [2025-09-16T19:22:32.112052] [INFO ] [MAIN ] MFSI: Fetched primary data is valid. [2025-09-16T19:22:32.112092] [DEBUG] [LOCAL ] MFSI: Saving primary data. [2025-09-16T19:22:32.112551] [DEBUG] [LOCAL ] MFSI: Trimmed 10984 records to 9004 (keeping 150 days of data) [2025-09-16T19:22:32.171720] [DEBUG] [LOCAL ] MFSI: Primary data saved (9004 records). [2025-09-16T19:22:32.171867] [INFO ] [MAIN ] MFSI: Full history saved. [2025-09-16T19:22:32.171964] [INFO ] [MAIN ] Update Progress: 1343/2369 symbols completed. MFSI took 34.4s. Estimated remaining: 1h 48m 51s. Rate: 9.4 symbols/min [2025-09-16T19:22:32.342396] [INFO ] [MAIN ] MGR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:22:32.348598] [DEBUG] [LOCAL ] MGR: Loaded 4099 existing records from primary storage. [2025-09-16T19:22:32.348729] [INFO ] [MAIN ] MGR: Loaded 4099 existing records. [2025-09-16T19:22:32.380053] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-08. [2025-09-16T19:22:32.460132] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T19:22:32.504616] [INFO ] [REMOTE ] MG: Full fetch complete (18051 records). [2025-09-16T19:22:32.522627] [DEBUG] [VALIDATOR ] MG: Validating primary data. [2025-09-16T19:22:32.547264] [WARN ] [MAIN ] MGR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:22:32.628722] [INFO ] [MAIN ] MGR: Wiping existing primary data before full refresh. [2025-09-16T19:22:32.650037] [INFO ] [LOCAL ] MGR: Starting deletion of primary data. [2025-09-16T19:22:32.666214] [DEBUG] [LOCAL ] MGR: Deleted primary data file. [2025-09-16T19:22:32.690021] [INFO ] [MAIN ] MGR: Starting full primary refresh. [2025-09-16T19:22:33.179435] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-02. [2025-09-16T19:22:33.195594] [DEBUG] [VALIDATOR ] MG: All expected trading days are present. [2025-09-16T19:22:33.207371] [DEBUG] [VALIDATOR ] MG: Primary data validation passed. [2025-09-16T19:22:33.207845] [INFO ] [MAIN ] MG: Fetched primary data is valid. [2025-09-16T19:22:33.208237] [DEBUG] [LOCAL ] MG: Saving primary data. [2025-09-16T19:22:33.268728] [DEBUG] [LOCAL ] MG: Trimmed 18051 records to 11519 (keeping 150 days of data) [2025-09-16T19:22:33.289758] [INFO ] [REMOTE ] MGR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:33.362436] [DEBUG] [LOCAL ] MG: Primary data saved (11519 records). [2025-09-16T19:22:33.362603] [INFO ] [MAIN ] MG: Full history saved. [2025-09-16T19:22:33.362718] [INFO ] [MAIN ] Update Progress: 1344/2369 symbols completed. MG took 30.1s. Estimated remaining: 1h 48m 41s. Rate: 9.4 symbols/min [2025-09-16T19:22:33.533742] [INFO ] [MAIN ] MGRB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:22:33.538513] [DEBUG] [LOCAL ] MGRB: Loaded 3036 existing records from primary storage. [2025-09-16T19:22:33.538628] [INFO ] [MAIN ] MGRB: Loaded 3036 existing records. [2025-09-16T19:22:33.713411] [WARN ] [MAIN ] MGRB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:22:33.713633] [INFO ] [MAIN ] MGRB: Wiping existing primary data before full refresh. [2025-09-16T19:22:33.713699] [INFO ] [LOCAL ] MGRB: Starting deletion of primary data. [2025-09-16T19:22:33.713906] [DEBUG] [LOCAL ] MGRB: Deleted primary data file. [2025-09-16T19:22:33.713949] [INFO ] [MAIN ] MGRB: Starting full primary refresh. [2025-09-16T19:22:33.984376] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-02. [2025-09-16T19:22:34.064771] [INFO ] [REMOTE ] MGRB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:34.207060] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-09-16T19:22:34.577135] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-09-16T19:22:34.774912] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-09. [2025-09-16T19:22:34.866245] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-09-16T19:22:35.574929] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-03. [2025-09-16T19:22:36.210838] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T19:22:36.216838] [INFO ] [REMOTE ] MGA: Full fetch complete (55097 records). [2025-09-16T19:22:36.216934] [DEBUG] [VALIDATOR ] MGA: Validating primary data. [2025-09-16T19:22:36.391406] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-02. [2025-09-16T19:22:36.598945] [DEBUG] [VALIDATOR ] MGA: All expected trading days are present. [2025-09-16T19:22:36.599025] [DEBUG] [VALIDATOR ] MGA: Primary data validation passed. [2025-09-16T19:22:36.599836] [INFO ] [MAIN ] MGA: Fetched primary data is valid. [2025-09-16T19:22:36.599880] [DEBUG] [LOCAL ] MGA: Saving primary data. [2025-09-16T19:22:36.601052] [DEBUG] [LOCAL ] MGA: Trimmed 55097 records to 37416 (keeping 150 days of data) [2025-09-16T19:22:36.834779] [DEBUG] [LOCAL ] MGA: Primary data saved (37416 records). [2025-09-16T19:22:36.834903] [INFO ] [MAIN ] MGA: Full history saved. [2025-09-16T19:22:36.835021] [INFO ] [MAIN ] Update Progress: 1345/2369 symbols completed. MGA took 28.7s. Estimated remaining: 1h 48m 32s. Rate: 9.4 symbols/min [2025-09-16T19:22:37.005026] [INFO ] [MAIN ] MGRD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:22:37.008195] [DEBUG] [LOCAL ] MGRD: Loaded 1640 existing records from primary storage. [2025-09-16T19:22:37.008264] [INFO ] [MAIN ] MGRD: Loaded 1640 existing records. [2025-09-16T19:22:37.112883] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-09-16T19:22:37.185497] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-03. [2025-09-16T19:22:37.190200] [WARN ] [MAIN ] MGRD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:22:37.191250] [INFO ] [MAIN ] MGRD: Wiping existing primary data before full refresh. [2025-09-16T19:22:37.191321] [INFO ] [LOCAL ] MGRD: Starting deletion of primary data. [2025-09-16T19:22:37.191522] [DEBUG] [LOCAL ] MGRD: Deleted primary data file. [2025-09-16T19:22:37.191567] [INFO ] [MAIN ] MGRD: Starting full primary refresh. [2025-09-16T19:22:37.555232] [INFO ] [REMOTE ] MGRD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:37.585596] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-09-16T19:22:37.612697] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-09-16T19:22:37.974878] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-02. [2025-09-16T19:22:38.422359] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-09-16T19:22:38.774913] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-03. [2025-09-16T19:22:39.203462] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-09-16T19:22:39.574892] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-04. [2025-09-16T19:22:40.374975] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-04. [2025-09-16T19:22:40.892015] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-09-16T19:22:41.174950] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-03. [2025-09-16T19:22:41.545951] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-09-16T19:22:41.656906] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-09-16T19:22:41.974911] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-05. [2025-09-16T19:22:42.422554] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-09-16T19:22:42.774940] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-04. [2025-09-16T19:22:43.372147] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-09-16T19:22:43.574952] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-04. [2025-09-16T19:22:44.277883] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-09-16T19:22:44.374966] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-05. [2025-09-16T19:22:44.973704] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-09-16T19:22:45.174968] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-05. [2025-09-16T19:22:45.754764] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-09-16T19:22:45.974910] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-06. [2025-09-16T19:22:46.658242] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-09-16T19:22:46.774901] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-05. [2025-09-16T19:22:47.574908] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-06. [2025-09-16T19:22:48.018292] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-09-16T19:22:48.374950] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-06. [2025-09-16T19:22:48.626972] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-09-16T19:22:48.773691] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-09-16T19:22:49.174970] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-07. [2025-09-16T19:22:49.737620] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-09-16T19:22:49.974953] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-07. [2025-09-16T19:22:50.442714] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-09-16T19:22:50.774937] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-08. [2025-09-16T19:22:51.212849] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-09-16T19:22:51.574934] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-09. [2025-09-16T19:22:52.028774] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-09-16T19:22:52.029556] [INFO ] [REMOTE ] MGRD: Full fetch complete (2516 records). [2025-09-16T19:22:52.029624] [DEBUG] [VALIDATOR ] MGRD: Validating primary data. [2025-09-16T19:22:52.377277] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-08. [2025-09-16T19:22:52.382623] [DEBUG] [VALIDATOR ] MGRD: All expected trading days are present. [2025-09-16T19:22:52.382729] [DEBUG] [VALIDATOR ] MGRD: Primary data validation passed. [2025-09-16T19:22:52.382861] [INFO ] [MAIN ] MGRD: Fetched primary data is valid. [2025-09-16T19:22:52.382900] [DEBUG] [LOCAL ] MGRD: Saving primary data. [2025-09-16T19:22:52.383274] [DEBUG] [LOCAL ] MGRD: Trimmed 2516 records to 1647 (keeping 150 days of data) [2025-09-16T19:22:52.394270] [DEBUG] [LOCAL ] MGRD: Primary data saved (1647 records). [2025-09-16T19:22:52.394361] [INFO ] [MAIN ] MGRD: Full history saved. [2025-09-16T19:22:52.394462] [INFO ] [MAIN ] Update Progress: 1346/2369 symbols completed. MGRD took 15.6s. Estimated remaining: 1h 48m 33s. Rate: 9.4 symbols/min [2025-09-16T19:22:52.577087] [INFO ] [MAIN ] MGRE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:22:52.584316] [DEBUG] [LOCAL ] MGRE: Loaded 4814 existing records from primary storage. [2025-09-16T19:22:52.584445] [INFO ] [MAIN ] MGRE: Loaded 4814 existing records. [2025-09-16T19:22:52.771182] [WARN ] [MAIN ] MGRE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:22:52.771445] [INFO ] [MAIN ] MGRE: Wiping existing primary data before full refresh. [2025-09-16T19:22:52.771492] [INFO ] [LOCAL ] MGRE: Starting deletion of primary data. [2025-09-16T19:22:52.771813] [DEBUG] [LOCAL ] MGRE: Deleted primary data file. [2025-09-16T19:22:52.771860] [INFO ] [MAIN ] MGRE: Starting full primary refresh. [2025-09-16T19:22:53.058925] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-09-16T19:22:53.135942] [INFO ] [REMOTE ] MGRE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:53.174862] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-09. [2025-09-16T19:22:53.714192] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-09-16T19:22:53.715154] [INFO ] [REMOTE ] MGRB: Full fetch complete (4542 records). [2025-09-16T19:22:53.715226] [DEBUG] [VALIDATOR ] MGRB: Validating primary data. [2025-09-16T19:22:53.980083] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-06. [2025-09-16T19:22:54.087736] [DEBUG] [VALIDATOR ] MGRB: All expected trading days are present. [2025-09-16T19:22:54.087820] [DEBUG] [VALIDATOR ] MGRB: Primary data validation passed. [2025-09-16T19:22:54.087960] [INFO ] [MAIN ] MGRB: Fetched primary data is valid. [2025-09-16T19:22:54.087999] [DEBUG] [LOCAL ] MGRB: Saving primary data. [2025-09-16T19:22:54.088362] [DEBUG] [LOCAL ] MGRB: Trimmed 4542 records to 3049 (keeping 150 days of data) [2025-09-16T19:22:54.106805] [DEBUG] [LOCAL ] MGRB: Primary data saved (3049 records). [2025-09-16T19:22:54.106872] [INFO ] [MAIN ] MGRB: Full history saved. [2025-09-16T19:22:54.106955] [INFO ] [MAIN ] Update Progress: 1347/2369 symbols completed. MGRB took 20.7s. Estimated remaining: 1h 48m 23s. Rate: 9.4 symbols/min [2025-09-16T19:22:54.279249] [INFO ] [MAIN ] MGY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:22:54.319608] [DEBUG] [LOCAL ] MGY: Loaded 38628 existing records from primary storage. [2025-09-16T19:22:54.319769] [INFO ] [MAIN ] MGY: Loaded 38628 existing records. [2025-09-16T19:22:54.498391] [WARN ] [MAIN ] MGY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:22:54.498630] [INFO ] [MAIN ] MGY: Wiping existing primary data before full refresh. [2025-09-16T19:22:54.498697] [INFO ] [LOCAL ] MGY: Starting deletion of primary data. [2025-09-16T19:22:54.499565] [DEBUG] [LOCAL ] MGY: Deleted primary data file. [2025-09-16T19:22:54.499612] [INFO ] [MAIN ] MGY: Starting full primary refresh. [2025-09-16T19:22:54.778981] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-02. [2025-09-16T19:22:54.858759] [INFO ] [REMOTE ] MGY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:22:55.571785] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-09-16T19:22:55.574834] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-03. [2025-09-16T19:22:56.124943] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-06 (2.14 seconds). [2025-09-16T19:22:56.274524] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-09-16T19:22:56.374855] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-07. [2025-09-16T19:22:57.174911] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-07. [2025-09-16T19:22:57.895992] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-09-16T19:22:57.974934] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-02. [2025-09-16T19:22:58.352515] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-09-16T19:22:58.774904] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-04. [2025-09-16T19:22:59.264270] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-09-16T19:22:59.574898] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-08. [2025-09-16T19:23:00.374909] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-08. [2025-09-16T19:23:01.174980] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-03. [2025-09-16T19:23:02.263185] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-09-16T19:23:02.275053] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-09. [2025-09-16T19:23:02.795523] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-09-16T19:23:02.806519] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-04. [2025-09-16T19:23:03.810118] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-04 (5.04 seconds). [2025-09-16T19:23:03.812988] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-05. [2025-09-16T19:23:03.941301] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-08 (4.37 seconds). [2025-09-16T19:23:04.350389] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-09-16T19:23:04.359488] [INFO ] [REMOTE ] MGM: Full fetch complete (58282 records). [2025-09-16T19:23:04.359774] [DEBUG] [VALIDATOR ] MGM: Validating primary data. [2025-09-16T19:23:04.395742] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-09. [2025-09-16T19:23:04.499099] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-09-16T19:23:04.775080] [DEBUG] [VALIDATOR ] MGM: All expected trading days are present. [2025-09-16T19:23:04.775247] [DEBUG] [VALIDATOR ] MGM: Primary data validation passed. [2025-09-16T19:23:04.776197] [INFO ] [MAIN ] MGM: Fetched primary data is valid. [2025-09-16T19:23:04.776261] [DEBUG] [LOCAL ] MGM: Saving primary data. [2025-09-16T19:23:04.777744] [DEBUG] [LOCAL ] MGM: Trimmed 58282 records to 39964 (keeping 150 days of data) [2025-09-16T19:23:04.897307] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-04 (2.09 seconds). [2025-09-16T19:23:05.054259] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-09-16T19:23:05.056239] [INFO ] [REMOTE ] MGR: Full fetch complete (6047 records). [2025-09-16T19:23:05.056818] [DEBUG] [VALIDATOR ] MGR: Validating primary data. [2025-09-16T19:23:05.176077] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-05. [2025-09-16T19:23:05.289036] [DEBUG] [LOCAL ] MGM: Primary data saved (39964 records). [2025-09-16T19:23:05.328757] [INFO ] [MAIN ] MGM: Full history saved. [2025-09-16T19:23:05.340138] [INFO ] [MAIN ] Update Progress: 1348/2369 symbols completed. MGM took 35.2s. Estimated remaining: 1h 48m 20s. Rate: 9.4 symbols/min [2025-09-16T19:23:05.567846] [DEBUG] [VALIDATOR ] MGR: All expected trading days are present. [2025-09-16T19:23:05.578173] [DEBUG] [VALIDATOR ] MGR: Primary data validation passed. [2025-09-16T19:23:05.598710] [INFO ] [MAIN ] MGR: Fetched primary data is valid. [2025-09-16T19:23:05.611998] [DEBUG] [LOCAL ] MGR: Saving primary data. [2025-09-16T19:23:05.630572] [INFO ] [MAIN ] MH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:23:05.631137] [DEBUG] [LOCAL ] MGR: Trimmed 6047 records to 4160 (keeping 150 days of data) [2025-09-16T19:23:05.631236] [DEBUG] [LOCAL ] MH: No existing data file found in primary storage. [2025-09-16T19:23:05.631598] [WARN ] [MAIN ] MH: No existing data found, performing full refresh instead. [2025-09-16T19:23:05.632002] [INFO ] [MAIN ] MH: Wiping existing primary data before full refresh. [2025-09-16T19:23:05.632074] [INFO ] [LOCAL ] MH: Starting deletion of primary data. [2025-09-16T19:23:05.632131] [DEBUG] [LOCAL ] MH: No primary data file found to delete. [2025-09-16T19:23:05.644678] [INFO ] [MAIN ] MH: Starting full primary refresh. [2025-09-16T19:23:05.769612] [DEBUG] [LOCAL ] MGR: Primary data saved (4160 records). [2025-09-16T19:23:05.785330] [INFO ] [MAIN ] MGR: Full history saved. [2025-09-16T19:23:05.785475] [INFO ] [MAIN ] Update Progress: 1349/2369 symbols completed. MGR took 33.6s. Estimated remaining: 1h 48m 10s. Rate: 9.4 symbols/min [2025-09-16T19:23:05.981172] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-06. [2025-09-16T19:23:06.148371] [INFO ] [MAIN ] MHD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:23:06.204027] [INFO ] [REMOTE ] MH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:23:06.214394] [DEBUG] [LOCAL ] MHD: Loaded 8707 existing records from primary storage. [2025-09-16T19:23:06.214545] [INFO ] [MAIN ] MHD: Loaded 8707 existing records. [2025-09-16T19:23:06.393307] [WARN ] [MAIN ] MHD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:23:06.393571] [INFO ] [MAIN ] MHD: Wiping existing primary data before full refresh. [2025-09-16T19:23:06.393618] [INFO ] [LOCAL ] MHD: Starting deletion of primary data. [2025-09-16T19:23:06.393991] [DEBUG] [LOCAL ] MHD: Deleted primary data file. [2025-09-16T19:23:06.394062] [INFO ] [MAIN ] MHD: Starting full primary refresh. [2025-09-16T19:23:06.766675] [INFO ] [REMOTE ] MHD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:23:06.774852] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-02. [2025-09-16T19:23:06.907685] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T19:23:06.932502] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-09-16T19:23:07.297381] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-09-16T19:23:07.574918] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-03. [2025-09-16T19:23:08.372300] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T19:23:08.375139] [INFO ] [REMOTE ] MH: Fetching primary data for 2025-02. [2025-09-16T19:23:08.577257] [DEBUG] [REMOTE ] MH: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:08.577632] [WARN ] [REMOTE ] MH: No data returned for month 2025-02 [2025-09-16T19:23:09.174885] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-07. [2025-09-16T19:23:09.877325] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-09-16T19:23:09.974985] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-08. [2025-09-16T19:23:10.676417] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-09-16T19:23:10.774907] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-04. [2025-09-16T19:23:11.574926] [INFO ] [REMOTE ] MH: Fetching primary data for 2025-03. [2025-09-16T19:23:11.727403] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-09-16T19:23:11.786104] [DEBUG] [REMOTE ] MH: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:11.786445] [WARN ] [REMOTE ] MH: No data returned for month 2025-03 [2025-09-16T19:23:12.374902] [INFO ] [REMOTE ] MH: Fetching primary data for 2025-04. [2025-09-16T19:23:12.572574] [DEBUG] [REMOTE ] MH: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:12.572933] [WARN ] [REMOTE ] MH: No data returned for month 2025-04 [2025-09-16T19:23:13.174914] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-09. [2025-09-16T19:23:13.966351] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-09-16T19:23:13.967791] [INFO ] [REMOTE ] MGRE: Full fetch complete (7158 records). [2025-09-16T19:23:13.967872] [DEBUG] [VALIDATOR ] MGRE: Validating primary data. [2025-09-16T19:23:13.979971] [INFO ] [REMOTE ] MH: Fetching primary data for 2025-05. [2025-09-16T19:23:14.234922] [DEBUG] [REMOTE ] MH: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:14.261459] [WARN ] [REMOTE ] MH: No data returned for month 2025-05 [2025-09-16T19:23:14.344280] [DEBUG] [VALIDATOR ] MGRE: All expected trading days are present. [2025-09-16T19:23:14.344365] [DEBUG] [VALIDATOR ] MGRE: Primary data validation passed. [2025-09-16T19:23:14.344572] [INFO ] [MAIN ] MGRE: Fetched primary data is valid. [2025-09-16T19:23:14.344615] [DEBUG] [LOCAL ] MGRE: Saving primary data. [2025-09-16T19:23:14.345043] [DEBUG] [LOCAL ] MGRE: Trimmed 7158 records to 4916 (keeping 150 days of data) [2025-09-16T19:23:14.373933] [DEBUG] [LOCAL ] MGRE: Primary data saved (4916 records). [2025-09-16T19:23:14.374010] [INFO ] [MAIN ] MGRE: Full history saved. [2025-09-16T19:23:14.374106] [INFO ] [MAIN ] Update Progress: 1350/2369 symbols completed. MGRE took 22.0s. Estimated remaining: 1h 48m 5s. Rate: 9.4 symbols/min [2025-09-16T19:23:14.551769] [INFO ] [MAIN ] MHF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:23:14.559812] [DEBUG] [LOCAL ] MHF: Loaded 6816 existing records from primary storage. [2025-09-16T19:23:14.559916] [INFO ] [MAIN ] MHF: Loaded 6816 existing records. [2025-09-16T19:23:14.749932] [WARN ] [MAIN ] MHF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:23:14.750183] [INFO ] [MAIN ] MHF: Wiping existing primary data before full refresh. [2025-09-16T19:23:14.750231] [INFO ] [LOCAL ] MHF: Starting deletion of primary data. [2025-09-16T19:23:14.750556] [DEBUG] [LOCAL ] MHF: Deleted primary data file. [2025-09-16T19:23:14.750602] [INFO ] [MAIN ] MHF: Starting full primary refresh. [2025-09-16T19:23:14.780066] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-05. [2025-09-16T19:23:15.109266] [INFO ] [REMOTE ] MHF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:23:15.574902] [INFO ] [REMOTE ] MH: Fetching primary data for 2025-06. [2025-09-16T19:23:15.660084] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-09-16T19:23:15.768553] [DEBUG] [REMOTE ] MH: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:15.768883] [WARN ] [REMOTE ] MH: No data returned for month 2025-06 [2025-09-16T19:23:16.374916] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-06. [2025-09-16T19:23:17.174914] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-06. [2025-09-16T19:23:17.974881] [INFO ] [REMOTE ] MH: Fetching primary data for 2025-07. [2025-09-16T19:23:18.046387] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-09-16T19:23:18.057586] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-09-16T19:23:18.572405] [INFO ] [REMOTE ] MH: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-09-16T19:23:18.774911] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-07. [2025-09-16T19:23:19.359145] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-09-16T19:23:19.574886] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-07. [2025-09-16T19:23:20.374978] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-02. [2025-09-16T19:23:21.149922] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-09-16T19:23:21.174881] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-03. [2025-09-16T19:23:21.331104] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T19:23:21.888150] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-09-16T19:23:21.974862] [INFO ] [REMOTE ] MH: Fetching primary data for 2025-08. [2025-09-16T19:23:22.774896] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-08. [2025-09-16T19:23:23.480122] [INFO ] [REMOTE ] MH: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T19:23:23.574873] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-08. [2025-09-16T19:23:24.376998] [INFO ] [REMOTE ] MH: Fetching primary data for 2025-09. [2025-09-16T19:23:24.383316] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-09-16T19:23:24.403797] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-09-16T19:23:25.174910] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-04. [2025-09-16T19:23:25.432121] [INFO ] [REMOTE ] MH: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-09-16T19:23:25.436772] [INFO ] [REMOTE ] MH: Full fetch complete (10717 records). [2025-09-16T19:23:25.436892] [DEBUG] [VALIDATOR ] MH: Validating primary data. [2025-09-16T19:23:25.617197] [WARN ] [VALIDATOR ] MH: Validation failed - insufficient historical data. Have 38 unique days, need 150 days [2025-09-16T19:23:25.617438] [FATAL] [MAIN ] MH: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:23:25.617486] [WARN ] [MAIN ] MH: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:23:25.617531] [INFO ] [LOCAL ] MH: Starting deletion of primary data. [2025-09-16T19:23:25.617587] [DEBUG] [LOCAL ] MH: No primary data file found to delete. [2025-09-16T19:23:25.617622] [INFO ] [LOCAL ] MH: Starting deletion of derivative data. [2025-09-16T19:23:25.617704] [DEBUG] [LOCAL ] MH: No derivative data file found to delete. [2025-09-16T19:23:25.617766] [INFO ] [MAIN ] Update Progress: 1351/2369 symbols completed. MH took 20.3s. Estimated remaining: 1h 48m 2s. Rate: 9.4 symbols/min [2025-09-16T19:23:25.787681] [INFO ] [MAIN ] MHK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:23:25.818379] [DEBUG] [LOCAL ] MHK: Loaded 31379 existing records from primary storage. [2025-09-16T19:23:25.818518] [INFO ] [MAIN ] MHK: Loaded 31379 existing records. [2025-09-16T19:23:25.964181] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-09-16T19:23:25.995799] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-09. [2025-09-16T19:23:26.002014] [WARN ] [MAIN ] MHK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:23:26.002854] [INFO ] [MAIN ] MHK: Wiping existing primary data before full refresh. [2025-09-16T19:23:26.003041] [INFO ] [LOCAL ] MHK: Starting deletion of primary data. [2025-09-16T19:23:26.004279] [DEBUG] [LOCAL ] MHK: Deleted primary data file. [2025-09-16T19:23:26.004330] [INFO ] [MAIN ] MHK: Starting full primary refresh. [2025-09-16T19:23:26.359598] [INFO ] [REMOTE ] MHK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:23:26.774874] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-05. [2025-09-16T19:23:26.967354] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T19:23:26.969603] [INFO ] [REMOTE ] MHD: Full fetch complete (13289 records). [2025-09-16T19:23:26.969693] [DEBUG] [VALIDATOR ] MHD: Validating primary data. [2025-09-16T19:23:27.251542] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-09-16T19:23:27.338248] [DEBUG] [VALIDATOR ] MHD: All expected trading days are present. [2025-09-16T19:23:27.338385] [DEBUG] [VALIDATOR ] MHD: Primary data validation passed. [2025-09-16T19:23:27.338700] [INFO ] [MAIN ] MHD: Fetched primary data is valid. [2025-09-16T19:23:27.338742] [DEBUG] [LOCAL ] MHD: Saving primary data. [2025-09-16T19:23:27.339244] [DEBUG] [LOCAL ] MHD: Trimmed 13289 records to 8795 (keeping 150 days of data) [2025-09-16T19:23:27.393606] [DEBUG] [LOCAL ] MHD: Primary data saved (8795 records). [2025-09-16T19:23:27.393752] [INFO ] [MAIN ] MHD: Full history saved. [2025-09-16T19:23:27.393838] [INFO ] [MAIN ] Update Progress: 1352/2369 symbols completed. MHD took 21.6s. Estimated remaining: 1h 47m 52s. Rate: 9.4 symbols/min [2025-09-16T19:23:27.560904] [INFO ] [MAIN ] MHLA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:23:27.561100] [DEBUG] [LOCAL ] MHLA: No existing data file found in primary storage. [2025-09-16T19:23:27.561136] [WARN ] [MAIN ] MHLA: No existing data found, performing full refresh instead. [2025-09-16T19:23:27.561290] [INFO ] [MAIN ] MHLA: Wiping existing primary data before full refresh. [2025-09-16T19:23:27.561329] [INFO ] [LOCAL ] MHLA: Starting deletion of primary data. [2025-09-16T19:23:27.561368] [DEBUG] [LOCAL ] MHLA: No primary data file found to delete. [2025-09-16T19:23:27.561400] [INFO ] [MAIN ] MHLA: Starting full primary refresh. [2025-09-16T19:23:27.581484] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-02. [2025-09-16T19:23:27.910926] [INFO ] [REMOTE ] MHLA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:23:28.374904] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-09. [2025-09-16T19:23:29.174876] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-02. [2025-09-16T19:23:29.245752] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-09-16T19:23:29.497913] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-09-16T19:23:29.974889] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-03. [2025-09-16T19:23:30.054102] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-09-16T19:23:30.060452] [INFO ] [REMOTE ] MGY: Full fetch complete (57000 records). [2025-09-16T19:23:30.060565] [DEBUG] [VALIDATOR ] MGY: Validating primary data. [2025-09-16T19:23:30.245107] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-03 (0.27 seconds). [2025-09-16T19:23:30.465078] [DEBUG] [VALIDATOR ] MGY: All expected trading days are present. [2025-09-16T19:23:30.465169] [DEBUG] [VALIDATOR ] MGY: Primary data validation passed. [2025-09-16T19:23:30.466101] [INFO ] [MAIN ] MGY: Fetched primary data is valid. [2025-09-16T19:23:30.466149] [DEBUG] [LOCAL ] MGY: Saving primary data. [2025-09-16T19:23:30.467462] [DEBUG] [LOCAL ] MGY: Trimmed 57000 records to 39012 (keeping 150 days of data) [2025-09-16T19:23:30.704426] [DEBUG] [LOCAL ] MGY: Primary data saved (39012 records). [2025-09-16T19:23:30.704571] [INFO ] [MAIN ] MGY: Full history saved. [2025-09-16T19:23:30.705573] [INFO ] [MAIN ] Update Progress: 1353/2369 symbols completed. MGY took 36.6s. Estimated remaining: 1h 47m 44s. Rate: 9.4 symbols/min [2025-09-16T19:23:30.780040] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-03. [2025-09-16T19:23:30.885635] [INFO ] [MAIN ] MHN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:23:30.894504] [DEBUG] [LOCAL ] MHN: Loaded 5513 existing records from primary storage. [2025-09-16T19:23:30.894629] [INFO ] [MAIN ] MHN: Loaded 5513 existing records. [2025-09-16T19:23:31.077706] [WARN ] [MAIN ] MHN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:23:31.077945] [INFO ] [MAIN ] MHN: Wiping existing primary data before full refresh. [2025-09-16T19:23:31.077995] [INFO ] [LOCAL ] MHN: Starting deletion of primary data. [2025-09-16T19:23:31.078264] [DEBUG] [LOCAL ] MHN: Deleted primary data file. [2025-09-16T19:23:31.078313] [INFO ] [MAIN ] MHN: Starting full primary refresh. [2025-09-16T19:23:31.442171] [INFO ] [REMOTE ] MHN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:23:31.574898] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-06. [2025-09-16T19:23:32.221427] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-09-16T19:23:32.357784] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T19:23:32.374843] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-04. [2025-09-16T19:23:33.174896] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-04. [2025-09-16T19:23:33.602566] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-09-16T19:23:33.979958] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-05. [2025-09-16T19:23:33.996872] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-09-16T19:23:34.509683] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-09-16T19:23:34.774892] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-02. [2025-09-16T19:23:35.572572] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-09-16T19:23:35.574991] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-03. [2025-09-16T19:23:36.306468] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-09-16T19:23:36.374868] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-04. [2025-09-16T19:23:37.174932] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-05. [2025-09-16T19:23:37.209495] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-09-16T19:23:37.974917] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-06. [2025-09-16T19:23:38.307953] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-09-16T19:23:38.765036] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T19:23:38.774839] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-06. [2025-09-16T19:23:39.574894] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-05. [2025-09-16T19:23:40.125891] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-09-16T19:23:40.277593] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-09-16T19:23:40.374938] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-06. [2025-09-16T19:23:40.859625] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-09-16T19:23:41.174919] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-07. [2025-09-16T19:23:41.923204] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-09-16T19:23:41.974890] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-08. [2025-09-16T19:23:42.763058] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T19:23:42.774848] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-09. [2025-09-16T19:23:43.500040] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T19:23:43.501629] [INFO ] [REMOTE ] MHN: Full fetch complete (8344 records). [2025-09-16T19:23:43.501713] [DEBUG] [VALIDATOR ] MHN: Validating primary data. [2025-09-16T19:23:43.580093] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-07. [2025-09-16T19:23:43.863470] [DEBUG] [VALIDATOR ] MHN: All expected trading days are present. [2025-09-16T19:23:43.863546] [DEBUG] [VALIDATOR ] MHN: Primary data validation passed. [2025-09-16T19:23:43.863791] [INFO ] [MAIN ] MHN: Fetched primary data is valid. [2025-09-16T19:23:43.863836] [DEBUG] [LOCAL ] MHN: Saving primary data. [2025-09-16T19:23:43.864263] [DEBUG] [LOCAL ] MHN: Trimmed 8344 records to 5615 (keeping 150 days of data) [2025-09-16T19:23:43.898156] [DEBUG] [LOCAL ] MHN: Primary data saved (5615 records). [2025-09-16T19:23:43.898245] [INFO ] [MAIN ] MHN: Full history saved. [2025-09-16T19:23:43.898319] [INFO ] [MAIN ] Update Progress: 1354/2369 symbols completed. MHN took 13.2s. Estimated remaining: 1h 47m 42s. Rate: 9.4 symbols/min [2025-09-16T19:23:44.067505] [INFO ] [MAIN ] MHO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:23:44.085724] [DEBUG] [LOCAL ] MHO: Loaded 17637 existing records from primary storage. [2025-09-16T19:23:44.085844] [INFO ] [MAIN ] MHO: Loaded 17637 existing records. [2025-09-16T19:23:44.262988] [WARN ] [MAIN ] MHO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:23:44.263238] [INFO ] [MAIN ] MHO: Wiping existing primary data before full refresh. [2025-09-16T19:23:44.263284] [INFO ] [LOCAL ] MHO: Starting deletion of primary data. [2025-09-16T19:23:44.263844] [DEBUG] [LOCAL ] MHO: Deleted primary data file. [2025-09-16T19:23:44.263892] [INFO ] [MAIN ] MHO: Starting full primary refresh. [2025-09-16T19:23:44.380162] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-07. [2025-09-16T19:23:44.617005] [INFO ] [REMOTE ] MHO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:23:44.659465] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-07 (0.28 seconds). [2025-09-16T19:23:45.174898] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-08. [2025-09-16T19:23:45.304936] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-09-16T19:23:45.506612] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-09-16T19:23:45.974950] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-09. [2025-09-16T19:23:46.310240] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-09 (0.34 seconds). [2025-09-16T19:23:46.310921] [INFO ] [REMOTE ] MHLA: Full fetch complete (1157 records). [2025-09-16T19:23:46.310986] [DEBUG] [VALIDATOR ] MHLA: Validating primary data. [2025-09-16T19:23:46.489714] [WARN ] [VALIDATOR ] MHLA: Validation failed - insufficient historical data. Have 133 unique days, need 150 days [2025-09-16T19:23:46.489893] [FATAL] [MAIN ] MHLA: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:23:46.489964] [WARN ] [MAIN ] MHLA: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:23:46.490034] [INFO ] [LOCAL ] MHLA: Starting deletion of primary data. [2025-09-16T19:23:46.490100] [DEBUG] [LOCAL ] MHLA: No primary data file found to delete. [2025-09-16T19:23:46.490137] [INFO ] [LOCAL ] MHLA: Starting deletion of derivative data. [2025-09-16T19:23:46.490177] [DEBUG] [LOCAL ] MHLA: No derivative data file found to delete. [2025-09-16T19:23:46.490248] [INFO ] [MAIN ] Update Progress: 1355/2369 symbols completed. MHLA took 19.1s. Estimated remaining: 1h 47m 33s. Rate: 9.4 symbols/min [2025-09-16T19:23:46.661555] [INFO ] [MAIN ] MIAX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:23:46.661741] [DEBUG] [LOCAL ] MIAX: No existing data file found in primary storage. [2025-09-16T19:23:46.661781] [WARN ] [MAIN ] MIAX: No existing data found, performing full refresh instead. [2025-09-16T19:23:46.661950] [INFO ] [MAIN ] MIAX: Wiping existing primary data before full refresh. [2025-09-16T19:23:46.662015] [INFO ] [LOCAL ] MIAX: Starting deletion of primary data. [2025-09-16T19:23:46.662062] [DEBUG] [LOCAL ] MIAX: No primary data file found to delete. [2025-09-16T19:23:46.662096] [INFO ] [MAIN ] MIAX: Starting full primary refresh. [2025-09-16T19:23:46.780039] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-02. [2025-09-16T19:23:47.025406] [INFO ] [REMOTE ] MIAX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:23:47.574897] [INFO ] [REMOTE ] MIAX: Fetching primary data for 2025-02. [2025-09-16T19:23:47.761295] [DEBUG] [REMOTE ] MIAX: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:47.761571] [WARN ] [REMOTE ] MIAX: No data returned for month 2025-02 [2025-09-16T19:23:48.085923] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T19:23:48.374914] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-03. [2025-09-16T19:23:49.174896] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-08. [2025-09-16T19:23:49.278897] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-09-16T19:23:49.974938] [INFO ] [REMOTE ] MIAX: Fetching primary data for 2025-03. [2025-09-16T19:23:50.164308] [DEBUG] [REMOTE ] MIAX: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:50.164620] [WARN ] [REMOTE ] MIAX: No data returned for month 2025-03 [2025-09-16T19:23:50.650678] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-09-16T19:23:50.774917] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-09. [2025-09-16T19:23:51.574941] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-07. [2025-09-16T19:23:52.216444] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-09-16T19:23:52.221569] [INFO ] [REMOTE ] MHK: Full fetch complete (46077 records). [2025-09-16T19:23:52.221710] [DEBUG] [VALIDATOR ] MHK: Validating primary data. [2025-09-16T19:23:52.376409] [INFO ] [REMOTE ] MIAX: Fetching primary data for 2025-04. [2025-09-16T19:23:52.526031] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-09-16T19:23:52.619347] [DEBUG] [VALIDATOR ] MHK: All expected trading days are present. [2025-09-16T19:23:52.619452] [DEBUG] [VALIDATOR ] MHK: Primary data validation passed. [2025-09-16T19:23:52.620310] [INFO ] [MAIN ] MHK: Fetched primary data is valid. [2025-09-16T19:23:52.620389] [DEBUG] [LOCAL ] MHK: Saving primary data. [2025-09-16T19:23:52.621609] [DEBUG] [LOCAL ] MHK: Trimmed 46077 records to 31599 (keeping 150 days of data) [2025-09-16T19:23:52.676974] [DEBUG] [REMOTE ] MIAX: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:52.713041] [WARN ] [REMOTE ] MIAX: No data returned for month 2025-04 [2025-09-16T19:23:52.818208] [DEBUG] [LOCAL ] MHK: Primary data saved (31599 records). [2025-09-16T19:23:52.818382] [INFO ] [MAIN ] MHK: Full history saved. [2025-09-16T19:23:52.818481] [INFO ] [MAIN ] Update Progress: 1356/2369 symbols completed. MHK took 27.2s. Estimated remaining: 1h 47m 27s. Rate: 9.4 symbols/min [2025-09-16T19:23:52.985252] [INFO ] [MAIN ] MIN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:23:52.995624] [DEBUG] [LOCAL ] MIN: Loaded 10623 existing records from primary storage. [2025-09-16T19:23:52.995769] [INFO ] [MAIN ] MIN: Loaded 10623 existing records. [2025-09-16T19:23:53.167670] [WARN ] [MAIN ] MIN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:23:53.167870] [INFO ] [MAIN ] MIN: Wiping existing primary data before full refresh. [2025-09-16T19:23:53.167914] [INFO ] [LOCAL ] MIN: Starting deletion of primary data. [2025-09-16T19:23:53.168293] [DEBUG] [LOCAL ] MIN: Deleted primary data file. [2025-09-16T19:23:53.168338] [INFO ] [MAIN ] MIN: Starting full primary refresh. [2025-09-16T19:23:53.180216] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-04. [2025-09-16T19:23:53.548910] [INFO ] [REMOTE ] MIN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:23:53.974958] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-08. [2025-09-16T19:23:54.395323] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-09-16T19:23:54.762686] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T19:23:54.774850] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-09. [2025-09-16T19:23:55.572932] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T19:23:55.574716] [INFO ] [REMOTE ] MHF: Full fetch complete (9397 records). [2025-09-16T19:23:55.574785] [DEBUG] [VALIDATOR ] MHF: Validating primary data. [2025-09-16T19:23:55.574893] [INFO ] [REMOTE ] MIAX: Fetching primary data for 2025-05. [2025-09-16T19:23:55.860310] [DEBUG] [REMOTE ] MIAX: Error fetching primary data for 2025-05: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:55.863011] [WARN ] [REMOTE ] MIAX: No data returned for month 2025-05 [2025-09-16T19:23:55.943269] [DEBUG] [VALIDATOR ] MHF: All expected trading days are present. [2025-09-16T19:23:55.943364] [DEBUG] [VALIDATOR ] MHF: Primary data validation passed. [2025-09-16T19:23:55.943690] [INFO ] [MAIN ] MHF: Fetched primary data is valid. [2025-09-16T19:23:55.943736] [DEBUG] [LOCAL ] MHF: Saving primary data. [2025-09-16T19:23:55.944323] [DEBUG] [LOCAL ] MHF: Trimmed 9397 records to 6905 (keeping 150 days of data) [2025-09-16T19:23:55.987779] [DEBUG] [LOCAL ] MHF: Primary data saved (6905 records). [2025-09-16T19:23:55.987945] [INFO ] [MAIN ] MHF: Full history saved. [2025-09-16T19:23:55.988053] [INFO ] [MAIN ] Update Progress: 1357/2369 symbols completed. MHF took 41.6s. Estimated remaining: 1h 47m 18s. Rate: 9.4 symbols/min [2025-09-16T19:23:56.374906] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-05. [2025-09-16T19:23:57.174928] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-02. [2025-09-16T19:23:57.245215] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-09-16T19:23:57.937673] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-09-16T19:23:57.974867] [INFO ] [REMOTE ] MIAX: Fetching primary data for 2025-06. [2025-09-16T19:23:58.158284] [DEBUG] [REMOTE ] MIAX: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:23:58.158628] [WARN ] [REMOTE ] MIAX: No data returned for month 2025-06 [2025-09-16T19:23:58.774959] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-03. [2025-09-16T19:23:59.528283] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-09-16T19:23:59.574872] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-04. [2025-09-16T19:24:00.374947] [INFO ] [REMOTE ] MIAX: Fetching primary data for 2025-07. [2025-09-16T19:24:00.403750] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-09-16T19:24:00.569561] [DEBUG] [REMOTE ] MIAX: Error fetching primary data for 2025-07: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:24:00.569911] [WARN ] [REMOTE ] MIAX: No data returned for month 2025-07 [2025-09-16T19:24:01.174906] [INFO ] [REMOTE ] MIAX: Fetching primary data for 2025-08. [2025-09-16T19:24:01.974886] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-06. [2025-09-16T19:24:02.379202] [INFO ] [REMOTE ] MIAX: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-09-16T19:24:02.774932] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-05. [2025-09-16T19:24:03.076833] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-09-16T19:24:03.411774] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-09-16T19:24:03.574899] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-06. [2025-09-16T19:24:04.374955] [INFO ] [REMOTE ] MIAX: Fetching primary data for 2025-09. [2025-09-16T19:24:04.467955] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-09-16T19:24:05.174922] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-07. [2025-09-16T19:24:05.653815] [INFO ] [REMOTE ] MIAX: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-09-16T19:24:05.658341] [INFO ] [REMOTE ] MIAX: Full fetch complete (7082 records). [2025-09-16T19:24:05.658430] [DEBUG] [VALIDATOR ] MIAX: Validating primary data. [2025-09-16T19:24:05.838196] [WARN ] [VALIDATOR ] MIAX: Validation failed - insufficient historical data. Have 23 unique days, need 150 days [2025-09-16T19:24:05.838423] [FATAL] [MAIN ] MIAX: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:24:05.838492] [WARN ] [MAIN ] MIAX: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:24:05.838545] [INFO ] [LOCAL ] MIAX: Starting deletion of primary data. [2025-09-16T19:24:05.838622] [DEBUG] [LOCAL ] MIAX: No primary data file found to delete. [2025-09-16T19:24:05.838705] [INFO ] [LOCAL ] MIAX: Starting deletion of derivative data. [2025-09-16T19:24:05.838767] [DEBUG] [LOCAL ] MIAX: No derivative data file found to delete. [2025-09-16T19:24:05.838861] [INFO ] [MAIN ] Update Progress: 1358/2369 symbols completed. MIAX took 19.3s. Estimated remaining: 1h 47m 14s. Rate: 9.4 symbols/min [2025-09-16T19:24:05.898483] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-09-16T19:24:05.974916] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-07. [2025-09-16T19:24:06.774931] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-08. [2025-09-16T19:24:07.299215] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-09-16T19:24:07.401835] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-09-16T19:24:07.574911] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-09. [2025-09-16T19:24:08.374929] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-08. [2025-09-16T19:24:08.503169] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-09-16T19:24:08.505030] [INFO ] [REMOTE ] MIN: Full fetch complete (13926 records). [2025-09-16T19:24:08.505101] [DEBUG] [VALIDATOR ] MIN: Validating primary data. [2025-09-16T19:24:08.865096] [DEBUG] [VALIDATOR ] MIN: All expected trading days are present. [2025-09-16T19:24:08.865171] [DEBUG] [VALIDATOR ] MIN: Primary data validation passed. [2025-09-16T19:24:08.865490] [INFO ] [MAIN ] MIN: Fetched primary data is valid. [2025-09-16T19:24:08.865537] [DEBUG] [LOCAL ] MIN: Saving primary data. [2025-09-16T19:24:08.866035] [DEBUG] [LOCAL ] MIN: Trimmed 13926 records to 10688 (keeping 150 days of data) [2025-09-16T19:24:08.928512] [DEBUG] [LOCAL ] MIN: Primary data saved (10688 records). [2025-09-16T19:24:08.928662] [INFO ] [MAIN ] MIN: Full history saved. [2025-09-16T19:24:08.928751] [INFO ] [MAIN ] Update Progress: 1359/2369 symbols completed. MIN took 16.1s. Estimated remaining: 1h 47m 6s. Rate: 9.4 symbols/min [2025-09-16T19:24:09.500753] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T19:24:09.506701] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-09. [2025-09-16T19:24:10.601461] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T19:24:10.604588] [INFO ] [REMOTE ] MHO: Full fetch complete (26988 records). [2025-09-16T19:24:10.604706] [DEBUG] [VALIDATOR ] MHO: Validating primary data. [2025-09-16T19:24:10.971370] [DEBUG] [VALIDATOR ] MHO: All expected trading days are present. [2025-09-16T19:24:10.971459] [DEBUG] [VALIDATOR ] MHO: Primary data validation passed. [2025-09-16T19:24:10.972037] [INFO ] [MAIN ] MHO: Fetched primary data is valid. [2025-09-16T19:24:10.972080] [DEBUG] [LOCAL ] MHO: Saving primary data. [2025-09-16T19:24:10.972793] [DEBUG] [LOCAL ] MHO: Trimmed 26988 records to 17852 (keeping 150 days of data) [2025-09-16T19:24:11.081220] [DEBUG] [LOCAL ] MHO: Primary data saved (17852 records). [2025-09-16T19:24:11.081371] [INFO ] [MAIN ] MHO: Full history saved. [2025-09-16T19:24:11.081465] [INFO ] [MAIN ] Update Progress: 1360/2369 symbols completed. MHO took 27.2s. Estimated remaining: 1h 46m 56s. Rate: 9.4 symbols/min [2025-09-16T19:24:11.669253] [INFO ] [MAIN ] MITP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:24:11.737826] [INFO ] [MAIN ] MITT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:24:11.773339] [INFO ] [MAIN ] MIR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:24:11.830882] [INFO ] [MAIN ] MITN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:24:11.831394] [DEBUG] [LOCAL ] MITN: No existing data file found in primary storage. [2025-09-16T19:24:11.831285] [DEBUG] [LOCAL ] MITP: Loaded 1616 existing records from primary storage. [2025-09-16T19:24:11.831537] [WARN ] [MAIN ] MITN: No existing data found, performing full refresh instead. [2025-09-16T19:24:11.831850] [INFO ] [MAIN ] MITN: Wiping existing primary data before full refresh. [2025-09-16T19:24:11.831899] [INFO ] [LOCAL ] MITN: Starting deletion of primary data. [2025-09-16T19:24:11.831961] [DEBUG] [LOCAL ] MITN: No primary data file found to delete. [2025-09-16T19:24:11.831999] [INFO ] [MAIN ] MITN: Starting full primary refresh. [2025-09-16T19:24:11.831669] [INFO ] [MAIN ] MITP: Loaded 1616 existing records. [2025-09-16T19:24:12.110819] [DEBUG] [LOCAL ] MITT: Loaded 15605 existing records from primary storage. [2025-09-16T19:24:12.158019] [DEBUG] [LOCAL ] MIR: Loaded 37755 existing records from primary storage. [2025-09-16T19:24:12.229285] [INFO ] [MAIN ] MITT: Loaded 15605 existing records. [2025-09-16T19:24:12.249351] [WARN ] [MAIN ] MITP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:24:12.249487] [INFO ] [MAIN ] MIR: Loaded 37755 existing records. [2025-09-16T19:24:12.328504] [INFO ] [MAIN ] MITP: Wiping existing primary data before full refresh. [2025-09-16T19:24:12.343573] [INFO ] [LOCAL ] MITP: Starting deletion of primary data. [2025-09-16T19:24:12.424730] [DEBUG] [LOCAL ] MITP: Deleted primary data file. [2025-09-16T19:24:12.492221] [INFO ] [MAIN ] MITP: Starting full primary refresh. [2025-09-16T19:24:12.740920] [WARN ] [MAIN ] MITT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:24:12.873318] [INFO ] [REMOTE ] MITN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:24:12.904111] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-02. [2025-09-16T19:24:12.898005] [INFO ] [MAIN ] MITT: Wiping existing primary data before full refresh. [2025-09-16T19:24:12.940049] [INFO ] [LOCAL ] MITT: Starting deletion of primary data. [2025-09-16T19:24:12.958699] [DEBUG] [LOCAL ] MITT: Deleted primary data file. [2025-09-16T19:24:12.962274] [WARN ] [MAIN ] MIR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:24:12.986950] [INFO ] [MAIN ] MITT: Starting full primary refresh. [2025-09-16T19:24:12.990340] [INFO ] [MAIN ] MIR: Wiping existing primary data before full refresh. [2025-09-16T19:24:13.001550] [INFO ] [LOCAL ] MIR: Starting deletion of primary data. [2025-09-16T19:24:13.026588] [DEBUG] [LOCAL ] MIR: Deleted primary data file. [2025-09-16T19:24:13.038811] [INFO ] [MAIN ] MIR: Starting full primary refresh. [2025-09-16T19:24:13.410848] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-09-16T19:24:13.455610] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-03. [2025-09-16T19:24:13.818365] [INFO ] [REMOTE ] MITP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:24:13.862079] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-02. [2025-09-16T19:24:13.943478] [INFO ] [REMOTE ] MITT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:24:13.988311] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-02. [2025-09-16T19:24:14.011995] [INFO ] [REMOTE ] MIR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:24:14.012519] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-02. [2025-09-16T19:24:14.117620] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-09-16T19:24:14.119328] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-04. [2025-09-16T19:24:14.370866] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-09-16T19:24:14.670765] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-09-16T19:24:14.774919] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-03. [2025-09-16T19:24:15.158539] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-09-16T19:24:15.175093] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-09-16T19:24:15.558239] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T19:24:15.574847] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-03. [2025-09-16T19:24:16.374924] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-05. [2025-09-16T19:24:16.788794] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-09-16T19:24:17.066300] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-09-16T19:24:17.174875] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-06. [2025-09-16T19:24:17.508762] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-09-16T19:24:17.974967] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-04. [2025-09-16T19:24:18.774923] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-03. [2025-09-16T19:24:18.909112] [DEBUG] [REMOTE ] MITT: Error fetching primary data for 2025-03: 'API call returned an empty response.' [2025-09-16T19:24:18.909484] [WARN ] [REMOTE ] MITT: No data returned for month 2025-03 [2025-09-16T19:24:19.574911] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-04. [2025-09-16T19:24:19.688578] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-09-16T19:24:19.930907] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-09-16T19:24:20.374947] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-04. [2025-09-16T19:24:21.174913] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-07. [2025-09-16T19:24:21.499833] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-09-16T19:24:21.781862] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-09-16T19:24:21.974912] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-05. [2025-09-16T19:24:22.774914] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-08. [2025-09-16T19:24:22.964487] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-09-16T19:24:23.123387] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-09-16T19:24:23.574913] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-09. [2025-09-16T19:24:23.924604] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-09-16T19:24:23.925476] [INFO ] [REMOTE ] MITN: Full fetch complete (1933 records). [2025-09-16T19:24:23.925557] [DEBUG] [VALIDATOR ] MITN: Validating primary data. [2025-09-16T19:24:24.115054] [WARN ] [VALIDATOR ] MITN: Validation failed - insufficient historical data. Have 149 unique days, need 150 days [2025-09-16T19:24:24.115240] [FATAL] [MAIN ] MITN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:24:24.115288] [WARN ] [MAIN ] MITN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:24:24.115333] [INFO ] [LOCAL ] MITN: Starting deletion of primary data. [2025-09-16T19:24:24.115391] [DEBUG] [LOCAL ] MITN: No primary data file found to delete. [2025-09-16T19:24:24.115446] [INFO ] [LOCAL ] MITN: Starting deletion of derivative data. [2025-09-16T19:24:24.115489] [DEBUG] [LOCAL ] MITN: No derivative data file found to delete. [2025-09-16T19:24:24.115548] [INFO ] [MAIN ] Update Progress: 1361/2369 symbols completed. MITN took 13.0s. Estimated remaining: 1h 46m 55s. Rate: 9.4 symbols/min [2025-09-16T19:24:24.303383] [INFO ] [MAIN ] MIY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:24:24.310087] [DEBUG] [LOCAL ] MIY: Loaded 4318 existing records from primary storage. [2025-09-16T19:24:24.310213] [INFO ] [MAIN ] MIY: Loaded 4318 existing records. [2025-09-16T19:24:24.379987] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-05. [2025-09-16T19:24:24.494381] [WARN ] [MAIN ] MIY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:24:24.494634] [INFO ] [MAIN ] MIY: Wiping existing primary data before full refresh. [2025-09-16T19:24:24.494699] [INFO ] [LOCAL ] MIY: Starting deletion of primary data. [2025-09-16T19:24:24.494930] [DEBUG] [LOCAL ] MIY: Deleted primary data file. [2025-09-16T19:24:24.494975] [INFO ] [MAIN ] MIY: Starting full primary refresh. [2025-09-16T19:24:24.843430] [INFO ] [REMOTE ] MIY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:24:25.025330] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-09-16T19:24:25.174928] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-06. [2025-09-16T19:24:25.668165] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-09-16T19:24:25.974908] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-02. [2025-09-16T19:24:26.545987] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-09-16T19:24:26.774907] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-03. [2025-09-16T19:24:27.270170] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-09-16T19:24:27.574921] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-06. [2025-09-16T19:24:28.374893] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-04. [2025-09-16T19:24:28.677768] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-09-16T19:24:29.055751] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T19:24:29.174911] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-07. [2025-09-16T19:24:29.594703] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-09-16T19:24:29.974906] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-08. [2025-09-16T19:24:30.332824] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-09-16T19:24:30.774915] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-09. [2025-09-16T19:24:31.105174] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-09 (0.33 seconds). [2025-09-16T19:24:31.105931] [INFO ] [REMOTE ] MITP: Full fetch complete (2544 records). [2025-09-16T19:24:31.105999] [DEBUG] [VALIDATOR ] MITP: Validating primary data. [2025-09-16T19:24:31.462339] [DEBUG] [VALIDATOR ] MITP: All expected trading days are present. [2025-09-16T19:24:31.462416] [DEBUG] [VALIDATOR ] MITP: Primary data validation passed. [2025-09-16T19:24:31.462519] [INFO ] [MAIN ] MITP: Fetched primary data is valid. [2025-09-16T19:24:31.462557] [DEBUG] [LOCAL ] MITP: Saving primary data. [2025-09-16T19:24:31.462909] [DEBUG] [LOCAL ] MITP: Trimmed 2544 records to 1620 (keeping 150 days of data) [2025-09-16T19:24:31.472780] [DEBUG] [LOCAL ] MITP: Primary data saved (1620 records). [2025-09-16T19:24:31.472843] [INFO ] [MAIN ] MITP: Full history saved. [2025-09-16T19:24:31.472936] [INFO ] [MAIN ] Update Progress: 1362/2369 symbols completed. MITP took 20.4s. Estimated remaining: 1h 46m 49s. Rate: 9.4 symbols/min [2025-09-16T19:24:31.575718] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-05. [2025-09-16T19:24:31.648193] [INFO ] [MAIN ] MKC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:24:31.686876] [DEBUG] [LOCAL ] MKC: Loaded 38721 existing records from primary storage. [2025-09-16T19:24:31.687036] [INFO ] [MAIN ] MKC: Loaded 38721 existing records. [2025-09-16T19:24:31.868802] [WARN ] [MAIN ] MKC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:24:31.869047] [INFO ] [MAIN ] MKC: Wiping existing primary data before full refresh. [2025-09-16T19:24:31.869101] [INFO ] [LOCAL ] MKC: Starting deletion of primary data. [2025-09-16T19:24:31.870021] [DEBUG] [LOCAL ] MKC: Deleted primary data file. [2025-09-16T19:24:31.870089] [INFO ] [MAIN ] MKC: Starting full primary refresh. [2025-09-16T19:24:32.244574] [INFO ] [REMOTE ] MKC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:24:32.374878] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-07. [2025-09-16T19:24:33.174897] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-02. [2025-09-16T19:24:33.333669] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T19:24:33.408474] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-09-16T19:24:33.974911] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-06. [2025-09-16T19:24:34.623661] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-09-16T19:24:34.774854] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-03. [2025-09-16T19:24:35.574890] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-05. [2025-09-16T19:24:35.670896] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T19:24:36.244773] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-09-16T19:24:36.374907] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-06. [2025-09-16T19:24:36.534570] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-09-16T19:24:37.147095] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T19:24:37.174978] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-07. [2025-09-16T19:24:37.910478] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-09-16T19:24:37.974908] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-08. [2025-09-16T19:24:38.571141] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-09-16T19:24:38.774921] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-08. [2025-09-16T19:24:39.574891] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-09. [2025-09-16T19:24:39.808429] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-09-16T19:24:40.301042] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T19:24:40.302667] [INFO ] [REMOTE ] MIY: Full fetch complete (6427 records). [2025-09-16T19:24:40.302746] [DEBUG] [VALIDATOR ] MIY: Validating primary data. [2025-09-16T19:24:40.380047] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-09. [2025-09-16T19:24:40.668313] [DEBUG] [VALIDATOR ] MIY: All expected trading days are present. [2025-09-16T19:24:40.668413] [DEBUG] [VALIDATOR ] MIY: Primary data validation passed. [2025-09-16T19:24:40.668631] [INFO ] [MAIN ] MIY: Fetched primary data is valid. [2025-09-16T19:24:40.668708] [DEBUG] [LOCAL ] MIY: Saving primary data. [2025-09-16T19:24:40.669140] [DEBUG] [LOCAL ] MIY: Trimmed 6427 records to 4430 (keeping 150 days of data) [2025-09-16T19:24:40.697063] [DEBUG] [LOCAL ] MIY: Primary data saved (4430 records). [2025-09-16T19:24:40.697161] [INFO ] [MAIN ] MIY: Full history saved. [2025-09-16T19:24:40.697265] [INFO ] [MAIN ] Update Progress: 1363/2369 symbols completed. MIY took 16.6s. Estimated remaining: 1h 46m 45s. Rate: 9.4 symbols/min [2025-09-16T19:24:40.871616] [INFO ] [MAIN ] MKL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:24:40.898874] [DEBUG] [LOCAL ] MKL: Loaded 25028 existing records from primary storage. [2025-09-16T19:24:40.899024] [INFO ] [MAIN ] MKL: Loaded 25028 existing records. [2025-09-16T19:24:41.081606] [WARN ] [MAIN ] MKL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:24:41.081897] [INFO ] [MAIN ] MKL: Wiping existing primary data before full refresh. [2025-09-16T19:24:41.081962] [INFO ] [LOCAL ] MKL: Starting deletion of primary data. [2025-09-16T19:24:41.082561] [DEBUG] [LOCAL ] MKL: Deleted primary data file. [2025-09-16T19:24:41.082616] [INFO ] [MAIN ] MKL: Starting full primary refresh. [2025-09-16T19:24:41.180304] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-04. [2025-09-16T19:24:41.436968] [INFO ] [REMOTE ] MKL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:24:41.468639] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T19:24:41.471601] [INFO ] [REMOTE ] MITT: Full fetch complete (20955 records). [2025-09-16T19:24:41.471724] [DEBUG] [VALIDATOR ] MITT: Validating primary data. [2025-09-16T19:24:41.668207] [WARN ] [VALIDATOR ] MITT: Validation failed - insufficient historical data. Have 129 unique days, need 150 days [2025-09-16T19:24:41.668594] [FATAL] [MAIN ] MITT: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:24:41.668683] [WARN ] [MAIN ] MITT: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:24:41.668787] [INFO ] [LOCAL ] MITT: Starting deletion of primary data. [2025-09-16T19:24:41.668866] [DEBUG] [LOCAL ] MITT: No primary data file found to delete. [2025-09-16T19:24:41.668904] [INFO ] [LOCAL ] MITT: Starting deletion of derivative data. [2025-09-16T19:24:41.669230] [DEBUG] [LOCAL ] MITT: Deleted derivative data file. [2025-09-16T19:24:41.669349] [INFO ] [MAIN ] Update Progress: 1364/2369 symbols completed. MITT took 30.6s. Estimated remaining: 1h 46m 34s. Rate: 9.4 symbols/min [2025-09-16T19:24:41.848777] [INFO ] [MAIN ] MLI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:24:41.893348] [DEBUG] [LOCAL ] MLI: Loaded 31883 existing records from primary storage. [2025-09-16T19:24:41.893552] [INFO ] [MAIN ] MLI: Loaded 31883 existing records. [2025-09-16T19:24:41.980020] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-07. [2025-09-16T19:24:42.082161] [WARN ] [MAIN ] MLI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:24:42.082452] [INFO ] [MAIN ] MLI: Wiping existing primary data before full refresh. [2025-09-16T19:24:42.082533] [INFO ] [LOCAL ] MLI: Starting deletion of primary data. [2025-09-16T19:24:42.083309] [DEBUG] [LOCAL ] MLI: Deleted primary data file. [2025-09-16T19:24:42.083370] [INFO ] [MAIN ] MLI: Starting full primary refresh. [2025-09-16T19:24:42.453299] [INFO ] [REMOTE ] MLI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:24:42.774951] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-02. [2025-09-16T19:24:43.023697] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-09-16T19:24:43.574917] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-02. [2025-09-16T19:24:43.737441] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-09-16T19:24:44.250593] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T19:24:44.374904] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-08. [2025-09-16T19:24:44.858750] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-09-16T19:24:45.175126] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-05. [2025-09-16T19:24:45.974939] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-03. [2025-09-16T19:24:46.032476] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-09-16T19:24:46.774935] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-03. [2025-09-16T19:24:46.961926] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T19:24:47.574926] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-09. [2025-09-16T19:24:47.628587] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-09-16T19:24:48.156344] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-09-16T19:24:48.374951] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-04. [2025-09-16T19:24:48.899282] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-09-16T19:24:48.905600] [INFO ] [REMOTE ] MIR: Full fetch complete (55151 records). [2025-09-16T19:24:48.905759] [DEBUG] [VALIDATOR ] MIR: Validating primary data. [2025-09-16T19:24:49.184599] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-04. [2025-09-16T19:24:49.305098] [DEBUG] [VALIDATOR ] MIR: All expected trading days are present. [2025-09-16T19:24:49.305188] [DEBUG] [VALIDATOR ] MIR: Primary data validation passed. [2025-09-16T19:24:49.306110] [INFO ] [MAIN ] MIR: Fetched primary data is valid. [2025-09-16T19:24:49.306191] [DEBUG] [LOCAL ] MIR: Saving primary data. [2025-09-16T19:24:49.307581] [DEBUG] [LOCAL ] MIR: Trimmed 55151 records to 38141 (keeping 150 days of data) [2025-09-16T19:24:49.536274] [DEBUG] [LOCAL ] MIR: Primary data saved (38141 records). [2025-09-16T19:24:49.536413] [INFO ] [MAIN ] MIR: Full history saved. [2025-09-16T19:24:49.536517] [INFO ] [MAIN ] Update Progress: 1365/2369 symbols completed. MIR took 38.5s. Estimated remaining: 1h 46m 29s. Rate: 9.4 symbols/min [2025-09-16T19:24:49.710972] [INFO ] [MAIN ] MLM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:24:49.731930] [DEBUG] [LOCAL ] MLM: Loaded 21374 existing records from primary storage. [2025-09-16T19:24:49.732053] [INFO ] [MAIN ] MLM: Loaded 21374 existing records. [2025-09-16T19:24:49.915449] [WARN ] [MAIN ] MLM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:24:49.915769] [INFO ] [MAIN ] MLM: Wiping existing primary data before full refresh. [2025-09-16T19:24:49.915837] [INFO ] [LOCAL ] MLM: Starting deletion of primary data. [2025-09-16T19:24:49.916490] [DEBUG] [LOCAL ] MLM: Deleted primary data file. [2025-09-16T19:24:49.916539] [INFO ] [MAIN ] MLM: Starting full primary refresh. [2025-09-16T19:24:49.980423] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-06. [2025-09-16T19:24:50.386348] [INFO ] [REMOTE ] MLM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:24:50.422072] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-09-16T19:24:50.767518] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T19:24:50.778769] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-05. [2025-09-16T19:24:51.575037] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-05. [2025-09-16T19:24:51.780707] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-09-16T19:24:52.318220] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T19:24:52.374924] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-06. [2025-09-16T19:24:53.055258] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T19:24:53.175024] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-02. [2025-09-16T19:24:53.974925] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-06. [2025-09-16T19:24:54.026629] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-09-16T19:24:54.659890] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-09-16T19:24:54.774919] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-03. [2025-09-16T19:24:55.195587] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-09-16T19:24:55.574934] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-07. [2025-09-16T19:24:56.232805] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-09-16T19:24:56.374919] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-07. [2025-09-16T19:24:57.174905] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-04. [2025-09-16T19:24:57.219534] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T19:24:57.774160] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-09-16T19:24:57.974897] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-08. [2025-09-16T19:24:58.325779] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-09-16T19:24:58.774936] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-07. [2025-09-16T19:24:59.271515] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-09-16T19:24:59.574880] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-08. [2025-09-16T19:25:00.116757] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-09-16T19:25:00.374903] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-05. [2025-09-16T19:25:01.175000] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-08. [2025-09-16T19:25:01.492142] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-09-16T19:25:01.531115] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-09-16T19:25:01.974945] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-06. [2025-09-16T19:25:02.510404] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-09-16T19:25:02.774948] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-09. [2025-09-16T19:25:03.248326] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-09-16T19:25:03.574917] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-07. [2025-09-16T19:25:04.273275] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-09-16T19:25:04.278992] [INFO ] [REMOTE ] MLI: Full fetch complete (47790 records). [2025-09-16T19:25:04.279140] [DEBUG] [VALIDATOR ] MLI: Validating primary data. [2025-09-16T19:25:04.383678] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-09. [2025-09-16T19:25:04.675716] [DEBUG] [VALIDATOR ] MLI: All expected trading days are present. [2025-09-16T19:25:04.675894] [DEBUG] [VALIDATOR ] MLI: Primary data validation passed. [2025-09-16T19:25:04.676978] [INFO ] [MAIN ] MLI: Fetched primary data is valid. [2025-09-16T19:25:04.677058] [DEBUG] [LOCAL ] MLI: Saving primary data. [2025-09-16T19:25:04.678475] [DEBUG] [LOCAL ] MLI: Trimmed 47790 records to 32230 (keeping 150 days of data) [2025-09-16T19:25:04.790072] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-09-16T19:25:04.909334] [DEBUG] [LOCAL ] MLI: Primary data saved (32230 records). [2025-09-16T19:25:04.909533] [INFO ] [MAIN ] MLI: Full history saved. [2025-09-16T19:25:04.911130] [INFO ] [MAIN ] Update Progress: 1366/2369 symbols completed. MLI took 23.2s. Estimated remaining: 1h 46m 29s. Rate: 9.4 symbols/min [2025-09-16T19:25:05.082784] [INFO ] [MAIN ] MLNK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:05.103150] [DEBUG] [LOCAL ] MLNK: Loaded 18738 existing records from primary storage. [2025-09-16T19:25:05.103297] [INFO ] [MAIN ] MLNK: Loaded 18738 existing records. [2025-09-16T19:25:05.180137] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-09. [2025-09-16T19:25:05.281083] [WARN ] [MAIN ] MLNK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:25:05.281358] [INFO ] [MAIN ] MLNK: Wiping existing primary data before full refresh. [2025-09-16T19:25:05.281405] [INFO ] [LOCAL ] MLNK: Starting deletion of primary data. [2025-09-16T19:25:05.281944] [DEBUG] [LOCAL ] MLNK: Deleted primary data file. [2025-09-16T19:25:05.281993] [INFO ] [MAIN ] MLNK: Starting full primary refresh. [2025-09-16T19:25:05.650179] [INFO ] [REMOTE ] MLNK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:05.661719] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-09-16T19:25:05.666246] [INFO ] [REMOTE ] MKL: Full fetch complete (37642 records). [2025-09-16T19:25:05.666414] [DEBUG] [VALIDATOR ] MKL: Validating primary data. [2025-09-16T19:25:05.980122] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-02. [2025-09-16T19:25:06.065600] [DEBUG] [VALIDATOR ] MKL: All expected trading days are present. [2025-09-16T19:25:06.065711] [DEBUG] [VALIDATOR ] MKL: Primary data validation passed. [2025-09-16T19:25:06.066527] [INFO ] [MAIN ] MKL: Fetched primary data is valid. [2025-09-16T19:25:06.066572] [DEBUG] [LOCAL ] MKL: Saving primary data. [2025-09-16T19:25:06.067538] [DEBUG] [LOCAL ] MKL: Trimmed 37642 records to 25288 (keeping 150 days of data) [2025-09-16T19:25:06.233061] [DEBUG] [LOCAL ] MKL: Primary data saved (25288 records). [2025-09-16T19:25:06.233199] [INFO ] [MAIN ] MKL: Full history saved. [2025-09-16T19:25:06.233286] [INFO ] [MAIN ] Update Progress: 1367/2369 symbols completed. MKL took 25.5s. Estimated remaining: 1h 46m 19s. Rate: 9.4 symbols/min [2025-09-16T19:25:06.403215] [INFO ] [MAIN ] MLP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:06.407084] [DEBUG] [LOCAL ] MLP: Loaded 2255 existing records from primary storage. [2025-09-16T19:25:06.407174] [INFO ] [MAIN ] MLP: Loaded 2255 existing records. [2025-09-16T19:25:06.583538] [WARN ] [MAIN ] MLP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:25:06.583787] [INFO ] [MAIN ] MLP: Wiping existing primary data before full refresh. [2025-09-16T19:25:06.583834] [INFO ] [LOCAL ] MLP: Starting deletion of primary data. [2025-09-16T19:25:06.584076] [DEBUG] [LOCAL ] MLP: Deleted primary data file. [2025-09-16T19:25:06.584127] [INFO ] [MAIN ] MLP: Starting full primary refresh. [2025-09-16T19:25:06.791501] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-08. [2025-09-16T19:25:06.939216] [INFO ] [REMOTE ] MLP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:06.957251] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-09-16T19:25:06.963986] [INFO ] [REMOTE ] MKC: Full fetch complete (57009 records). [2025-09-16T19:25:06.964137] [DEBUG] [VALIDATOR ] MKC: Validating primary data. [2025-09-16T19:25:07.222978] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-09-16T19:25:07.373925] [DEBUG] [VALIDATOR ] MKC: All expected trading days are present. [2025-09-16T19:25:07.374014] [DEBUG] [VALIDATOR ] MKC: Primary data validation passed. [2025-09-16T19:25:07.374906] [INFO ] [MAIN ] MKC: Fetched primary data is valid. [2025-09-16T19:25:07.374953] [DEBUG] [LOCAL ] MKC: Saving primary data. [2025-09-16T19:25:07.376500] [DEBUG] [LOCAL ] MKC: Trimmed 57009 records to 39110 (keeping 150 days of data) [2025-09-16T19:25:07.590859] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-02. [2025-09-16T19:25:07.627506] [DEBUG] [LOCAL ] MKC: Primary data saved (39110 records). [2025-09-16T19:25:07.627690] [INFO ] [MAIN ] MKC: Full history saved. [2025-09-16T19:25:07.627783] [INFO ] [MAIN ] Update Progress: 1368/2369 symbols completed. MKC took 36.2s. Estimated remaining: 1h 46m 9s. Rate: 9.4 symbols/min [2025-09-16T19:25:07.796935] [INFO ] [MAIN ] MLR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:07.807757] [DEBUG] [LOCAL ] MLR: Loaded 8409 existing records from primary storage. [2025-09-16T19:25:07.807902] [INFO ] [MAIN ] MLR: Loaded 8409 existing records. [2025-09-16T19:25:07.985255] [WARN ] [MAIN ] MLR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:25:07.985486] [INFO ] [MAIN ] MLR: Wiping existing primary data before full refresh. [2025-09-16T19:25:07.985531] [INFO ] [LOCAL ] MLR: Starting deletion of primary data. [2025-09-16T19:25:07.985896] [DEBUG] [LOCAL ] MLR: Deleted primary data file. [2025-09-16T19:25:07.985942] [INFO ] [MAIN ] MLR: Starting full primary refresh. [2025-09-16T19:25:08.095622] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-09-16T19:25:08.343875] [INFO ] [REMOTE ] MLR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:08.362309] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-09-16T19:25:08.374838] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-09. [2025-09-16T19:25:09.174915] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-03. [2025-09-16T19:25:09.631072] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-09-16T19:25:09.634656] [INFO ] [REMOTE ] MLM: Full fetch complete (35261 records). [2025-09-16T19:25:09.634738] [DEBUG] [VALIDATOR ] MLM: Validating primary data. [2025-09-16T19:25:09.731417] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-09-16T19:25:09.983063] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-04. [2025-09-16T19:25:10.021380] [DEBUG] [VALIDATOR ] MLM: All expected trading days are present. [2025-09-16T19:25:10.021460] [DEBUG] [VALIDATOR ] MLM: Primary data validation passed. [2025-09-16T19:25:10.022217] [INFO ] [MAIN ] MLM: Fetched primary data is valid. [2025-09-16T19:25:10.022271] [DEBUG] [LOCAL ] MLM: Saving primary data. [2025-09-16T19:25:10.023132] [DEBUG] [LOCAL ] MLM: Trimmed 35261 records to 21554 (keeping 150 days of data) [2025-09-16T19:25:10.160627] [DEBUG] [LOCAL ] MLM: Primary data saved (21554 records). [2025-09-16T19:25:10.160815] [INFO ] [MAIN ] MLM: Full history saved. [2025-09-16T19:25:10.160899] [INFO ] [MAIN ] Update Progress: 1369/2369 symbols completed. MLM took 20.6s. Estimated remaining: 1h 46m 0s. Rate: 9.4 symbols/min [2025-09-16T19:25:10.328709] [INFO ] [MAIN ] MMC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:10.368340] [DEBUG] [LOCAL ] MMC: Loaded 38185 existing records from primary storage. [2025-09-16T19:25:10.368485] [INFO ] [MAIN ] MMC: Loaded 38185 existing records. [2025-09-16T19:25:10.552178] [WARN ] [MAIN ] MMC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:25:10.552433] [INFO ] [MAIN ] MMC: Wiping existing primary data before full refresh. [2025-09-16T19:25:10.552480] [INFO ] [LOCAL ] MMC: Starting deletion of primary data. [2025-09-16T19:25:10.553524] [DEBUG] [LOCAL ] MMC: Deleted primary data file. [2025-09-16T19:25:10.553577] [INFO ] [MAIN ] MMC: Starting full primary refresh. [2025-09-16T19:25:10.668021] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-09-16T19:25:10.780118] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-02. [2025-09-16T19:25:10.920771] [INFO ] [REMOTE ] MMC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:11.574963] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-02. [2025-09-16T19:25:11.661325] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-09-16T19:25:12.374899] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-03. [2025-09-16T19:25:13.174873] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-05. [2025-09-16T19:25:13.226055] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-09-16T19:25:13.537067] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-09-16T19:25:13.804337] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-09-16T19:25:13.974905] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-03. [2025-09-16T19:25:14.774969] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-04. [2025-09-16T19:25:15.574909] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-06. [2025-09-16T19:25:15.742901] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-09-16T19:25:15.910311] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-09-16T19:25:15.963539] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-09-16T19:25:16.374912] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-05. [2025-09-16T19:25:17.174899] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-03. [2025-09-16T19:25:17.471269] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-09-16T19:25:17.974911] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-07. [2025-09-16T19:25:18.025587] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-09-16T19:25:18.291015] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-09-16T19:25:18.774973] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-04. [2025-09-16T19:25:19.574918] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-08. [2025-09-16T19:25:19.717317] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-09-16T19:25:20.036365] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-09-16T19:25:20.374905] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-04. [2025-09-16T19:25:21.174907] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-05. [2025-09-16T19:25:21.975374] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-06. [2025-09-16T19:25:21.977131] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-09-16T19:25:22.193703] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T19:25:22.774948] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-06. [2025-09-16T19:25:23.117187] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-09-16T19:25:23.574866] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-09. [2025-09-16T19:25:23.582280] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-09-16T19:25:24.136364] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T19:25:24.137426] [INFO ] [REMOTE ] MLP: Full fetch complete (3228 records). [2025-09-16T19:25:24.137502] [DEBUG] [VALIDATOR ] MLP: Validating primary data. [2025-09-16T19:25:24.380009] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-07. [2025-09-16T19:25:24.499069] [DEBUG] [VALIDATOR ] MLP: All expected trading days are present. [2025-09-16T19:25:24.499155] [DEBUG] [VALIDATOR ] MLP: Primary data validation passed. [2025-09-16T19:25:24.499289] [INFO ] [MAIN ] MLP: Fetched primary data is valid. [2025-09-16T19:25:24.499329] [DEBUG] [LOCAL ] MLP: Saving primary data. [2025-09-16T19:25:24.499700] [DEBUG] [LOCAL ] MLP: Trimmed 3228 records to 2271 (keeping 150 days of data) [2025-09-16T19:25:24.513549] [DEBUG] [LOCAL ] MLP: Primary data saved (2271 records). [2025-09-16T19:25:24.513660] [INFO ] [MAIN ] MLP: Full history saved. [2025-09-16T19:25:24.513744] [INFO ] [MAIN ] Update Progress: 1370/2369 symbols completed. MLP took 18.3s. Estimated remaining: 1h 45m 60s. Rate: 9.4 symbols/min [2025-09-16T19:25:24.685518] [INFO ] [MAIN ] MMD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:24.695257] [DEBUG] [LOCAL ] MMD: Loaded 7760 existing records from primary storage. [2025-09-16T19:25:24.695394] [INFO ] [MAIN ] MMD: Loaded 7760 existing records. [2025-09-16T19:25:24.880508] [WARN ] [MAIN ] MMD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:25:24.880769] [INFO ] [MAIN ] MMD: Wiping existing primary data before full refresh. [2025-09-16T19:25:24.880816] [INFO ] [LOCAL ] MMD: Starting deletion of primary data. [2025-09-16T19:25:24.881182] [DEBUG] [LOCAL ] MMD: Deleted primary data file. [2025-09-16T19:25:24.881227] [INFO ] [MAIN ] MMD: Starting full primary refresh. [2025-09-16T19:25:25.180015] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-05. [2025-09-16T19:25:25.239820] [INFO ] [REMOTE ] MMD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:25.337868] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-09-16T19:25:25.974904] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-07. [2025-09-16T19:25:26.774925] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-02. [2025-09-16T19:25:27.029015] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-09-16T19:25:27.089575] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-09-16T19:25:27.574938] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-08. [2025-09-16T19:25:27.598209] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-09-16T19:25:28.374915] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-06. [2025-09-16T19:25:28.640339] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-09-16T19:25:29.174905] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-09. [2025-09-16T19:25:29.663340] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T19:25:29.974914] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-03. [2025-09-16T19:25:30.066171] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T19:25:30.068250] [INFO ] [REMOTE ] MLR: Full fetch complete (14400 records). [2025-09-16T19:25:30.068353] [DEBUG] [VALIDATOR ] MLR: Validating primary data. [2025-09-16T19:25:30.429663] [DEBUG] [VALIDATOR ] MLR: All expected trading days are present. [2025-09-16T19:25:30.429748] [DEBUG] [VALIDATOR ] MLR: Primary data validation passed. [2025-09-16T19:25:30.430070] [INFO ] [MAIN ] MLR: Fetched primary data is valid. [2025-09-16T19:25:30.430110] [DEBUG] [LOCAL ] MLR: Saving primary data. [2025-09-16T19:25:30.430702] [DEBUG] [LOCAL ] MLR: Trimmed 14400 records to 8483 (keeping 150 days of data) [2025-09-16T19:25:30.484894] [DEBUG] [LOCAL ] MLR: Primary data saved (8483 records). [2025-09-16T19:25:30.485043] [INFO ] [MAIN ] MLR: Full history saved. [2025-09-16T19:25:30.485133] [INFO ] [MAIN ] Update Progress: 1371/2369 symbols completed. MLR took 22.9s. Estimated remaining: 1h 45m 53s. Rate: 9.4 symbols/min [2025-09-16T19:25:30.656580] [INFO ] [MAIN ] MMI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:30.666022] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-09-16T19:25:30.682479] [DEBUG] [LOCAL ] MMI: Loaded 10782 existing records from primary storage. [2025-09-16T19:25:30.682738] [INFO ] [MAIN ] MMI: Loaded 10782 existing records. [2025-09-16T19:25:30.780042] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-07. [2025-09-16T19:25:30.883670] [WARN ] [MAIN ] MMI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:25:30.883922] [INFO ] [MAIN ] MMI: Wiping existing primary data before full refresh. [2025-09-16T19:25:30.883967] [INFO ] [LOCAL ] MMI: Starting deletion of primary data. [2025-09-16T19:25:30.884349] [DEBUG] [LOCAL ] MMI: Deleted primary data file. [2025-09-16T19:25:30.884404] [INFO ] [MAIN ] MMI: Starting full primary refresh. [2025-09-16T19:25:31.237389] [INFO ] [REMOTE ] MMI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:31.574901] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-04. [2025-09-16T19:25:32.374900] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-08. [2025-09-16T19:25:32.536853] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-09-16T19:25:32.643713] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-09-16T19:25:33.174882] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-08. [2025-09-16T19:25:33.242635] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-09-16T19:25:33.974894] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-09. [2025-09-16T19:25:34.775013] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-02. [2025-09-16T19:25:34.989878] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-09-16T19:25:35.322566] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-09-16T19:25:35.326565] [INFO ] [REMOTE ] MLNK: Full fetch complete (27273 records). [2025-09-16T19:25:35.326666] [DEBUG] [VALIDATOR ] MLNK: Validating primary data. [2025-09-16T19:25:35.581968] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-09. [2025-09-16T19:25:35.692725] [DEBUG] [VALIDATOR ] MLNK: All expected trading days are present. [2025-09-16T19:25:35.692802] [DEBUG] [VALIDATOR ] MLNK: Primary data validation passed. [2025-09-16T19:25:35.693354] [INFO ] [MAIN ] MLNK: Fetched primary data is valid. [2025-09-16T19:25:35.693394] [DEBUG] [LOCAL ] MLNK: Saving primary data. [2025-09-16T19:25:35.694179] [DEBUG] [LOCAL ] MLNK: Trimmed 27273 records to 19036 (keeping 150 days of data) [2025-09-16T19:25:35.803004] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-09-16T19:25:35.818579] [DEBUG] [LOCAL ] MLNK: Primary data saved (19036 records). [2025-09-16T19:25:35.818715] [INFO ] [MAIN ] MLNK: Full history saved. [2025-09-16T19:25:35.818796] [INFO ] [MAIN ] Update Progress: 1372/2369 symbols completed. MLNK took 30.9s. Estimated remaining: 1h 45m 46s. Rate: 9.4 symbols/min [2025-09-16T19:25:35.986971] [INFO ] [MAIN ] MMKT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:35.987187] [DEBUG] [LOCAL ] MMKT: No existing data file found in primary storage. [2025-09-16T19:25:35.987232] [WARN ] [MAIN ] MMKT: No existing data found, performing full refresh instead. [2025-09-16T19:25:35.987384] [INFO ] [MAIN ] MMKT: Wiping existing primary data before full refresh. [2025-09-16T19:25:35.987424] [INFO ] [LOCAL ] MMKT: Starting deletion of primary data. [2025-09-16T19:25:35.987465] [DEBUG] [LOCAL ] MMKT: No primary data file found to delete. [2025-09-16T19:25:35.987498] [INFO ] [MAIN ] MMKT: Starting full primary refresh. [2025-09-16T19:25:36.340728] [INFO ] [REMOTE ] MMKT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:36.374844] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-02. [2025-09-16T19:25:36.712777] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-09-16T19:25:37.068204] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-09-16T19:25:37.074358] [INFO ] [REMOTE ] MMC: Full fetch complete (56292 records). [2025-09-16T19:25:37.074479] [DEBUG] [VALIDATOR ] MMC: Validating primary data. [2025-09-16T19:25:37.178122] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-03. [2025-09-16T19:25:37.485558] [DEBUG] [VALIDATOR ] MMC: All expected trading days are present. [2025-09-16T19:25:37.485659] [DEBUG] [VALIDATOR ] MMC: Primary data validation passed. [2025-09-16T19:25:37.486549] [INFO ] [MAIN ] MMC: Fetched primary data is valid. [2025-09-16T19:25:37.487057] [DEBUG] [LOCAL ] MMC: Saving primary data. [2025-09-16T19:25:37.488268] [DEBUG] [LOCAL ] MMC: Trimmed 56292 records to 38574 (keeping 150 days of data) [2025-09-16T19:25:37.703409] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-09-16T19:25:37.741164] [DEBUG] [LOCAL ] MMC: Primary data saved (38574 records). [2025-09-16T19:25:37.741460] [INFO ] [MAIN ] MMC: Full history saved. [2025-09-16T19:25:37.742519] [INFO ] [MAIN ] Update Progress: 1373/2369 symbols completed. MMC took 27.6s. Estimated remaining: 1h 45m 36s. Rate: 9.4 symbols/min [2025-09-16T19:25:37.913269] [INFO ] [MAIN ] MMM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:37.955670] [DEBUG] [LOCAL ] MMM: Loaded 39261 existing records from primary storage. [2025-09-16T19:25:37.955831] [INFO ] [MAIN ] MMM: Loaded 39261 existing records. [2025-09-16T19:25:37.984678] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-04. [2025-09-16T19:25:38.133635] [WARN ] [MAIN ] MMM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:25:38.133883] [INFO ] [MAIN ] MMM: Wiping existing primary data before full refresh. [2025-09-16T19:25:38.133928] [INFO ] [LOCAL ] MMM: Starting deletion of primary data. [2025-09-16T19:25:38.134942] [DEBUG] [LOCAL ] MMM: Deleted primary data file. [2025-09-16T19:25:38.134991] [INFO ] [MAIN ] MMM: Starting full primary refresh. [2025-09-16T19:25:38.490500] [INFO ] [REMOTE ] MMM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:38.493875] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-09-16T19:25:38.774892] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-02. [2025-09-16T19:25:39.574902] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-03. [2025-09-16T19:25:40.374860] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-05. [2025-09-16T19:25:40.481783] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-09-16T19:25:40.644678] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-09-16T19:25:40.961705] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-09-16T19:25:41.174885] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-03. [2025-09-16T19:25:41.974886] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-04. [2025-09-16T19:25:42.774957] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-05. [2025-09-16T19:25:43.007903] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-09-16T19:25:43.088625] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-09-16T19:25:43.128324] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-09-16T19:25:43.574978] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-06. [2025-09-16T19:25:43.914906] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-09-16T19:25:44.374907] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-04. [2025-09-16T19:25:45.174912] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-07. [2025-09-16T19:25:45.780969] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-09-16T19:25:45.974941] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-08. [2025-09-16T19:25:46.024273] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-09-16T19:25:46.414903] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-09-16T19:25:46.774910] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-09. [2025-09-16T19:25:47.276008] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-09-16T19:25:47.276803] [INFO ] [REMOTE ] MMKT: Full fetch complete (1520 records). [2025-09-16T19:25:47.276872] [DEBUG] [VALIDATOR ] MMKT: Validating primary data. [2025-09-16T19:25:47.463632] [WARN ] [VALIDATOR ] MMKT: Validation failed - insufficient historical data. Have 148 unique days, need 150 days [2025-09-16T19:25:47.463858] [FATAL] [MAIN ] MMKT: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:25:47.463904] [WARN ] [MAIN ] MMKT: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:25:47.463950] [INFO ] [LOCAL ] MMKT: Starting deletion of primary data. [2025-09-16T19:25:47.464008] [DEBUG] [LOCAL ] MMKT: No primary data file found to delete. [2025-09-16T19:25:47.464061] [INFO ] [LOCAL ] MMKT: Starting deletion of derivative data. [2025-09-16T19:25:47.464104] [DEBUG] [LOCAL ] MMKT: No derivative data file found to delete. [2025-09-16T19:25:47.464163] [INFO ] [MAIN ] Update Progress: 1374/2369 symbols completed. MMKT took 11.6s. Estimated remaining: 1h 45m 32s. Rate: 9.4 symbols/min [2025-09-16T19:25:47.580002] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-06. [2025-09-16T19:25:47.645881] [INFO ] [MAIN ] MMS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:47.669128] [DEBUG] [LOCAL ] MMS: Loaded 24361 existing records from primary storage. [2025-09-16T19:25:47.669260] [INFO ] [MAIN ] MMS: Loaded 24361 existing records. [2025-09-16T19:25:47.857727] [WARN ] [MAIN ] MMS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:25:47.858625] [INFO ] [MAIN ] MMS: Wiping existing primary data before full refresh. [2025-09-16T19:25:47.858697] [INFO ] [LOCAL ] MMS: Starting deletion of primary data. [2025-09-16T19:25:47.859616] [DEBUG] [LOCAL ] MMS: Deleted primary data file. [2025-09-16T19:25:47.859708] [INFO ] [MAIN ] MMS: Starting full primary refresh. [2025-09-16T19:25:48.233872] [INFO ] [REMOTE ] MMS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:48.240393] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-09-16T19:25:48.374893] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-07. [2025-09-16T19:25:49.147253] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T19:25:49.174864] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-08. [2025-09-16T19:25:49.974940] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-02. [2025-09-16T19:25:50.088890] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-09-16T19:25:50.774901] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-09. [2025-09-16T19:25:51.465807] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T19:25:51.485904] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-09-16T19:25:51.487557] [INFO ] [REMOTE ] MMD: Full fetch complete (12217 records). [2025-09-16T19:25:51.487623] [DEBUG] [VALIDATOR ] MMD: Validating primary data. [2025-09-16T19:25:51.579983] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-05. [2025-09-16T19:25:51.849356] [DEBUG] [VALIDATOR ] MMD: All expected trading days are present. [2025-09-16T19:25:51.849434] [DEBUG] [VALIDATOR ] MMD: Primary data validation passed. [2025-09-16T19:25:51.849723] [INFO ] [MAIN ] MMD: Fetched primary data is valid. [2025-09-16T19:25:51.849768] [DEBUG] [LOCAL ] MMD: Saving primary data. [2025-09-16T19:25:51.850200] [DEBUG] [LOCAL ] MMD: Trimmed 12217 records to 7827 (keeping 150 days of data) [2025-09-16T19:25:51.897864] [DEBUG] [LOCAL ] MMD: Primary data saved (7827 records). [2025-09-16T19:25:51.897976] [INFO ] [MAIN ] MMD: Full history saved. [2025-09-16T19:25:51.898059] [INFO ] [MAIN ] Update Progress: 1375/2369 symbols completed. MMD took 27.4s. Estimated remaining: 1h 45m 25s. Rate: 9.4 symbols/min [2025-09-16T19:25:52.066994] [INFO ] [MAIN ] MMT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:25:52.075584] [DEBUG] [LOCAL ] MMT: Loaded 7640 existing records from primary storage. [2025-09-16T19:25:52.075734] [INFO ] [MAIN ] MMT: Loaded 7640 existing records. [2025-09-16T19:25:52.250632] [WARN ] [MAIN ] MMT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:25:52.250866] [INFO ] [MAIN ] MMT: Wiping existing primary data before full refresh. [2025-09-16T19:25:52.250911] [INFO ] [LOCAL ] MMT: Starting deletion of primary data. [2025-09-16T19:25:52.251175] [DEBUG] [LOCAL ] MMT: Deleted primary data file. [2025-09-16T19:25:52.251219] [INFO ] [MAIN ] MMT: Starting full primary refresh. [2025-09-16T19:25:52.379982] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-03. [2025-09-16T19:25:52.611675] [INFO ] [REMOTE ] MMT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:25:52.621765] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-09-16T19:25:53.174914] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-02. [2025-09-16T19:25:53.950508] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-09-16T19:25:53.974854] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-03. [2025-09-16T19:25:54.007896] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-09-16T19:25:54.774909] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-06. [2025-09-16T19:25:54.941434] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-09-16T19:25:55.519855] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-09-16T19:25:55.574854] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-07. [2025-09-16T19:25:56.374897] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-05. [2025-09-16T19:25:56.468893] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-09-16T19:25:57.174913] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-08. [2025-09-16T19:25:57.974931] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-04. [2025-09-16T19:25:58.021407] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-09-16T19:25:58.488228] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-05 (2.11 seconds). [2025-09-16T19:25:58.697011] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-09-16T19:25:58.774868] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-04. [2025-09-16T19:25:59.574889] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-06. [2025-09-16T19:26:00.271145] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T19:26:00.374897] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-05. [2025-09-16T19:26:01.174873] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-09. [2025-09-16T19:26:01.274313] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-09-16T19:26:01.666862] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-09-16T19:26:01.794694] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-09-16T19:26:01.798089] [INFO ] [REMOTE ] MMI: Full fetch complete (16809 records). [2025-09-16T19:26:01.798244] [DEBUG] [VALIDATOR ] MMI: Validating primary data. [2025-09-16T19:26:01.980125] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-06. [2025-09-16T19:26:02.209087] [DEBUG] [VALIDATOR ] MMI: All expected trading days are present. [2025-09-16T19:26:02.209168] [DEBUG] [VALIDATOR ] MMI: Primary data validation passed. [2025-09-16T19:26:02.209531] [INFO ] [MAIN ] MMI: Fetched primary data is valid. [2025-09-16T19:26:02.209582] [DEBUG] [LOCAL ] MMI: Saving primary data. [2025-09-16T19:26:02.210199] [DEBUG] [LOCAL ] MMI: Trimmed 16809 records to 10858 (keeping 150 days of data) [2025-09-16T19:26:02.280304] [DEBUG] [LOCAL ] MMI: Primary data saved (10858 records). [2025-09-16T19:26:02.280457] [INFO ] [MAIN ] MMI: Full history saved. [2025-09-16T19:26:02.280579] [INFO ] [MAIN ] Update Progress: 1376/2369 symbols completed. MMI took 31.8s. Estimated remaining: 1h 45m 21s. Rate: 9.4 symbols/min [2025-09-16T19:26:02.452450] [INFO ] [MAIN ] MMU: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:26:02.464838] [DEBUG] [LOCAL ] MMU: Loaded 12090 existing records from primary storage. [2025-09-16T19:26:02.464978] [INFO ] [MAIN ] MMU: Loaded 12090 existing records. [2025-09-16T19:26:02.645714] [WARN ] [MAIN ] MMU: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:26:02.645996] [INFO ] [MAIN ] MMU: Wiping existing primary data before full refresh. [2025-09-16T19:26:02.646044] [INFO ] [LOCAL ] MMU: Starting deletion of primary data. [2025-09-16T19:26:02.646487] [DEBUG] [LOCAL ] MMU: Deleted primary data file. [2025-09-16T19:26:02.646534] [INFO ] [MAIN ] MMU: Starting full primary refresh. [2025-09-16T19:26:02.785939] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-05. [2025-09-16T19:26:03.013969] [INFO ] [REMOTE ] MMU: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:26:03.465873] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T19:26:03.482083] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-09-16T19:26:03.574864] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-07. [2025-09-16T19:26:04.374897] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-07. [2025-09-16T19:26:04.878537] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-09-16T19:26:05.174914] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-02. [2025-09-16T19:26:05.831869] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-09-16T19:26:05.974929] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-06. [2025-09-16T19:26:06.418762] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-09-16T19:26:06.745944] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T19:26:06.774893] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-07. [2025-09-16T19:26:07.564602] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T19:26:07.574842] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-08. [2025-09-16T19:26:08.212040] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-09-16T19:26:08.374904] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-08. [2025-09-16T19:26:09.174924] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-09. [2025-09-16T19:26:09.909073] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T19:26:09.910461] [INFO ] [REMOTE ] MMT: Full fetch complete (12480 records). [2025-09-16T19:26:09.910544] [DEBUG] [VALIDATOR ] MMT: Validating primary data. [2025-09-16T19:26:09.980004] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-03. [2025-09-16T19:26:10.269003] [DEBUG] [VALIDATOR ] MMT: All expected trading days are present. [2025-09-16T19:26:10.269093] [DEBUG] [VALIDATOR ] MMT: Primary data validation passed. [2025-09-16T19:26:10.269451] [INFO ] [MAIN ] MMT: Fetched primary data is valid. [2025-09-16T19:26:10.269514] [DEBUG] [LOCAL ] MMT: Saving primary data. [2025-09-16T19:26:10.270071] [DEBUG] [LOCAL ] MMT: Trimmed 12480 records to 7760 (keeping 150 days of data) [2025-09-16T19:26:10.316387] [DEBUG] [LOCAL ] MMT: Primary data saved (7760 records). [2025-09-16T19:26:10.316544] [INFO ] [MAIN ] MMT: Full history saved. [2025-09-16T19:26:10.316695] [INFO ] [MAIN ] Update Progress: 1377/2369 symbols completed. MMT took 18.4s. Estimated remaining: 1h 45m 16s. Rate: 9.4 symbols/min [2025-09-16T19:26:10.463842] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-09-16T19:26:10.528296] [INFO ] [MAIN ] MNR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:26:10.546923] [DEBUG] [LOCAL ] MNR: Loaded 18562 existing records from primary storage. [2025-09-16T19:26:10.547045] [INFO ] [MAIN ] MNR: Loaded 18562 existing records. [2025-09-16T19:26:10.726715] [WARN ] [MAIN ] MNR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:26:10.726978] [INFO ] [MAIN ] MNR: Wiping existing primary data before full refresh. [2025-09-16T19:26:10.727024] [INFO ] [LOCAL ] MNR: Starting deletion of primary data. [2025-09-16T19:26:10.727589] [DEBUG] [LOCAL ] MNR: Deleted primary data file. [2025-09-16T19:26:10.727633] [INFO ] [MAIN ] MNR: Starting full primary refresh. [2025-09-16T19:26:10.781253] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-08. [2025-09-16T19:26:11.100110] [INFO ] [REMOTE ] MNR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:26:11.106561] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-09-16T19:26:11.574950] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-09. [2025-09-16T19:26:12.235878] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-09-16T19:26:12.374898] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-09. [2025-09-16T19:26:13.174934] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-04. [2025-09-16T19:26:13.408531] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-09-16T19:26:13.430095] [INFO ] [REMOTE ] MMM: Full fetch complete (57925 records). [2025-09-16T19:26:13.430967] [DEBUG] [VALIDATOR ] MMM: Validating primary data. [2025-09-16T19:26:13.431951] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-09-16T19:26:13.517197] [INFO ] [REMOTE ] MMS: Full fetch complete (37710 records). [2025-09-16T19:26:13.539425] [DEBUG] [VALIDATOR ] MMS: Validating primary data. [2025-09-16T19:26:13.989188] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-02. [2025-09-16T19:26:14.147931] [DEBUG] [VALIDATOR ] MMM: All expected trading days are present. [2025-09-16T19:26:14.151692] [DEBUG] [VALIDATOR ] MMM: Primary data validation passed. [2025-09-16T19:26:14.171991] [INFO ] [MAIN ] MMM: Fetched primary data is valid. [2025-09-16T19:26:14.177199] [DEBUG] [LOCAL ] MMM: Saving primary data. [2025-09-16T19:26:14.201051] [DEBUG] [LOCAL ] MMM: Trimmed 57925 records to 39647 (keeping 150 days of data) [2025-09-16T19:26:14.397580] [DEBUG] [VALIDATOR ] MMS: All expected trading days are present. [2025-09-16T19:26:14.397480] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-09-16T19:26:14.403507] [DEBUG] [VALIDATOR ] MMS: Primary data validation passed. [2025-09-16T19:26:14.404561] [INFO ] [MAIN ] MMS: Fetched primary data is valid. [2025-09-16T19:26:14.405188] [DEBUG] [LOCAL ] MMS: Saving primary data. [2025-09-16T19:26:14.412990] [DEBUG] [LOCAL ] MMS: Trimmed 37710 records to 24619 (keeping 150 days of data) [2025-09-16T19:26:14.725933] [DEBUG] [LOCAL ] MMS: Primary data saved (24619 records). [2025-09-16T19:26:14.727772] [INFO ] [MAIN ] MMS: Full history saved. [2025-09-16T19:26:14.728155] [INFO ] [MAIN ] Update Progress: 1378/2369 symbols completed. MMS took 27.3s. Estimated remaining: 1h 45m 8s. Rate: 9.4 symbols/min [2025-09-16T19:26:14.785611] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-05. [2025-09-16T19:26:14.913511] [INFO ] [MAIN ] MNSO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:26:14.915888] [DEBUG] [LOCAL ] MMM: Primary data saved (39647 records). [2025-09-16T19:26:14.916071] [INFO ] [MAIN ] MMM: Full history saved. [2025-09-16T19:26:14.916975] [INFO ] [MAIN ] Update Progress: 1379/2369 symbols completed. MMM took 37.2s. Estimated remaining: 1h 44m 57s. Rate: 9.4 symbols/min [2025-09-16T19:26:15.099166] [INFO ] [MAIN ] MNTN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:26:15.099971] [DEBUG] [LOCAL ] MNTN: No existing data file found in primary storage. [2025-09-16T19:26:15.100030] [WARN ] [MAIN ] MNTN: No existing data found, performing full refresh instead. [2025-09-16T19:26:15.109821] [INFO ] [MAIN ] MNTN: Wiping existing primary data before full refresh. [2025-09-16T19:26:15.109926] [INFO ] [LOCAL ] MNTN: Starting deletion of primary data. [2025-09-16T19:26:15.110489] [DEBUG] [LOCAL ] MNTN: No primary data file found to delete. [2025-09-16T19:26:15.110559] [INFO ] [MAIN ] MNTN: Starting full primary refresh. [2025-09-16T19:26:15.110403] [DEBUG] [LOCAL ] MNSO: Loaded 31713 existing records from primary storage. [2025-09-16T19:26:15.143394] [INFO ] [MAIN ] MNSO: Loaded 31713 existing records. [2025-09-16T19:26:15.507941] [WARN ] [MAIN ] MNSO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:26:15.593089] [INFO ] [MAIN ] MNSO: Wiping existing primary data before full refresh. [2025-09-16T19:26:15.597981] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-09-16T19:26:15.601616] [INFO ] [LOCAL ] MNSO: Starting deletion of primary data. [2025-09-16T19:26:15.602520] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-06. [2025-09-16T19:26:15.660526] [DEBUG] [LOCAL ] MNSO: Deleted primary data file. [2025-09-16T19:26:15.685350] [INFO ] [MAIN ] MNSO: Starting full primary refresh. [2025-09-16T19:26:15.685283] [INFO ] [REMOTE ] MNTN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:26:15.861982] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-09-16T19:26:16.076905] [INFO ] [REMOTE ] MNSO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:26:16.374906] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-03. [2025-09-16T19:26:16.601812] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-09-16T19:26:17.174934] [INFO ] [REMOTE ] MNTN: Fetching primary data for 2025-02. [2025-09-16T19:26:17.290493] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-09-16T19:26:17.372115] [DEBUG] [REMOTE ] MNTN: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:26:17.372383] [WARN ] [REMOTE ] MNTN: No data returned for month 2025-02 [2025-09-16T19:26:17.974916] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-02. [2025-09-16T19:26:18.774907] [INFO ] [REMOTE ] MNTN: Fetching primary data for 2025-03. [2025-09-16T19:26:18.952841] [DEBUG] [REMOTE ] MNTN: Error fetching primary data for 2025-03: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:26:18.953298] [WARN ] [REMOTE ] MNTN: No data returned for month 2025-03 [2025-09-16T19:26:19.574931] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-07. [2025-09-16T19:26:19.626270] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-09-16T19:26:20.101464] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-09-16T19:26:20.374933] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-08. [2025-09-16T19:26:21.174915] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-04. [2025-09-16T19:26:21.451535] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-09-16T19:26:21.974934] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-09. [2025-09-16T19:26:22.339002] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-09-16T19:26:22.636991] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-09-16T19:26:22.639726] [INFO ] [REMOTE ] MMU: Full fetch complete (18739 records). [2025-09-16T19:26:22.639824] [DEBUG] [VALIDATOR ] MMU: Validating primary data. [2025-09-16T19:26:22.780051] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-03. [2025-09-16T19:26:23.026448] [DEBUG] [VALIDATOR ] MMU: All expected trading days are present. [2025-09-16T19:26:23.026531] [DEBUG] [VALIDATOR ] MMU: Primary data validation passed. [2025-09-16T19:26:23.026949] [INFO ] [MAIN ] MMU: Fetched primary data is valid. [2025-09-16T19:26:23.026997] [DEBUG] [LOCAL ] MMU: Saving primary data. [2025-09-16T19:26:23.027721] [DEBUG] [LOCAL ] MMU: Trimmed 18739 records to 12224 (keeping 150 days of data) [2025-09-16T19:26:23.103636] [DEBUG] [LOCAL ] MMU: Primary data saved (12224 records). [2025-09-16T19:26:23.103826] [INFO ] [MAIN ] MMU: Full history saved. [2025-09-16T19:26:23.103928] [INFO ] [MAIN ] Update Progress: 1380/2369 symbols completed. MMU took 20.8s. Estimated remaining: 1h 44m 52s. Rate: 9.4 symbols/min [2025-09-16T19:26:23.277468] [INFO ] [MAIN ] MO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:26:23.318746] [DEBUG] [LOCAL ] MO: Loaded 39714 existing records from primary storage. [2025-09-16T19:26:23.318907] [INFO ] [MAIN ] MO: Loaded 39714 existing records. [2025-09-16T19:26:23.504523] [WARN ] [MAIN ] MO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:26:23.504903] [INFO ] [MAIN ] MO: Wiping existing primary data before full refresh. [2025-09-16T19:26:23.504951] [INFO ] [LOCAL ] MO: Starting deletion of primary data. [2025-09-16T19:26:23.505964] [DEBUG] [LOCAL ] MO: Deleted primary data file. [2025-09-16T19:26:23.506013] [INFO ] [MAIN ] MO: Starting full primary refresh. [2025-09-16T19:26:23.581521] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-05. [2025-09-16T19:26:23.867946] [INFO ] [REMOTE ] MO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:26:24.374930] [INFO ] [REMOTE ] MNTN: Fetching primary data for 2025-04. [2025-09-16T19:26:24.607815] [DEBUG] [REMOTE ] MNTN: Error fetching primary data for 2025-04: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:26:24.608143] [WARN ] [REMOTE ] MNTN: No data returned for month 2025-04 [2025-09-16T19:26:24.662072] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-09-16T19:26:24.663380] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-09-16T19:26:25.174934] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-02. [2025-09-16T19:26:25.974893] [INFO ] [REMOTE ] MNTN: Fetching primary data for 2025-05. [2025-09-16T19:26:26.774898] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-06. [2025-09-16T19:26:26.797792] [INFO ] [REMOTE ] MNTN: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-09-16T19:26:27.379995] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-09-16T19:26:27.574863] [INFO ] [REMOTE ] MNTN: Fetching primary data for 2025-06. [2025-09-16T19:26:28.063680] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T19:26:28.374902] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-04. [2025-09-16T19:26:28.991522] [INFO ] [REMOTE ] MNTN: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-09-16T19:26:29.174893] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-07. [2025-09-16T19:26:29.892292] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-09-16T19:26:29.974851] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-05. [2025-09-16T19:26:30.312466] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-09-16T19:26:30.774929] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-08. [2025-09-16T19:26:31.574910] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-03. [2025-09-16T19:26:31.802984] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-09-16T19:26:31.971912] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-09-16T19:26:32.374933] [INFO ] [REMOTE ] MNTN: Fetching primary data for 2025-07. [2025-09-16T19:26:33.174936] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-06. [2025-09-16T19:26:33.933599] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-03 (2.36 seconds). [2025-09-16T19:26:33.974885] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-09. [2025-09-16T19:26:34.037214] [INFO ] [REMOTE ] MNTN: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-09-16T19:26:34.403337] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-09-16T19:26:34.774895] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-07. [2025-09-16T19:26:34.975730] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-09-16T19:26:34.978528] [INFO ] [REMOTE ] MNR: Full fetch complete (27882 records). [2025-09-16T19:26:34.978600] [DEBUG] [VALIDATOR ] MNR: Validating primary data. [2025-09-16T19:26:35.365588] [DEBUG] [VALIDATOR ] MNR: All expected trading days are present. [2025-09-16T19:26:35.365737] [DEBUG] [VALIDATOR ] MNR: Primary data validation passed. [2025-09-16T19:26:35.366403] [INFO ] [MAIN ] MNR: Fetched primary data is valid. [2025-09-16T19:26:35.366461] [DEBUG] [LOCAL ] MNR: Saving primary data. [2025-09-16T19:26:35.367360] [DEBUG] [LOCAL ] MNR: Trimmed 27882 records to 18688 (keeping 150 days of data) [2025-09-16T19:26:35.496735] [DEBUG] [LOCAL ] MNR: Primary data saved (18688 records). [2025-09-16T19:26:35.496902] [INFO ] [MAIN ] MNR: Full history saved. [2025-09-16T19:26:35.497024] [INFO ] [MAIN ] Update Progress: 1381/2369 symbols completed. MNR took 25.2s. Estimated remaining: 1h 44m 50s. Rate: 9.4 symbols/min [2025-09-16T19:26:35.580055] [INFO ] [REMOTE ] MNTN: Fetching primary data for 2025-08. [2025-09-16T19:26:35.724033] [INFO ] [MAIN ] MOD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:26:35.777200] [DEBUG] [LOCAL ] MOD: Loaded 32979 existing records from primary storage. [2025-09-16T19:26:35.777410] [INFO ] [MAIN ] MOD: Loaded 32979 existing records. [2025-09-16T19:26:36.024126] [WARN ] [MAIN ] MOD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:26:36.024605] [INFO ] [MAIN ] MOD: Wiping existing primary data before full refresh. [2025-09-16T19:26:36.024707] [INFO ] [LOCAL ] MOD: Starting deletion of primary data. [2025-09-16T19:26:36.026017] [DEBUG] [LOCAL ] MOD: Deleted primary data file. [2025-09-16T19:26:36.026091] [INFO ] [MAIN ] MOD: Starting full primary refresh. [2025-09-16T19:26:36.275129] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-09-16T19:26:36.382695] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-08. [2025-09-16T19:26:36.559417] [INFO ] [REMOTE ] MOD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:26:37.174877] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-04. [2025-09-16T19:26:37.230183] [INFO ] [REMOTE ] MNTN: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-09-16T19:26:37.974877] [INFO ] [REMOTE ] MNTN: Fetching primary data for 2025-09. [2025-09-16T19:26:38.065778] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-09-16T19:26:38.774886] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-09. [2025-09-16T19:26:39.230587] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-04 (2.06 seconds). [2025-09-16T19:26:39.273842] [INFO ] [REMOTE ] MNTN: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T19:26:39.278722] [INFO ] [REMOTE ] MNTN: Full fetch complete (23218 records). [2025-09-16T19:26:39.278804] [DEBUG] [VALIDATOR ] MNTN: Validating primary data. [2025-09-16T19:26:39.462191] [WARN ] [VALIDATOR ] MNTN: Validation failed - insufficient historical data. Have 80 unique days, need 150 days [2025-09-16T19:26:39.462559] [FATAL] [MAIN ] MNTN: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:26:39.462623] [WARN ] [MAIN ] MNTN: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:26:39.462694] [INFO ] [LOCAL ] MNTN: Starting deletion of primary data. [2025-09-16T19:26:39.462756] [DEBUG] [LOCAL ] MNTN: No primary data file found to delete. [2025-09-16T19:26:39.462802] [INFO ] [LOCAL ] MNTN: Starting deletion of derivative data. [2025-09-16T19:26:39.462859] [DEBUG] [LOCAL ] MNTN: No derivative data file found to delete. [2025-09-16T19:26:39.462938] [INFO ] [MAIN ] Update Progress: 1382/2369 symbols completed. MNTN took 24.5s. Estimated remaining: 1h 44m 42s. Rate: 9.4 symbols/min [2025-09-16T19:26:39.580160] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-02. [2025-09-16T19:26:39.641480] [INFO ] [MAIN ] MODG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:26:39.681548] [DEBUG] [LOCAL ] MODG: Loaded 38153 existing records from primary storage. [2025-09-16T19:26:39.681730] [INFO ] [MAIN ] MODG: Loaded 38153 existing records. [2025-09-16T19:26:39.863141] [WARN ] [MAIN ] MODG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:26:39.863369] [INFO ] [MAIN ] MODG: Wiping existing primary data before full refresh. [2025-09-16T19:26:39.863423] [INFO ] [LOCAL ] MODG: Starting deletion of primary data. [2025-09-16T19:26:39.864162] [DEBUG] [LOCAL ] MODG: Deleted primary data file. [2025-09-16T19:26:39.864226] [INFO ] [MAIN ] MODG: Starting full primary refresh. [2025-09-16T19:26:40.219939] [INFO ] [REMOTE ] MODG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:26:40.328869] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-09-16T19:26:40.335671] [INFO ] [REMOTE ] MNSO: Full fetch complete (48509 records). [2025-09-16T19:26:40.335775] [DEBUG] [VALIDATOR ] MNSO: Validating primary data. [2025-09-16T19:26:40.380055] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-05. [2025-09-16T19:26:40.713107] [DEBUG] [VALIDATOR ] MNSO: All expected trading days are present. [2025-09-16T19:26:40.713230] [DEBUG] [VALIDATOR ] MNSO: Primary data validation passed. [2025-09-16T19:26:40.714007] [INFO ] [MAIN ] MNSO: Fetched primary data is valid. [2025-09-16T19:26:40.714067] [DEBUG] [LOCAL ] MNSO: Saving primary data. [2025-09-16T19:26:40.715046] [DEBUG] [LOCAL ] MNSO: Trimmed 48509 records to 31998 (keeping 150 days of data) [2025-09-16T19:26:40.925471] [DEBUG] [LOCAL ] MNSO: Primary data saved (31998 records). [2025-09-16T19:26:40.925683] [INFO ] [MAIN ] MNSO: Full history saved. [2025-09-16T19:26:40.926399] [INFO ] [MAIN ] Update Progress: 1383/2369 symbols completed. MNSO took 26.2s. Estimated remaining: 1h 44m 32s. Rate: 9.4 symbols/min [2025-09-16T19:26:41.096499] [INFO ] [MAIN ] MOH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:26:41.136484] [DEBUG] [LOCAL ] MOH: Loaded 31518 existing records from primary storage. [2025-09-16T19:26:41.136619] [INFO ] [MAIN ] MOH: Loaded 31518 existing records. [2025-09-16T19:26:41.180082] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-02. [2025-09-16T19:26:41.317673] [WARN ] [MAIN ] MOH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:26:41.318043] [INFO ] [MAIN ] MOH: Wiping existing primary data before full refresh. [2025-09-16T19:26:41.318126] [INFO ] [LOCAL ] MOH: Starting deletion of primary data. [2025-09-16T19:26:41.318880] [DEBUG] [LOCAL ] MOH: Deleted primary data file. [2025-09-16T19:26:41.318935] [INFO ] [MAIN ] MOH: Starting full primary refresh. [2025-09-16T19:26:41.675105] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-02 (2.09 seconds). [2025-09-16T19:26:41.719077] [INFO ] [REMOTE ] MOH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:26:41.974902] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-02. [2025-09-16T19:26:42.423426] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-09-16T19:26:42.621592] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T19:26:42.774910] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-03. [2025-09-16T19:26:43.521898] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-09-16T19:26:43.574931] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-03. [2025-09-16T19:26:44.131246] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T19:26:44.374935] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-04. [2025-09-16T19:26:44.716530] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T19:26:45.174954] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-06. [2025-09-16T19:26:45.974971] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-04. [2025-09-16T19:26:46.134442] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-09-16T19:26:46.774949] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-03. [2025-09-16T19:26:47.377931] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-06 (2.20 seconds). [2025-09-16T19:26:47.572385] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-09-16T19:26:47.580542] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-07. [2025-09-16T19:26:48.374955] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-05. [2025-09-16T19:26:48.589235] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T19:26:49.174973] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-04. [2025-09-16T19:26:49.855274] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-09-16T19:26:49.905440] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-07 (2.32 seconds). [2025-09-16T19:26:49.974913] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-08. [2025-09-16T19:26:50.616364] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-09-16T19:26:50.774911] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-05. [2025-09-16T19:26:51.574967] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-05. [2025-09-16T19:26:51.890441] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-09-16T19:26:52.375048] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-06. [2025-09-16T19:26:52.374971] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-09-16T19:26:53.174937] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-06. [2025-09-16T19:26:53.334457] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T19:26:53.687621] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-09-16T19:26:53.974941] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-06. [2025-09-16T19:26:54.768949] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-09-16T19:26:54.777415] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-07. [2025-09-16T19:26:55.585253] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-09. [2025-09-16T19:26:55.604817] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T19:26:56.374934] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-07. [2025-09-16T19:26:56.735724] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-09-16T19:26:57.174969] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-07. [2025-09-16T19:26:57.725947] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-09-16T19:26:57.732655] [INFO ] [REMOTE ] MO: Full fetch complete (58433 records). [2025-09-16T19:26:57.732767] [DEBUG] [VALIDATOR ] MO: Validating primary data. [2025-09-16T19:26:57.990098] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-08. [2025-09-16T19:26:58.119439] [DEBUG] [VALIDATOR ] MO: All expected trading days are present. [2025-09-16T19:26:58.119532] [DEBUG] [VALIDATOR ] MO: Primary data validation passed. [2025-09-16T19:26:58.120411] [INFO ] [MAIN ] MO: Fetched primary data is valid. [2025-09-16T19:26:58.120468] [DEBUG] [LOCAL ] MO: Saving primary data. [2025-09-16T19:26:58.121955] [DEBUG] [LOCAL ] MO: Trimmed 58433 records to 40104 (keeping 150 days of data) [2025-09-16T19:26:58.260726] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T19:26:58.447091] [DEBUG] [LOCAL ] MO: Primary data saved (40104 records). [2025-09-16T19:26:58.448569] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-09-16T19:26:58.447599] [INFO ] [MAIN ] MO: Full history saved. [2025-09-16T19:26:58.458140] [INFO ] [MAIN ] Update Progress: 1384/2369 symbols completed. MO took 35.4s. Estimated remaining: 1h 44m 34s. Rate: 9.4 symbols/min [2025-09-16T19:26:58.626958] [INFO ] [MAIN ] MOS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:26:58.665312] [DEBUG] [LOCAL ] MOS: Loaded 39505 existing records from primary storage. [2025-09-16T19:26:58.665489] [INFO ] [MAIN ] MOS: Loaded 39505 existing records. [2025-09-16T19:26:58.780053] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-08. [2025-09-16T19:26:58.846765] [WARN ] [MAIN ] MOS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:26:58.847000] [INFO ] [MAIN ] MOS: Wiping existing primary data before full refresh. [2025-09-16T19:26:58.847091] [INFO ] [LOCAL ] MOS: Starting deletion of primary data. [2025-09-16T19:26:58.848512] [DEBUG] [LOCAL ] MOS: Deleted primary data file. [2025-09-16T19:26:58.848571] [INFO ] [MAIN ] MOS: Starting full primary refresh. [2025-09-16T19:26:59.197382] [INFO ] [REMOTE ] MOS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:26:59.574883] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-02. [2025-09-16T19:26:59.721430] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-09-16T19:27:00.374927] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-08. [2025-09-16T19:27:00.627350] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-09-16T19:27:01.155186] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-09-16T19:27:01.174845] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-03. [2025-09-16T19:27:01.974936] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-09. [2025-09-16T19:27:02.092103] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-09-16T19:27:02.774906] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-09. [2025-09-16T19:27:03.017889] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-09-16T19:27:03.574903] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-09. [2025-09-16T19:27:03.677465] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-09-16T19:27:03.684977] [INFO ] [REMOTE ] MODG: Full fetch complete (56163 records). [2025-09-16T19:27:03.685097] [DEBUG] [VALIDATOR ] MODG: Validating primary data. [2025-09-16T19:27:04.070494] [DEBUG] [VALIDATOR ] MODG: All expected trading days are present. [2025-09-16T19:27:04.070571] [DEBUG] [VALIDATOR ] MODG: Primary data validation passed. [2025-09-16T19:27:04.071439] [INFO ] [MAIN ] MODG: Fetched primary data is valid. [2025-09-16T19:27:04.071507] [DEBUG] [LOCAL ] MODG: Saving primary data. [2025-09-16T19:27:04.072859] [DEBUG] [LOCAL ] MODG: Trimmed 56163 records to 38532 (keeping 150 days of data) [2025-09-16T19:27:04.296291] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T19:27:04.303880] [INFO ] [REMOTE ] MOD: Full fetch complete (50282 records). [2025-09-16T19:27:04.304781] [DEBUG] [VALIDATOR ] MOD: Validating primary data. [2025-09-16T19:27:04.379598] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-04. [2025-09-16T19:27:04.549920] [DEBUG] [LOCAL ] MODG: Primary data saved (38532 records). [2025-09-16T19:27:04.588669] [INFO ] [MAIN ] MODG: Full history saved. [2025-09-16T19:27:04.599665] [INFO ] [MAIN ] Update Progress: 1385/2369 symbols completed. MODG took 25.1s. Estimated remaining: 1h 44m 27s. Rate: 9.4 symbols/min [2025-09-16T19:27:04.890073] [INFO ] [MAIN ] MOV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:27:04.887610] [DEBUG] [VALIDATOR ] MOD: All expected trading days are present. [2025-09-16T19:27:04.890264] [DEBUG] [VALIDATOR ] MOD: Primary data validation passed. [2025-09-16T19:27:04.891289] [INFO ] [MAIN ] MOD: Fetched primary data is valid. [2025-09-16T19:27:04.891421] [DEBUG] [LOCAL ] MOD: Saving primary data. [2025-09-16T19:27:04.893490] [DEBUG] [LOCAL ] MOD: Trimmed 50282 records to 33252 (keeping 150 days of data) [2025-09-16T19:27:04.993242] [DEBUG] [LOCAL ] MOV: Loaded 15399 existing records from primary storage. [2025-09-16T19:27:04.994605] [INFO ] [MAIN ] MOV: Loaded 15399 existing records. [2025-09-16T19:27:05.180072] [WARN ] [MAIN ] MOV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:27:05.184630] [INFO ] [MAIN ] MOV: Wiping existing primary data before full refresh. [2025-09-16T19:27:05.185810] [INFO ] [LOCAL ] MOV: Starting deletion of primary data. [2025-09-16T19:27:05.186882] [DEBUG] [LOCAL ] MOV: Deleted primary data file. [2025-09-16T19:27:05.187428] [INFO ] [MAIN ] MOV: Starting full primary refresh. [2025-09-16T19:27:05.610916] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-09-16T19:27:05.647784] [INFO ] [REMOTE ] MOV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:27:05.648541] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-02. [2025-09-16T19:27:05.648255] [INFO ] [REMOTE ] MOH: Full fetch complete (45472 records). [2025-09-16T19:27:05.649581] [DEBUG] [VALIDATOR ] MOH: Validating primary data. [2025-09-16T19:27:05.985841] [DEBUG] [LOCAL ] MOD: Primary data saved (33252 records). [2025-09-16T19:27:05.991178] [INFO ] [MAIN ] MOD: Full history saved. [2025-09-16T19:27:05.996575] [INFO ] [MAIN ] Update Progress: 1386/2369 symbols completed. MOD took 30.5s. Estimated remaining: 1h 44m 18s. Rate: 9.4 symbols/min [2025-09-16T19:27:06.082769] [DEBUG] [VALIDATOR ] MOH: All expected trading days are present. [2025-09-16T19:27:06.098181] [DEBUG] [VALIDATOR ] MOH: Primary data validation passed. [2025-09-16T19:27:06.131470] [INFO ] [MAIN ] MOH: Fetched primary data is valid. [2025-09-16T19:27:06.148286] [DEBUG] [LOCAL ] MOH: Saving primary data. [2025-09-16T19:27:06.181207] [DEBUG] [LOCAL ] MOH: Trimmed 45472 records to 31868 (keeping 150 days of data) [2025-09-16T19:27:06.235563] [INFO ] [MAIN ] MP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:27:06.356877] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-09-16T19:27:06.375450] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-05. [2025-09-16T19:27:06.398856] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-09-16T19:27:06.422087] [DEBUG] [LOCAL ] MP: Loaded 39704 existing records from primary storage. [2025-09-16T19:27:06.422886] [INFO ] [MAIN ] MP: Loaded 39704 existing records. [2025-09-16T19:27:06.607788] [WARN ] [MAIN ] MP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:27:06.609247] [INFO ] [MAIN ] MP: Wiping existing primary data before full refresh. [2025-09-16T19:27:06.609795] [INFO ] [LOCAL ] MP: Starting deletion of primary data. [2025-09-16T19:27:06.611515] [DEBUG] [LOCAL ] MP: Deleted primary data file. [2025-09-16T19:27:06.611885] [INFO ] [MAIN ] MP: Starting full primary refresh. [2025-09-16T19:27:06.777050] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-03. [2025-09-16T19:27:07.045273] [INFO ] [REMOTE ] MP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:27:07.065793] [DEBUG] [LOCAL ] MOH: Primary data saved (31868 records). [2025-09-16T19:27:07.065973] [INFO ] [MAIN ] MOH: Full history saved. [2025-09-16T19:27:07.067440] [INFO ] [MAIN ] Update Progress: 1387/2369 symbols completed. MOH took 26.1s. Estimated remaining: 1h 44m 7s. Rate: 9.4 symbols/min [2025-09-16T19:27:07.248121] [INFO ] [MAIN ] MPA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:27:07.255182] [DEBUG] [LOCAL ] MPA: Loaded 3946 existing records from primary storage. [2025-09-16T19:27:07.255305] [INFO ] [MAIN ] MPA: Loaded 3946 existing records. [2025-09-16T19:27:07.430097] [WARN ] [MAIN ] MPA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:27:07.430334] [INFO ] [MAIN ] MPA: Wiping existing primary data before full refresh. [2025-09-16T19:27:07.430380] [INFO ] [LOCAL ] MPA: Starting deletion of primary data. [2025-09-16T19:27:07.430620] [DEBUG] [LOCAL ] MPA: Deleted primary data file. [2025-09-16T19:27:07.430722] [INFO ] [MAIN ] MPA: Starting full primary refresh. [2025-09-16T19:27:07.580008] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-02. [2025-09-16T19:27:07.785894] [INFO ] [REMOTE ] MPA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:27:08.036560] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-09-16T19:27:08.249935] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-09-16T19:27:08.374877] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-02. [2025-09-16T19:27:08.973852] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-09-16T19:27:09.174884] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-04. [2025-09-16T19:27:09.553793] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-09-16T19:27:09.974899] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-03. [2025-09-16T19:27:10.546332] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-09-16T19:27:10.565468] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-09-16T19:27:10.774917] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-06. [2025-09-16T19:27:11.574902] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-05. [2025-09-16T19:27:12.374957] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-04. [2025-09-16T19:27:12.616886] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-09-16T19:27:12.669685] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-09-16T19:27:13.178362] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-03. [2025-09-16T19:27:13.180598] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-09-16T19:27:13.974960] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-05. [2025-09-16T19:27:14.565077] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-09-16T19:27:14.774899] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-06. [2025-09-16T19:27:14.887946] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-09-16T19:27:15.549802] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T19:27:15.574879] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-07. [2025-09-16T19:27:16.374922] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-06. [2025-09-16T19:27:17.174920] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-04. [2025-09-16T19:27:17.391519] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-09-16T19:27:17.457442] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-09-16T19:27:17.974947] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-08. [2025-09-16T19:27:18.774955] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-07. [2025-09-16T19:27:19.421401] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-04 (2.25 seconds). [2025-09-16T19:27:19.574947] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-05. [2025-09-16T19:27:19.834334] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-09-16T19:27:19.858574] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-09-16T19:27:20.374966] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-07. [2025-09-16T19:27:20.993520] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-09-16T19:27:21.174972] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-08. [2025-09-16T19:27:21.585535] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-05 (2.01 seconds). [2025-09-16T19:27:21.974925] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-09. [2025-09-16T19:27:22.164943] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-09-16T19:27:22.774916] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-06. [2025-09-16T19:27:23.574947] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-08. [2025-09-16T19:27:23.615527] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-09-16T19:27:23.621941] [INFO ] [REMOTE ] MOS: Full fetch complete (58190 records). [2025-09-16T19:27:23.622075] [DEBUG] [VALIDATOR ] MOS: Validating primary data. [2025-09-16T19:27:24.002038] [DEBUG] [VALIDATOR ] MOS: All expected trading days are present. [2025-09-16T19:27:24.002126] [DEBUG] [VALIDATOR ] MOS: Primary data validation passed. [2025-09-16T19:27:24.003001] [INFO ] [MAIN ] MOS: Fetched primary data is valid. [2025-09-16T19:27:24.003049] [DEBUG] [LOCAL ] MOS: Saving primary data. [2025-09-16T19:27:24.004210] [DEBUG] [LOCAL ] MOS: Trimmed 58190 records to 39894 (keeping 150 days of data) [2025-09-16T19:27:24.223028] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-09-16T19:27:24.259496] [DEBUG] [LOCAL ] MOS: Primary data saved (39894 records). [2025-09-16T19:27:24.259717] [INFO ] [MAIN ] MOS: Full history saved. [2025-09-16T19:27:24.259836] [INFO ] [MAIN ] Update Progress: 1388/2369 symbols completed. MOS took 25.8s. Estimated remaining: 1h 44m 9s. Rate: 9.4 symbols/min [2025-09-16T19:27:24.380006] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-09. [2025-09-16T19:27:24.431237] [INFO ] [MAIN ] MPC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:27:24.469764] [DEBUG] [LOCAL ] MPC: Loaded 38350 existing records from primary storage. [2025-09-16T19:27:24.469908] [INFO ] [MAIN ] MPC: Loaded 38350 existing records. [2025-09-16T19:27:24.647877] [WARN ] [MAIN ] MPC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:27:24.648115] [INFO ] [MAIN ] MPC: Wiping existing primary data before full refresh. [2025-09-16T19:27:24.648163] [INFO ] [LOCAL ] MPC: Starting deletion of primary data. [2025-09-16T19:27:24.649069] [DEBUG] [LOCAL ] MPC: Deleted primary data file. [2025-09-16T19:27:24.649118] [INFO ] [MAIN ] MPC: Starting full primary refresh. [2025-09-16T19:27:25.003711] [INFO ] [REMOTE ] MPC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:27:25.174937] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-02. [2025-09-16T19:27:25.409598] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-06 (2.63 seconds). [2025-09-16T19:27:25.521194] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-09-16T19:27:25.524354] [INFO ] [REMOTE ] MOV: Full fetch complete (24008 records). [2025-09-16T19:27:25.524457] [DEBUG] [VALIDATOR ] MOV: Validating primary data. [2025-09-16T19:27:25.883335] [DEBUG] [VALIDATOR ] MOV: All expected trading days are present. [2025-09-16T19:27:25.883456] [DEBUG] [VALIDATOR ] MOV: Primary data validation passed. [2025-09-16T19:27:25.884006] [INFO ] [MAIN ] MOV: Fetched primary data is valid. [2025-09-16T19:27:25.884082] [DEBUG] [LOCAL ] MOV: Saving primary data. [2025-09-16T19:27:25.884776] [DEBUG] [LOCAL ] MOV: Trimmed 24008 records to 15530 (keeping 150 days of data) [2025-09-16T19:27:25.980789] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-09. [2025-09-16T19:27:25.989721] [DEBUG] [LOCAL ] MOV: Primary data saved (15530 records). [2025-09-16T19:27:25.989856] [INFO ] [MAIN ] MOV: Full history saved. [2025-09-16T19:27:25.989956] [INFO ] [MAIN ] Update Progress: 1389/2369 symbols completed. MOV took 21.4s. Estimated remaining: 1h 43m 59s. Rate: 9.4 symbols/min [2025-09-16T19:27:26.194320] [INFO ] [MAIN ] MPLX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:27:26.229862] [DEBUG] [LOCAL ] MPLX: Loaded 37164 existing records from primary storage. [2025-09-16T19:27:26.230005] [INFO ] [MAIN ] MPLX: Loaded 37164 existing records. [2025-09-16T19:27:26.404436] [WARN ] [MAIN ] MPLX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:27:26.404714] [INFO ] [MAIN ] MPLX: Wiping existing primary data before full refresh. [2025-09-16T19:27:26.404785] [INFO ] [LOCAL ] MPLX: Starting deletion of primary data. [2025-09-16T19:27:26.405678] [DEBUG] [LOCAL ] MPLX: Deleted primary data file. [2025-09-16T19:27:26.405734] [INFO ] [MAIN ] MPLX: Starting full primary refresh. [2025-09-16T19:27:26.688452] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-09-16T19:27:26.715565] [INFO ] [REMOTE ] MPA: Full fetch complete (5711 records). [2025-09-16T19:27:26.715709] [DEBUG] [VALIDATOR ] MPA: Validating primary data. [2025-09-16T19:27:26.789479] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-07. [2025-09-16T19:27:26.768597] [INFO ] [REMOTE ] MPLX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:27:27.139529] [DEBUG] [VALIDATOR ] MPA: All expected trading days are present. [2025-09-16T19:27:27.139670] [DEBUG] [VALIDATOR ] MPA: Primary data validation passed. [2025-09-16T19:27:27.139917] [INFO ] [MAIN ] MPA: Fetched primary data is valid. [2025-09-16T19:27:27.139971] [DEBUG] [LOCAL ] MPA: Saving primary data. [2025-09-16T19:27:27.140563] [DEBUG] [LOCAL ] MPA: Trimmed 5711 records to 4016 (keeping 150 days of data) [2025-09-16T19:27:27.167159] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-09-16T19:27:27.201689] [DEBUG] [LOCAL ] MPA: Primary data saved (4016 records). [2025-09-16T19:27:27.201782] [INFO ] [MAIN ] MPA: Full history saved. [2025-09-16T19:27:27.201862] [INFO ] [MAIN ] Update Progress: 1390/2369 symbols completed. MPA took 20.1s. Estimated remaining: 1h 43m 49s. Rate: 9.4 symbols/min [2025-09-16T19:27:27.374256] [INFO ] [MAIN ] MPV: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:27:27.379098] [DEBUG] [LOCAL ] MPV: Loaded 3301 existing records from primary storage. [2025-09-16T19:27:27.379183] [INFO ] [MAIN ] MPV: Loaded 3301 existing records. [2025-09-16T19:27:27.565919] [WARN ] [MAIN ] MPV: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:27:27.566315] [INFO ] [MAIN ] MPV: Wiping existing primary data before full refresh. [2025-09-16T19:27:27.566366] [INFO ] [LOCAL ] MPV: Starting deletion of primary data. [2025-09-16T19:27:27.566609] [DEBUG] [LOCAL ] MPV: Deleted primary data file. [2025-09-16T19:27:27.566684] [INFO ] [MAIN ] MPV: Starting full primary refresh. [2025-09-16T19:27:27.580084] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-03. [2025-09-16T19:27:27.931178] [INFO ] [REMOTE ] MPV: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:27:28.375044] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-02. [2025-09-16T19:27:29.174965] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-02. [2025-09-16T19:27:29.486792] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-07 (2.70 seconds). [2025-09-16T19:27:29.573752] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-09-16T19:27:29.792395] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-09-16T19:27:29.974936] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-04. [2025-09-16T19:27:30.057779] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-09-16T19:27:30.774920] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-08. [2025-09-16T19:27:31.574946] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-03. [2025-09-16T19:27:31.897203] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-09-16T19:27:32.133037] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-09-16T19:27:32.374889] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-05. [2025-09-16T19:27:33.148230] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-08 (2.37 seconds). [2025-09-16T19:27:33.174865] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-09. [2025-09-16T19:27:33.974894] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-04. [2025-09-16T19:27:34.218948] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-09-16T19:27:34.437110] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-09-16T19:27:34.774911] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-03. [2025-09-16T19:27:35.574904] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-05. [2025-09-16T19:27:35.774731] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-09 (2.60 seconds). [2025-09-16T19:27:35.781842] [INFO ] [REMOTE ] MP: Full fetch complete (58249 records). [2025-09-16T19:27:35.782053] [DEBUG] [VALIDATOR ] MP: Validating primary data. [2025-09-16T19:27:36.185050] [DEBUG] [VALIDATOR ] MP: All expected trading days are present. [2025-09-16T19:27:36.185144] [DEBUG] [VALIDATOR ] MP: Primary data validation passed. [2025-09-16T19:27:36.186031] [INFO ] [MAIN ] MP: Fetched primary data is valid. [2025-09-16T19:27:36.186080] [DEBUG] [LOCAL ] MP: Saving primary data. [2025-09-16T19:27:36.187420] [DEBUG] [LOCAL ] MP: Trimmed 58249 records to 40094 (keeping 150 days of data) [2025-09-16T19:27:36.284686] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-09-16T19:27:36.381088] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-06. [2025-09-16T19:27:36.455362] [DEBUG] [LOCAL ] MP: Primary data saved (40094 records). [2025-09-16T19:27:36.455560] [INFO ] [MAIN ] MP: Full history saved. [2025-09-16T19:27:36.455671] [INFO ] [MAIN ] Update Progress: 1391/2369 symbols completed. MP took 30.4s. Estimated remaining: 1h 43m 45s. Rate: 9.4 symbols/min [2025-09-16T19:27:36.635202] [INFO ] [MAIN ] MPW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:27:36.667306] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-09-16T19:27:36.711039] [DEBUG] [LOCAL ] MPW: Loaded 39513 existing records from primary storage. [2025-09-16T19:27:36.711215] [INFO ] [MAIN ] MPW: Loaded 39513 existing records. [2025-09-16T19:27:36.891794] [WARN ] [MAIN ] MPW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:27:36.892084] [INFO ] [MAIN ] MPW: Wiping existing primary data before full refresh. [2025-09-16T19:27:36.892131] [INFO ] [LOCAL ] MPW: Starting deletion of primary data. [2025-09-16T19:27:36.893487] [DEBUG] [LOCAL ] MPW: Deleted primary data file. [2025-09-16T19:27:36.893551] [INFO ] [MAIN ] MPW: Starting full primary refresh. [2025-09-16T19:27:37.053182] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-09-16T19:27:37.179983] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-04. [2025-09-16T19:27:37.283393] [INFO ] [REMOTE ] MPW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:27:37.974898] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-02. [2025-09-16T19:27:38.774893] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-07. [2025-09-16T19:27:39.169666] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-09-16T19:27:39.195801] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-09-16T19:27:39.574913] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-05. [2025-09-16T19:27:39.723282] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-09-16T19:27:40.374918] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-03. [2025-09-16T19:27:40.998958] [DEBUG] [REMOTE ] MPW: Error fetching primary data for 2025-03: Server request status code is 503. [2025-09-16T19:27:40.999277] [WARN ] [REMOTE ] MPW: No data returned for month 2025-03 [2025-09-16T19:27:41.174933] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-04. [2025-09-16T19:27:41.330297] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-09-16T19:27:41.974984] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-06. [2025-09-16T19:27:42.774933] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-06. [2025-09-16T19:27:43.186591] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-09-16T19:27:43.574877] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-05. [2025-09-16T19:27:43.659459] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T19:27:44.237291] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T19:27:44.374939] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-08. [2025-09-16T19:27:44.676300] [DEBUG] [REMOTE ] MPV: Error fetching primary data for 2025-08: Server request status code is 503. [2025-09-16T19:27:44.676621] [WARN ] [REMOTE ] MPV: No data returned for month 2025-08 [2025-09-16T19:27:45.161124] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-09-16T19:27:45.174870] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-06. [2025-09-16T19:27:45.974922] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-09. [2025-09-16T19:27:46.595297] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-09-16T19:27:46.596443] [INFO ] [REMOTE ] MPV: Full fetch complete (5250 records). [2025-09-16T19:27:46.596513] [DEBUG] [VALIDATOR ] MPV: Validating primary data. [2025-09-16T19:27:46.780461] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-07. [2025-09-16T19:27:46.786405] [WARN ] [VALIDATOR ] MPV: Validation failed - insufficient historical data. Have 129 unique days, need 150 days [2025-09-16T19:27:46.787831] [FATAL] [MAIN ] MPV: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:27:46.787891] [WARN ] [MAIN ] MPV: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:27:46.787938] [INFO ] [LOCAL ] MPV: Starting deletion of primary data. [2025-09-16T19:27:46.788006] [DEBUG] [LOCAL ] MPV: No primary data file found to delete. [2025-09-16T19:27:46.788046] [INFO ] [LOCAL ] MPV: Starting deletion of derivative data. [2025-09-16T19:27:46.788369] [DEBUG] [LOCAL ] MPV: Deleted derivative data file. [2025-09-16T19:27:46.788553] [INFO ] [MAIN ] Update Progress: 1392/2369 symbols completed. MPV took 19.6s. Estimated remaining: 1h 43m 41s. Rate: 9.4 symbols/min [2025-09-16T19:27:46.962601] [INFO ] [MAIN ] MPX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:27:46.967140] [DEBUG] [LOCAL ] MPX: Loaded 2844 existing records from primary storage. [2025-09-16T19:27:46.967221] [INFO ] [MAIN ] MPX: Loaded 2844 existing records. [2025-09-16T19:27:47.154183] [WARN ] [MAIN ] MPX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:27:47.154715] [INFO ] [MAIN ] MPX: Wiping existing primary data before full refresh. [2025-09-16T19:27:47.154930] [INFO ] [LOCAL ] MPX: Starting deletion of primary data. [2025-09-16T19:27:47.155392] [DEBUG] [LOCAL ] MPX: Deleted primary data file. [2025-09-16T19:27:47.155451] [INFO ] [MAIN ] MPX: Starting full primary refresh. [2025-09-16T19:27:47.458271] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-06 (2.28 seconds). [2025-09-16T19:27:47.581546] [INFO ] [REMOTE ] MPX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:27:47.581685] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-07. [2025-09-16T19:27:48.372066] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T19:27:48.381773] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-07. [2025-09-16T19:27:49.174907] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-08. [2025-09-16T19:27:49.493254] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-09-16T19:27:49.974931] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-02. [2025-09-16T19:27:50.275140] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-09-16T19:27:50.362876] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-09-16T19:27:50.712758] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T19:27:50.774932] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-03. [2025-09-16T19:27:51.386454] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-09-16T19:27:51.574912] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-04. [2025-09-16T19:27:52.154511] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-09-16T19:27:52.374966] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-05. [2025-09-16T19:27:52.847988] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-09-16T19:27:53.174912] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-09. [2025-09-16T19:27:53.974919] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-06. [2025-09-16T19:27:54.539619] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-09-16T19:27:54.774940] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-08. [2025-09-16T19:27:54.901672] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-09-16T19:27:54.907940] [INFO ] [REMOTE ] MPLX: Full fetch complete (55382 records). [2025-09-16T19:27:54.908045] [DEBUG] [VALIDATOR ] MPLX: Validating primary data. [2025-09-16T19:27:55.293239] [DEBUG] [VALIDATOR ] MPLX: All expected trading days are present. [2025-09-16T19:27:55.293319] [DEBUG] [VALIDATOR ] MPLX: Primary data validation passed. [2025-09-16T19:27:55.294139] [INFO ] [MAIN ] MPLX: Fetched primary data is valid. [2025-09-16T19:27:55.294203] [DEBUG] [LOCAL ] MPLX: Saving primary data. [2025-09-16T19:27:55.295301] [DEBUG] [LOCAL ] MPLX: Trimmed 55382 records to 37526 (keeping 150 days of data) [2025-09-16T19:27:55.527555] [DEBUG] [LOCAL ] MPLX: Primary data saved (37526 records). [2025-09-16T19:27:55.527737] [INFO ] [MAIN ] MPLX: Full history saved. [2025-09-16T19:27:55.528819] [INFO ] [MAIN ] Update Progress: 1393/2369 symbols completed. MPLX took 29.5s. Estimated remaining: 1h 43m 36s. Rate: 9.4 symbols/min [2025-09-16T19:27:55.585462] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-08. [2025-09-16T19:27:55.703155] [INFO ] [MAIN ] MQT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:27:55.710415] [DEBUG] [LOCAL ] MQT: Loaded 5280 existing records from primary storage. [2025-09-16T19:27:55.710551] [INFO ] [MAIN ] MQT: Loaded 5280 existing records. [2025-09-16T19:27:55.886093] [WARN ] [MAIN ] MQT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:27:55.886330] [INFO ] [MAIN ] MQT: Wiping existing primary data before full refresh. [2025-09-16T19:27:55.886378] [INFO ] [LOCAL ] MQT: Starting deletion of primary data. [2025-09-16T19:27:55.886743] [DEBUG] [LOCAL ] MQT: Deleted primary data file. [2025-09-16T19:27:55.886801] [INFO ] [MAIN ] MQT: Starting full primary refresh. [2025-09-16T19:27:56.241582] [INFO ] [REMOTE ] MQT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:27:56.374862] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-02. [2025-09-16T19:27:56.469556] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T19:27:56.831068] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-09-16T19:27:57.174940] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-03. [2025-09-16T19:27:57.974935] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-07. [2025-09-16T19:27:57.991918] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-09-16T19:27:58.317841] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-08 (2.73 seconds). [2025-09-16T19:27:58.569611] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-09-16T19:27:58.774929] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-04. [2025-09-16T19:27:59.574904] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-09. [2025-09-16T19:27:59.628055] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-09-16T19:28:00.374908] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-08. [2025-09-16T19:28:00.968680] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-09-16T19:28:01.174931] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-05. [2025-09-16T19:28:01.370130] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-09-16T19:28:01.376570] [INFO ] [REMOTE ] MPC: Full fetch complete (56772 records). [2025-09-16T19:28:01.376770] [DEBUG] [VALIDATOR ] MPC: Validating primary data. [2025-09-16T19:28:01.675435] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-09-16T19:28:01.772924] [DEBUG] [VALIDATOR ] MPC: All expected trading days are present. [2025-09-16T19:28:01.773014] [DEBUG] [VALIDATOR ] MPC: Primary data validation passed. [2025-09-16T19:28:01.774023] [INFO ] [MAIN ] MPC: Fetched primary data is valid. [2025-09-16T19:28:01.774115] [DEBUG] [LOCAL ] MPC: Saving primary data. [2025-09-16T19:28:01.775406] [DEBUG] [LOCAL ] MPC: Trimmed 56772 records to 38723 (keeping 150 days of data) [2025-09-16T19:28:01.996487] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-09. [2025-09-16T19:28:02.031483] [DEBUG] [LOCAL ] MPC: Primary data saved (38723 records). [2025-09-16T19:28:02.031617] [INFO ] [MAIN ] MPC: Full history saved. [2025-09-16T19:28:02.033248] [INFO ] [MAIN ] Update Progress: 1394/2369 symbols completed. MPC took 37.8s. Estimated remaining: 1h 43m 30s. Rate: 9.4 symbols/min [2025-09-16T19:28:02.200341] [INFO ] [MAIN ] MQY: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:02.216118] [DEBUG] [LOCAL ] MQY: Loaded 11648 existing records from primary storage. [2025-09-16T19:28:02.216247] [INFO ] [MAIN ] MQY: Loaded 11648 existing records. [2025-09-16T19:28:02.390544] [WARN ] [MAIN ] MQY: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:02.390810] [INFO ] [MAIN ] MQY: Wiping existing primary data before full refresh. [2025-09-16T19:28:02.390856] [INFO ] [LOCAL ] MQY: Starting deletion of primary data. [2025-09-16T19:28:02.391216] [DEBUG] [LOCAL ] MQY: Deleted primary data file. [2025-09-16T19:28:02.391260] [INFO ] [MAIN ] MQY: Starting full primary refresh. [2025-09-16T19:28:02.674422] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-09-16T19:28:02.681003] [INFO ] [REMOTE ] MPX: Full fetch complete (4786 records). [2025-09-16T19:28:02.691257] [DEBUG] [VALIDATOR ] MPX: Validating primary data. [2025-09-16T19:28:02.783314] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-06. [2025-09-16T19:28:02.822795] [INFO ] [REMOTE ] MQY: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:03.109213] [DEBUG] [VALIDATOR ] MPX: All expected trading days are present. [2025-09-16T19:28:03.109289] [DEBUG] [VALIDATOR ] MPX: Primary data validation passed. [2025-09-16T19:28:03.109429] [INFO ] [MAIN ] MPX: Fetched primary data is valid. [2025-09-16T19:28:03.109469] [DEBUG] [LOCAL ] MPX: Saving primary data. [2025-09-16T19:28:03.109857] [DEBUG] [LOCAL ] MPX: Trimmed 4786 records to 2879 (keeping 150 days of data) [2025-09-16T19:28:03.127689] [DEBUG] [LOCAL ] MPX: Primary data saved (2879 records). [2025-09-16T19:28:03.127755] [INFO ] [MAIN ] MPX: Full history saved. [2025-09-16T19:28:03.127842] [INFO ] [MAIN ] Update Progress: 1395/2369 symbols completed. MPX took 16.3s. Estimated remaining: 1h 43m 20s. Rate: 9.4 symbols/min [2025-09-16T19:28:03.299797] [INFO ] [MAIN ] MRC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:03.326276] [DEBUG] [LOCAL ] MRC: Loaded 26794 existing records from primary storage. [2025-09-16T19:28:03.326426] [INFO ] [MAIN ] MRC: Loaded 26794 existing records. [2025-09-16T19:28:03.505401] [WARN ] [MAIN ] MRC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:03.505637] [INFO ] [MAIN ] MRC: Wiping existing primary data before full refresh. [2025-09-16T19:28:03.505706] [INFO ] [LOCAL ] MRC: Starting deletion of primary data. [2025-09-16T19:28:03.506185] [DEBUG] [LOCAL ] MRC: Deleted primary data file. [2025-09-16T19:28:03.506230] [INFO ] [MAIN ] MRC: Starting full primary refresh. [2025-09-16T19:28:03.576834] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-02. [2025-09-16T19:28:03.615253] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-09-16T19:28:03.866795] [INFO ] [REMOTE ] MRC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:04.374900] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-02. [2025-09-16T19:28:04.489384] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-09-16T19:28:05.174948] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-07. [2025-09-16T19:28:05.875198] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-09-16T19:28:05.902837] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T19:28:05.974924] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-09. [2025-09-16T19:28:06.774910] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-08. [2025-09-16T19:28:07.492663] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-09-16T19:28:07.574992] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-03. [2025-09-16T19:28:07.912787] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-09-16T19:28:07.919346] [INFO ] [REMOTE ] MPW: Full fetch complete (49996 records). [2025-09-16T19:28:07.919468] [DEBUG] [VALIDATOR ] MPW: Validating primary data. [2025-09-16T19:28:08.106831] [WARN ] [VALIDATOR ] MPW: Validation failed - insufficient historical data. Have 129 unique days, need 150 days [2025-09-16T19:28:08.107369] [FATAL] [MAIN ] MPW: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:28:08.107419] [WARN ] [MAIN ] MPW: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:28:08.107459] [INFO ] [LOCAL ] MPW: Starting deletion of primary data. [2025-09-16T19:28:08.107519] [DEBUG] [LOCAL ] MPW: No primary data file found to delete. [2025-09-16T19:28:08.107570] [INFO ] [LOCAL ] MPW: Starting deletion of derivative data. [2025-09-16T19:28:08.107853] [DEBUG] [LOCAL ] MPW: Deleted derivative data file. [2025-09-16T19:28:08.107956] [INFO ] [MAIN ] Update Progress: 1396/2369 symbols completed. MPW took 31.7s. Estimated remaining: 1h 43m 13s. Rate: 9.4 symbols/min [2025-09-16T19:28:08.278176] [INFO ] [MAIN ] MRK: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:08.316536] [DEBUG] [LOCAL ] MRK: Loaded 39751 existing records from primary storage. [2025-09-16T19:28:08.316732] [INFO ] [MAIN ] MRK: Loaded 39751 existing records. [2025-09-16T19:28:08.380097] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-03. [2025-09-16T19:28:08.495053] [WARN ] [MAIN ] MRK: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:08.495293] [INFO ] [MAIN ] MRK: Wiping existing primary data before full refresh. [2025-09-16T19:28:08.495339] [INFO ] [LOCAL ] MRK: Starting deletion of primary data. [2025-09-16T19:28:08.496178] [DEBUG] [LOCAL ] MRK: Deleted primary data file. [2025-09-16T19:28:08.496239] [INFO ] [MAIN ] MRK: Starting full primary refresh. [2025-09-16T19:28:08.818140] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-09-16T19:28:08.885972] [INFO ] [REMOTE ] MRK: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:09.174911] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-02. [2025-09-16T19:28:09.665547] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-09-16T19:28:09.974935] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-09. [2025-09-16T19:28:10.739785] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-09-16T19:28:10.741628] [INFO ] [REMOTE ] MQT: Full fetch complete (9341 records). [2025-09-16T19:28:10.741750] [DEBUG] [VALIDATOR ] MQT: Validating primary data. [2025-09-16T19:28:10.784867] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-04. [2025-09-16T19:28:11.105131] [DEBUG] [VALIDATOR ] MQT: All expected trading days are present. [2025-09-16T19:28:11.105221] [DEBUG] [VALIDATOR ] MQT: Primary data validation passed. [2025-09-16T19:28:11.105450] [INFO ] [MAIN ] MQT: Fetched primary data is valid. [2025-09-16T19:28:11.105489] [DEBUG] [LOCAL ] MQT: Saving primary data. [2025-09-16T19:28:11.105921] [DEBUG] [LOCAL ] MQT: Trimmed 9341 records to 5341 (keeping 150 days of data) [2025-09-16T19:28:11.142501] [DEBUG] [LOCAL ] MQT: Primary data saved (5341 records). [2025-09-16T19:28:11.142765] [INFO ] [MAIN ] MQT: Full history saved. [2025-09-16T19:28:11.142871] [INFO ] [MAIN ] Update Progress: 1397/2369 symbols completed. MQT took 15.6s. Estimated remaining: 1h 43m 4s. Rate: 9.4 symbols/min [2025-09-16T19:28:11.222763] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-09-16T19:28:11.494583] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-09-16T19:28:11.574865] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-05. [2025-09-16T19:28:12.374905] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-03. [2025-09-16T19:28:12.578263] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-09-16T19:28:13.174949] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-04. [2025-09-16T19:28:13.975002] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-06. [2025-09-16T19:28:14.306735] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-09-16T19:28:14.631663] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-09-16T19:28:14.658801] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T19:28:14.774886] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-05. [2025-09-16T19:28:15.574945] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-07. [2025-09-16T19:28:16.178839] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T19:28:16.374938] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-04. [2025-09-16T19:28:16.526506] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-09-16T19:28:17.174904] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-08. [2025-09-16T19:28:17.925380] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T19:28:17.974854] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-09. [2025-09-16T19:28:18.620733] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-04 (2.25 seconds). [2025-09-16T19:28:18.695659] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-09-16T19:28:18.698587] [INFO ] [REMOTE ] MQY: Full fetch complete (18324 records). [2025-09-16T19:28:18.698689] [DEBUG] [VALIDATOR ] MQY: Validating primary data. [2025-09-16T19:28:18.779974] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-06. [2025-09-16T19:28:19.094855] [DEBUG] [VALIDATOR ] MQY: All expected trading days are present. [2025-09-16T19:28:19.094949] [DEBUG] [VALIDATOR ] MQY: Primary data validation passed. [2025-09-16T19:28:19.095379] [INFO ] [MAIN ] MQY: Fetched primary data is valid. [2025-09-16T19:28:19.095427] [DEBUG] [LOCAL ] MQY: Saving primary data. [2025-09-16T19:28:19.096021] [DEBUG] [LOCAL ] MQY: Trimmed 18324 records to 11779 (keeping 150 days of data) [2025-09-16T19:28:19.169931] [DEBUG] [LOCAL ] MQY: Primary data saved (11779 records). [2025-09-16T19:28:19.170076] [INFO ] [MAIN ] MQY: Full history saved. [2025-09-16T19:28:19.170164] [INFO ] [MAIN ] Update Progress: 1398/2369 symbols completed. MQY took 17.1s. Estimated remaining: 1h 42m 59s. Rate: 9.4 symbols/min [2025-09-16T19:28:19.574906] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-05. [2025-09-16T19:28:19.811059] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-09-16T19:28:20.374913] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-07. [2025-09-16T19:28:21.643976] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-09-16T19:28:21.653568] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-08. [2025-09-16T19:28:21.920540] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-05 (2.35 seconds). [2025-09-16T19:28:21.974912] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-06. [2025-09-16T19:28:22.939332] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-09-16T19:28:22.947259] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-09. [2025-09-16T19:28:24.127443] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-06 (2.15 seconds). [2025-09-16T19:28:24.139554] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-07. [2025-09-16T19:28:24.164771] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-09-16T19:28:24.169092] [INFO ] [REMOTE ] MRC: Full fetch complete (41079 records). [2025-09-16T19:28:24.169189] [DEBUG] [VALIDATOR ] MRC: Validating primary data. [2025-09-16T19:28:24.567309] [DEBUG] [VALIDATOR ] MRC: All expected trading days are present. [2025-09-16T19:28:24.567399] [DEBUG] [VALIDATOR ] MRC: Primary data validation passed. [2025-09-16T19:28:24.568108] [INFO ] [MAIN ] MRC: Fetched primary data is valid. [2025-09-16T19:28:24.568158] [DEBUG] [LOCAL ] MRC: Saving primary data. [2025-09-16T19:28:24.569252] [DEBUG] [LOCAL ] MRC: Trimmed 41079 records to 27104 (keeping 150 days of data) [2025-09-16T19:28:24.739396] [DEBUG] [LOCAL ] MRC: Primary data saved (27104 records). [2025-09-16T19:28:24.739554] [INFO ] [MAIN ] MRC: Full history saved. [2025-09-16T19:28:24.739751] [INFO ] [MAIN ] Update Progress: 1399/2369 symbols completed. MRC took 21.6s. Estimated remaining: 1h 42m 52s. Rate: 9.4 symbols/min [2025-09-16T19:28:26.474101] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-07 (2.33 seconds). [2025-09-16T19:28:26.485969] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-08. [2025-09-16T19:28:28.592729] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-09-16T19:28:28.604288] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-09. [2025-09-16T19:28:30.776942] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-09-16T19:28:30.789845] [INFO ] [REMOTE ] MRK: Full fetch complete (58469 records). [2025-09-16T19:28:30.790091] [DEBUG] [VALIDATOR ] MRK: Validating primary data. [2025-09-16T19:28:31.191453] [DEBUG] [VALIDATOR ] MRK: All expected trading days are present. [2025-09-16T19:28:31.191531] [DEBUG] [VALIDATOR ] MRK: Primary data validation passed. [2025-09-16T19:28:31.192372] [INFO ] [MAIN ] MRK: Fetched primary data is valid. [2025-09-16T19:28:31.192419] [DEBUG] [LOCAL ] MRK: Saving primary data. [2025-09-16T19:28:31.193705] [DEBUG] [LOCAL ] MRK: Trimmed 58469 records to 40141 (keeping 150 days of data) [2025-09-16T19:28:31.441748] [DEBUG] [LOCAL ] MRK: Primary data saved (40141 records). [2025-09-16T19:28:31.441874] [INFO ] [MAIN ] MRK: Full history saved. [2025-09-16T19:28:31.441980] [INFO ] [MAIN ] Update Progress: 1400/2369 symbols completed. MRK took 23.3s. Estimated remaining: 1h 42m 46s. Rate: 9.4 symbols/min [2025-09-16T19:28:32.116121] [INFO ] [MAIN ] MS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:32.117911] [INFO ] [MAIN ] MRP: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:32.204307] [INFO ] [MAIN ] MSB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:32.209824] [INFO ] [MAIN ] MSA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:32.221565] [DEBUG] [LOCAL ] MSB: Loaded 5948 existing records from primary storage. [2025-09-16T19:28:32.221837] [INFO ] [MAIN ] MSB: Loaded 5948 existing records. [2025-09-16T19:28:32.347736] [DEBUG] [LOCAL ] MSA: Loaded 13825 existing records from primary storage. [2025-09-16T19:28:32.357287] [INFO ] [MAIN ] MSA: Loaded 13825 existing records. [2025-09-16T19:28:32.526078] [DEBUG] [LOCAL ] MRP: Loaded 34180 existing records from primary storage. [2025-09-16T19:28:32.529861] [WARN ] [MAIN ] MSB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:32.545542] [INFO ] [MAIN ] MRP: Loaded 34180 existing records. [2025-09-16T19:28:32.556728] [DEBUG] [LOCAL ] MS: Loaded 39657 existing records from primary storage. [2025-09-16T19:28:32.597043] [INFO ] [MAIN ] MSB: Wiping existing primary data before full refresh. [2025-09-16T19:28:32.597270] [INFO ] [MAIN ] MS: Loaded 39657 existing records. [2025-09-16T19:28:32.608115] [INFO ] [LOCAL ] MSB: Starting deletion of primary data. [2025-09-16T19:28:32.657142] [DEBUG] [LOCAL ] MSB: Deleted primary data file. [2025-09-16T19:28:32.685438] [INFO ] [MAIN ] MSB: Starting full primary refresh. [2025-09-16T19:28:32.917507] [WARN ] [MAIN ] MSA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:32.987377] [INFO ] [MAIN ] MSA: Wiping existing primary data before full refresh. [2025-09-16T19:28:33.018983] [INFO ] [LOCAL ] MSA: Starting deletion of primary data. [2025-09-16T19:28:33.048961] [WARN ] [MAIN ] MRP: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:33.092158] [DEBUG] [LOCAL ] MSA: Deleted primary data file. [2025-09-16T19:28:33.114891] [INFO ] [MAIN ] MSA: Starting full primary refresh. [2025-09-16T19:28:33.115018] [WARN ] [MAIN ] MS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:33.115156] [INFO ] [MAIN ] MRP: Wiping existing primary data before full refresh. [2025-09-16T19:28:33.142030] [INFO ] [MAIN ] MS: Wiping existing primary data before full refresh. [2025-09-16T19:28:33.156420] [INFO ] [LOCAL ] MRP: Starting deletion of primary data. [2025-09-16T19:28:33.161606] [INFO ] [LOCAL ] MS: Starting deletion of primary data. [2025-09-16T19:28:33.187356] [DEBUG] [LOCAL ] MS: Deleted primary data file. [2025-09-16T19:28:33.197663] [DEBUG] [LOCAL ] MRP: Deleted primary data file. [2025-09-16T19:28:33.203341] [INFO ] [MAIN ] MRP: Starting full primary refresh. [2025-09-16T19:28:33.220695] [INFO ] [MAIN ] MS: Starting full primary refresh. [2025-09-16T19:28:34.221792] [INFO ] [REMOTE ] MSB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:34.254945] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-02. [2025-09-16T19:28:34.428257] [INFO ] [REMOTE ] MSA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:34.461743] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-02. [2025-09-16T19:28:34.465829] [INFO ] [REMOTE ] MRP: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:34.491331] [INFO ] [REMOTE ] MRP: Fetching primary data for 2025-02. [2025-09-16T19:28:34.539944] [INFO ] [REMOTE ] MS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:34.540509] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-02. [2025-09-16T19:28:35.285343] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-09-16T19:28:35.288198] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-03. [2025-09-16T19:28:35.540939] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-09-16T19:28:35.545076] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-03. [2025-09-16T19:28:35.981254] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T19:28:35.995833] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-03. [2025-09-16T19:28:36.014982] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-09-16T19:28:36.018256] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-04. [2025-09-16T19:28:36.049279] [INFO ] [REMOTE ] MRP: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-09-16T19:28:36.059259] [INFO ] [REMOTE ] MRP: Fetching primary data for 2025-03. [2025-09-16T19:28:36.450282] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-09-16T19:28:36.454800] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-04. [2025-09-16T19:28:36.502404] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-09-16T19:28:36.506125] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-05. [2025-09-16T19:28:36.985139] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-09-16T19:28:36.987517] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-06. [2025-09-16T19:28:37.358599] [INFO ] [REMOTE ] MRP: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-09-16T19:28:37.369823] [INFO ] [REMOTE ] MRP: Fetching primary data for 2025-04. [2025-09-16T19:28:37.385138] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T19:28:37.390545] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-05. [2025-09-16T19:28:37.411279] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-09-16T19:28:37.413575] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-07. [2025-09-16T19:28:38.063000] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-09-16T19:28:38.074623] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-04. [2025-09-16T19:28:38.280009] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T19:28:38.472969] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-09-16T19:28:38.707805] [INFO ] [REMOTE ] MRP: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-09-16T19:28:38.774895] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-08. [2025-09-16T19:28:39.537903] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-09-16T19:28:39.574917] [INFO ] [REMOTE ] MRP: Fetching primary data for 2025-05. [2025-09-16T19:28:40.202998] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-04 (2.13 seconds). [2025-09-16T19:28:40.374958] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-05. [2025-09-16T19:28:40.803731] [INFO ] [REMOTE ] MRP: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-09-16T19:28:41.174919] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-09. [2025-09-16T19:28:41.902369] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T19:28:41.903678] [INFO ] [REMOTE ] MSB: Full fetch complete (9134 records). [2025-09-16T19:28:41.903773] [DEBUG] [VALIDATOR ] MSB: Validating primary data. [2025-09-16T19:28:41.979990] [INFO ] [REMOTE ] MRP: Fetching primary data for 2025-06. [2025-09-16T19:28:42.286078] [DEBUG] [VALIDATOR ] MSB: All expected trading days are present. [2025-09-16T19:28:42.286206] [DEBUG] [VALIDATOR ] MSB: Primary data validation passed. [2025-09-16T19:28:42.286507] [INFO ] [MAIN ] MSB: Fetched primary data is valid. [2025-09-16T19:28:42.286605] [DEBUG] [LOCAL ] MSB: Saving primary data. [2025-09-16T19:28:42.287244] [DEBUG] [LOCAL ] MSB: Trimmed 9134 records to 5977 (keeping 150 days of data) [2025-09-16T19:28:42.325623] [DEBUG] [LOCAL ] MSB: Primary data saved (5977 records). [2025-09-16T19:28:42.325876] [INFO ] [MAIN ] MSB: Full history saved. [2025-09-16T19:28:42.326054] [INFO ] [MAIN ] Update Progress: 1401/2369 symbols completed. MSB took 10.9s. Estimated remaining: 1h 42m 43s. Rate: 9.4 symbols/min [2025-09-16T19:28:42.525587] [INFO ] [MAIN ] MSC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:42.528588] [DEBUG] [LOCAL ] MSC: No existing data file found in primary storage. [2025-09-16T19:28:42.528662] [WARN ] [MAIN ] MSC: No existing data found, performing full refresh instead. [2025-09-16T19:28:42.530588] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-05 (2.16 seconds). [2025-09-16T19:28:42.529366] [INFO ] [MAIN ] MSC: Wiping existing primary data before full refresh. [2025-09-16T19:28:42.541166] [INFO ] [LOCAL ] MSC: Starting deletion of primary data. [2025-09-16T19:28:42.541299] [DEBUG] [LOCAL ] MSC: No primary data file found to delete. [2025-09-16T19:28:42.541481] [INFO ] [MAIN ] MSC: Starting full primary refresh. [2025-09-16T19:28:42.782738] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-06. [2025-09-16T19:28:42.903356] [INFO ] [REMOTE ] MSC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:43.509674] [INFO ] [REMOTE ] MRP: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-09-16T19:28:43.574918] [INFO ] [REMOTE ] MRP: Fetching primary data for 2025-07. [2025-09-16T19:28:44.374907] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-02. [2025-09-16T19:28:44.795508] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-06 (2.01 seconds). [2025-09-16T19:28:44.912574] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-09-16T19:28:45.174871] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-07. [2025-09-16T19:28:45.217734] [INFO ] [REMOTE ] MRP: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-09-16T19:28:45.974907] [INFO ] [REMOTE ] MRP: Fetching primary data for 2025-08. [2025-09-16T19:28:46.774861] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-03. [2025-09-16T19:28:47.199793] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-09-16T19:28:47.298237] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-09-16T19:28:47.565522] [INFO ] [REMOTE ] MRP: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-09-16T19:28:47.574880] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-04. [2025-09-16T19:28:47.893165] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-04 (0.32 seconds). [2025-09-16T19:28:48.374931] [INFO ] [REMOTE ] MRP: Fetching primary data for 2025-09. [2025-09-16T19:28:49.174976] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-05. [2025-09-16T19:28:49.504116] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-09-16T19:28:49.974427] [INFO ] [REMOTE ] MRP: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-09-16T19:28:49.979661] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-08. [2025-09-16T19:28:49.981625] [INFO ] [REMOTE ] MRP: Full fetch complete (51748 records). [2025-09-16T19:28:49.981726] [DEBUG] [VALIDATOR ] MRP: Validating primary data. [2025-09-16T19:28:50.359885] [DEBUG] [VALIDATOR ] MRP: All expected trading days are present. [2025-09-16T19:28:50.359975] [DEBUG] [VALIDATOR ] MRP: Primary data validation passed. [2025-09-16T19:28:50.360890] [INFO ] [MAIN ] MRP: Fetched primary data is valid. [2025-09-16T19:28:50.360940] [DEBUG] [LOCAL ] MRP: Saving primary data. [2025-09-16T19:28:50.362231] [DEBUG] [LOCAL ] MRP: Trimmed 51748 records to 34515 (keeping 150 days of data) [2025-09-16T19:28:50.571625] [DEBUG] [LOCAL ] MRP: Primary data saved (34515 records). [2025-09-16T19:28:50.571781] [INFO ] [MAIN ] MRP: Full history saved. [2025-09-16T19:28:50.571880] [INFO ] [MAIN ] Update Progress: 1402/2369 symbols completed. MRP took 19.1s. Estimated remaining: 1h 42m 38s. Rate: 9.4 symbols/min [2025-09-16T19:28:50.758473] [INFO ] [MAIN ] MSCI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:50.774910] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-06. [2025-09-16T19:28:50.785113] [DEBUG] [LOCAL ] MSCI: Loaded 25068 existing records from primary storage. [2025-09-16T19:28:50.785253] [INFO ] [MAIN ] MSCI: Loaded 25068 existing records. [2025-09-16T19:28:50.972843] [WARN ] [MAIN ] MSCI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:50.973102] [INFO ] [MAIN ] MSCI: Wiping existing primary data before full refresh. [2025-09-16T19:28:50.973171] [INFO ] [LOCAL ] MSCI: Starting deletion of primary data. [2025-09-16T19:28:50.973849] [DEBUG] [LOCAL ] MSCI: Deleted primary data file. [2025-09-16T19:28:50.973898] [INFO ] [MAIN ] MSCI: Starting full primary refresh. [2025-09-16T19:28:51.234235] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-09-16T19:28:51.345683] [INFO ] [REMOTE ] MSCI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:51.574900] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-02. [2025-09-16T19:28:52.087950] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-09-16T19:28:52.374928] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-07. [2025-09-16T19:28:52.821579] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-09-16T19:28:53.113234] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-09-16T19:28:53.174905] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-03. [2025-09-16T19:28:53.974893] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-09. [2025-09-16T19:28:54.782140] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-06. [2025-09-16T19:28:54.787949] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-09-16T19:28:55.574925] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-08. [2025-09-16T19:28:55.594128] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-09-16T19:28:55.602925] [INFO ] [REMOTE ] MS: Full fetch complete (58374 records). [2025-09-16T19:28:55.603156] [DEBUG] [VALIDATOR ] MS: Validating primary data. [2025-09-16T19:28:55.900399] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-09-16T19:28:56.031670] [DEBUG] [VALIDATOR ] MS: All expected trading days are present. [2025-09-16T19:28:56.031837] [DEBUG] [VALIDATOR ] MS: Primary data validation passed. [2025-09-16T19:28:56.033173] [INFO ] [MAIN ] MS: Fetched primary data is valid. [2025-09-16T19:28:56.033742] [DEBUG] [LOCAL ] MS: Saving primary data. [2025-09-16T19:28:56.033659] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-09-16T19:28:56.038913] [DEBUG] [LOCAL ] MS: Trimmed 58374 records to 40047 (keeping 150 days of data) [2025-09-16T19:28:56.295955] [DEBUG] [LOCAL ] MS: Primary data saved (40047 records). [2025-09-16T19:28:56.296082] [INFO ] [MAIN ] MS: Full history saved. [2025-09-16T19:28:56.296173] [INFO ] [MAIN ] Update Progress: 1403/2369 symbols completed. MS took 24.9s. Estimated remaining: 1h 42m 31s. Rate: 9.4 symbols/min [2025-09-16T19:28:56.380098] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-07. [2025-09-16T19:28:56.467988] [INFO ] [MAIN ] MSD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:56.479874] [DEBUG] [LOCAL ] MSD: Loaded 11977 existing records from primary storage. [2025-09-16T19:28:56.479997] [INFO ] [MAIN ] MSD: Loaded 11977 existing records. [2025-09-16T19:28:56.652199] [WARN ] [MAIN ] MSD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:56.652475] [INFO ] [MAIN ] MSD: Wiping existing primary data before full refresh. [2025-09-16T19:28:56.652520] [INFO ] [LOCAL ] MSD: Starting deletion of primary data. [2025-09-16T19:28:56.652889] [DEBUG] [LOCAL ] MSD: Deleted primary data file. [2025-09-16T19:28:56.652936] [INFO ] [MAIN ] MSD: Starting full primary refresh. [2025-09-16T19:28:57.000317] [INFO ] [REMOTE ] MSD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:57.175017] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-09. [2025-09-16T19:28:57.547110] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-09-16T19:28:57.599071] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-09-16T19:28:57.600636] [INFO ] [REMOTE ] MSC: Full fetch complete (1883 records). [2025-09-16T19:28:57.600744] [DEBUG] [VALIDATOR ] MSC: Validating primary data. [2025-09-16T19:28:57.801833] [WARN ] [VALIDATOR ] MSC: Validation failed - insufficient historical data. Have 143 unique days, need 150 days [2025-09-16T19:28:57.802130] [FATAL] [MAIN ] MSC: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:28:57.802220] [WARN ] [MAIN ] MSC: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:28:57.802288] [INFO ] [LOCAL ] MSC: Starting deletion of primary data. [2025-09-16T19:28:57.802379] [DEBUG] [LOCAL ] MSC: No primary data file found to delete. [2025-09-16T19:28:57.802436] [INFO ] [LOCAL ] MSC: Starting deletion of derivative data. [2025-09-16T19:28:57.802506] [DEBUG] [LOCAL ] MSC: No derivative data file found to delete. [2025-09-16T19:28:57.802593] [INFO ] [MAIN ] Update Progress: 1404/2369 symbols completed. MSC took 15.5s. Estimated remaining: 1h 42m 21s. Rate: 9.4 symbols/min [2025-09-16T19:28:57.974169] [INFO ] [MAIN ] MSDL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:28:57.974904] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-08. [2025-09-16T19:28:58.002260] [DEBUG] [LOCAL ] MSDL: Loaded 27074 existing records from primary storage. [2025-09-16T19:28:58.002403] [INFO ] [MAIN ] MSDL: Loaded 27074 existing records. [2025-09-16T19:28:58.181202] [WARN ] [MAIN ] MSDL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:28:58.181480] [INFO ] [MAIN ] MSDL: Wiping existing primary data before full refresh. [2025-09-16T19:28:58.181528] [INFO ] [LOCAL ] MSDL: Starting deletion of primary data. [2025-09-16T19:28:58.182305] [DEBUG] [LOCAL ] MSDL: Deleted primary data file. [2025-09-16T19:28:58.182357] [INFO ] [MAIN ] MSDL: Starting full primary refresh. [2025-09-16T19:28:58.540912] [INFO ] [REMOTE ] MSDL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:28:58.774904] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-02. [2025-09-16T19:28:59.027196] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T19:28:59.574938] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-09. [2025-09-16T19:29:00.211897] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-09-16T19:29:00.374850] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-02. [2025-09-16T19:29:00.685312] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-09-16T19:29:00.687979] [INFO ] [REMOTE ] MSA: Full fetch complete (20281 records). [2025-09-16T19:29:00.688053] [DEBUG] [VALIDATOR ] MSA: Validating primary data. [2025-09-16T19:29:01.052142] [DEBUG] [VALIDATOR ] MSA: All expected trading days are present. [2025-09-16T19:29:01.052228] [DEBUG] [VALIDATOR ] MSA: Primary data validation passed. [2025-09-16T19:29:01.052635] [INFO ] [MAIN ] MSA: Fetched primary data is valid. [2025-09-16T19:29:01.052695] [DEBUG] [LOCAL ] MSA: Saving primary data. [2025-09-16T19:29:01.053296] [DEBUG] [LOCAL ] MSA: Trimmed 20281 records to 13942 (keeping 150 days of data) [2025-09-16T19:29:01.148839] [DEBUG] [LOCAL ] MSA: Primary data saved (13942 records). [2025-09-16T19:29:01.148987] [INFO ] [MAIN ] MSA: Full history saved. [2025-09-16T19:29:01.149071] [INFO ] [MAIN ] Update Progress: 1405/2369 symbols completed. MSA took 29.7s. Estimated remaining: 1h 42m 13s. Rate: 9.4 symbols/min [2025-09-16T19:29:01.180116] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-03. [2025-09-16T19:29:01.323382] [INFO ] [MAIN ] MSGE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:29:01.342467] [DEBUG] [LOCAL ] MSGE: Loaded 19047 existing records from primary storage. [2025-09-16T19:29:01.342607] [INFO ] [MAIN ] MSGE: Loaded 19047 existing records. [2025-09-16T19:29:01.510589] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-09-16T19:29:01.532542] [WARN ] [MAIN ] MSGE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:29:01.532754] [INFO ] [MAIN ] MSGE: Wiping existing primary data before full refresh. [2025-09-16T19:29:01.532801] [INFO ] [LOCAL ] MSGE: Starting deletion of primary data. [2025-09-16T19:29:01.533328] [DEBUG] [LOCAL ] MSGE: Deleted primary data file. [2025-09-16T19:29:01.533378] [INFO ] [MAIN ] MSGE: Starting full primary refresh. [2025-09-16T19:29:01.881967] [INFO ] [REMOTE ] MSGE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:29:01.974855] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-02. [2025-09-16T19:29:02.539797] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-09-16T19:29:02.774938] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-04. [2025-09-16T19:29:02.883949] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-09-16T19:29:03.574903] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-03. [2025-09-16T19:29:04.251511] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-09-16T19:29:04.374940] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-04. [2025-09-16T19:29:04.845751] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T19:29:05.174974] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-03. [2025-09-16T19:29:05.870334] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-09-16T19:29:05.974938] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-04. [2025-09-16T19:29:06.111841] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T19:29:06.774932] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-05. [2025-09-16T19:29:07.291478] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T19:29:07.574950] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-05. [2025-09-16T19:29:08.094136] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-09-16T19:29:08.374868] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-06. [2025-09-16T19:29:08.790809] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-09-16T19:29:09.174908] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-06. [2025-09-16T19:29:09.696111] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T19:29:09.974885] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-07. [2025-09-16T19:29:10.462554] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T19:29:10.774894] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-07. [2025-09-16T19:29:11.307011] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-09-16T19:29:11.574943] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-08. [2025-09-16T19:29:11.921345] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-09-16T19:29:12.374916] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-05. [2025-09-16T19:29:13.129913] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-09-16T19:29:13.174865] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-09. [2025-09-16T19:29:13.813032] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-09-16T19:29:13.974945] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-08. [2025-09-16T19:29:14.607512] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-09-16T19:29:14.612573] [INFO ] [REMOTE ] MSCI: Full fetch complete (38002 records). [2025-09-16T19:29:14.612694] [DEBUG] [VALIDATOR ] MSCI: Validating primary data. [2025-09-16T19:29:14.783979] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-04. [2025-09-16T19:29:14.985511] [DEBUG] [VALIDATOR ] MSCI: All expected trading days are present. [2025-09-16T19:29:14.985603] [DEBUG] [VALIDATOR ] MSCI: Primary data validation passed. [2025-09-16T19:29:14.986319] [INFO ] [MAIN ] MSCI: Fetched primary data is valid. [2025-09-16T19:29:14.986367] [DEBUG] [LOCAL ] MSCI: Saving primary data. [2025-09-16T19:29:14.987437] [DEBUG] [LOCAL ] MSCI: Trimmed 38002 records to 25367 (keeping 150 days of data) [2025-09-16T19:29:15.150267] [DEBUG] [LOCAL ] MSCI: Primary data saved (25367 records). [2025-09-16T19:29:15.150413] [INFO ] [MAIN ] MSCI: Full history saved. [2025-09-16T19:29:15.150666] [INFO ] [MAIN ] Update Progress: 1406/2369 symbols completed. MSCI took 24.6s. Estimated remaining: 1h 42m 11s. Rate: 9.4 symbols/min [2025-09-16T19:29:15.290349] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-09-16T19:29:15.344400] [INFO ] [MAIN ] MSGS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:29:15.357257] [DEBUG] [LOCAL ] MSGS: Loaded 12220 existing records from primary storage. [2025-09-16T19:29:15.357378] [INFO ] [MAIN ] MSGS: Loaded 12220 existing records. [2025-09-16T19:29:15.538277] [WARN ] [MAIN ] MSGS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:29:15.538494] [INFO ] [MAIN ] MSGS: Wiping existing primary data before full refresh. [2025-09-16T19:29:15.538538] [INFO ] [LOCAL ] MSGS: Starting deletion of primary data. [2025-09-16T19:29:15.538933] [DEBUG] [LOCAL ] MSGS: Deleted primary data file. [2025-09-16T19:29:15.538979] [INFO ] [MAIN ] MSGS: Starting full primary refresh. [2025-09-16T19:29:15.581616] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-09. [2025-09-16T19:29:15.888788] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-09-16T19:29:15.924309] [INFO ] [REMOTE ] MSGS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:29:16.374911] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-02. [2025-09-16T19:29:16.555055] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-09-16T19:29:16.558785] [INFO ] [REMOTE ] MSGE: Full fetch complete (29247 records). [2025-09-16T19:29:16.558866] [DEBUG] [VALIDATOR ] MSGE: Validating primary data. [2025-09-16T19:29:16.940471] [DEBUG] [VALIDATOR ] MSGE: All expected trading days are present. [2025-09-16T19:29:16.940550] [DEBUG] [VALIDATOR ] MSGE: Primary data validation passed. [2025-09-16T19:29:16.941203] [INFO ] [MAIN ] MSGE: Fetched primary data is valid. [2025-09-16T19:29:16.941247] [DEBUG] [LOCAL ] MSGE: Saving primary data. [2025-09-16T19:29:16.941978] [DEBUG] [LOCAL ] MSGE: Trimmed 29247 records to 19265 (keeping 150 days of data) [2025-09-16T19:29:17.054798] [DEBUG] [LOCAL ] MSGE: Primary data saved (19265 records). [2025-09-16T19:29:17.054929] [INFO ] [MAIN ] MSGE: Full history saved. [2025-09-16T19:29:17.055008] [INFO ] [MAIN ] Update Progress: 1407/2369 symbols completed. MSGE took 15.9s. Estimated remaining: 1h 42m 2s. Rate: 9.4 symbols/min [2025-09-16T19:29:17.179994] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-06. [2025-09-16T19:29:17.247007] [INFO ] [MAIN ] MSI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:29:17.284238] [DEBUG] [LOCAL ] MSI: Loaded 29648 existing records from primary storage. [2025-09-16T19:29:17.284406] [INFO ] [MAIN ] MSI: Loaded 29648 existing records. [2025-09-16T19:29:17.474301] [WARN ] [MAIN ] MSI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:29:17.475245] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-09-16T19:29:17.475522] [INFO ] [MAIN ] MSI: Wiping existing primary data before full refresh. [2025-09-16T19:29:17.478990] [INFO ] [LOCAL ] MSI: Starting deletion of primary data. [2025-09-16T19:29:17.479826] [DEBUG] [LOCAL ] MSI: Deleted primary data file. [2025-09-16T19:29:17.479902] [INFO ] [MAIN ] MSI: Starting full primary refresh. [2025-09-16T19:29:17.826949] [INFO ] [REMOTE ] MSI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:29:17.974888] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-02. [2025-09-16T19:29:18.664608] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-09-16T19:29:18.774853] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-07. [2025-09-16T19:29:19.466718] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-09-16T19:29:19.574988] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-03. [2025-09-16T19:29:20.262949] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-09-16T19:29:20.330141] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-09-16T19:29:20.374966] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-08. [2025-09-16T19:29:21.174934] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-05. [2025-09-16T19:29:21.881237] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-09-16T19:29:21.904318] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T19:29:21.974887] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-03. [2025-09-16T19:29:22.774900] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-06. [2025-09-16T19:29:23.458670] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-09-16T19:29:23.574886] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-07. [2025-09-16T19:29:23.619496] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-09-16T19:29:24.374882] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-09. [2025-09-16T19:29:24.683099] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-09-16T19:29:25.174946] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-08. [2025-09-16T19:29:25.627997] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-09-16T19:29:25.632824] [INFO ] [REMOTE ] MSDL: Full fetch complete (41569 records). [2025-09-16T19:29:25.632895] [DEBUG] [VALIDATOR ] MSDL: Validating primary data. [2025-09-16T19:29:25.982707] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-04. [2025-09-16T19:29:26.009883] [DEBUG] [VALIDATOR ] MSDL: All expected trading days are present. [2025-09-16T19:29:26.009960] [DEBUG] [VALIDATOR ] MSDL: Primary data validation passed. [2025-09-16T19:29:26.010663] [INFO ] [MAIN ] MSDL: Fetched primary data is valid. [2025-09-16T19:29:26.010707] [DEBUG] [LOCAL ] MSDL: Saving primary data. [2025-09-16T19:29:26.011672] [DEBUG] [LOCAL ] MSDL: Trimmed 41569 records to 27398 (keeping 150 days of data) [2025-09-16T19:29:26.190398] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-09-16T19:29:26.201292] [DEBUG] [LOCAL ] MSDL: Primary data saved (27398 records). [2025-09-16T19:29:26.206893] [INFO ] [MAIN ] MSDL: Full history saved. [2025-09-16T19:29:26.207009] [INFO ] [MAIN ] Update Progress: 1408/2369 symbols completed. MSDL took 28.4s. Estimated remaining: 1h 41m 58s. Rate: 9.4 symbols/min [2025-09-16T19:29:26.377833] [INFO ] [MAIN ] MSIF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:29:26.389382] [DEBUG] [LOCAL ] MSIF: Loaded 10557 existing records from primary storage. [2025-09-16T19:29:26.389513] [INFO ] [MAIN ] MSIF: Loaded 10557 existing records. [2025-09-16T19:29:26.566117] [WARN ] [MAIN ] MSIF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:29:26.566373] [INFO ] [MAIN ] MSIF: Wiping existing primary data before full refresh. [2025-09-16T19:29:26.566419] [INFO ] [LOCAL ] MSIF: Starting deletion of primary data. [2025-09-16T19:29:26.566878] [DEBUG] [LOCAL ] MSIF: Deleted primary data file. [2025-09-16T19:29:26.566931] [INFO ] [MAIN ] MSIF: Starting full primary refresh. [2025-09-16T19:29:26.780101] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-04. [2025-09-16T19:29:26.927519] [INFO ] [REMOTE ] MSIF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:29:27.574950] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-09. [2025-09-16T19:29:27.626280] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-09-16T19:29:27.916147] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-09-16T19:29:28.374910] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-05. [2025-09-16T19:29:28.438144] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-09-16T19:29:28.440898] [INFO ] [REMOTE ] MSD: Full fetch complete (16671 records). [2025-09-16T19:29:28.440971] [DEBUG] [VALIDATOR ] MSD: Validating primary data. [2025-09-16T19:29:28.815193] [DEBUG] [VALIDATOR ] MSD: All expected trading days are present. [2025-09-16T19:29:28.815269] [DEBUG] [VALIDATOR ] MSD: Primary data validation passed. [2025-09-16T19:29:28.815620] [INFO ] [MAIN ] MSD: Fetched primary data is valid. [2025-09-16T19:29:28.815677] [DEBUG] [LOCAL ] MSD: Saving primary data. [2025-09-16T19:29:28.816274] [DEBUG] [LOCAL ] MSD: Trimmed 16671 records to 12137 (keeping 150 days of data) [2025-09-16T19:29:28.894236] [DEBUG] [LOCAL ] MSD: Primary data saved (12137 records). [2025-09-16T19:29:28.894499] [INFO ] [MAIN ] MSD: Full history saved. [2025-09-16T19:29:28.894598] [INFO ] [MAIN ] Update Progress: 1409/2369 symbols completed. MSD took 32.6s. Estimated remaining: 1h 41m 49s. Rate: 9.4 symbols/min [2025-09-16T19:29:29.068572] [INFO ] [MAIN ] MSM: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:29:29.093065] [DEBUG] [LOCAL ] MSM: Loaded 25248 existing records from primary storage. [2025-09-16T19:29:29.093202] [INFO ] [MAIN ] MSM: Loaded 25248 existing records. [2025-09-16T19:29:29.183811] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-05. [2025-09-16T19:29:29.274385] [WARN ] [MAIN ] MSM: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:29:29.274698] [INFO ] [MAIN ] MSM: Wiping existing primary data before full refresh. [2025-09-16T19:29:29.274747] [INFO ] [LOCAL ] MSM: Starting deletion of primary data. [2025-09-16T19:29:29.275611] [DEBUG] [LOCAL ] MSM: Deleted primary data file. [2025-09-16T19:29:29.275685] [INFO ] [MAIN ] MSM: Starting full primary refresh. [2025-09-16T19:29:29.517855] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-09-16T19:29:29.669927] [INFO ] [REMOTE ] MSM: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:29:29.974917] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-02. [2025-09-16T19:29:30.774972] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-06. [2025-09-16T19:29:30.818974] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-09-16T19:29:30.857273] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-09-16T19:29:31.574913] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-02. [2025-09-16T19:29:31.908490] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-09-16T19:29:32.374939] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-03. [2025-09-16T19:29:32.896914] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-09-16T19:29:33.174493] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T19:29:33.177243] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-03. [2025-09-16T19:29:33.974914] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-06. [2025-09-16T19:29:34.443845] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T19:29:34.774905] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-04. [2025-09-16T19:29:35.435333] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-09-16T19:29:35.574901] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-07. [2025-09-16T19:29:36.153069] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-09-16T19:29:36.374898] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-05. [2025-09-16T19:29:37.160305] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T19:29:37.174861] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-08. [2025-09-16T19:29:37.417825] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-09-16T19:29:37.974897] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-04. [2025-09-16T19:29:38.686011] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-09-16T19:29:38.712191] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-09-16T19:29:38.774861] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-09. [2025-09-16T19:29:39.574914] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-05. [2025-09-16T19:29:39.932975] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-09-16T19:29:39.937575] [INFO ] [REMOTE ] MSI: Full fetch complete (44563 records). [2025-09-16T19:29:39.937708] [DEBUG] [VALIDATOR ] MSI: Validating primary data. [2025-09-16T19:29:40.230144] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-09-16T19:29:40.350586] [DEBUG] [VALIDATOR ] MSI: All expected trading days are present. [2025-09-16T19:29:40.350685] [DEBUG] [VALIDATOR ] MSI: Primary data validation passed. [2025-09-16T19:29:40.351407] [INFO ] [MAIN ] MSI: Fetched primary data is valid. [2025-09-16T19:29:40.351448] [DEBUG] [LOCAL ] MSI: Saving primary data. [2025-09-16T19:29:40.352424] [DEBUG] [LOCAL ] MSI: Trimmed 44563 records to 29947 (keeping 150 days of data) [2025-09-16T19:29:40.409151] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-06. [2025-09-16T19:29:40.542359] [DEBUG] [LOCAL ] MSI: Primary data saved (29947 records). [2025-09-16T19:29:40.542499] [INFO ] [MAIN ] MSI: Full history saved. [2025-09-16T19:29:40.543581] [INFO ] [MAIN ] Update Progress: 1410/2369 symbols completed. MSI took 23.5s. Estimated remaining: 1h 41m 46s. Rate: 9.4 symbols/min [2025-09-16T19:29:40.714475] [INFO ] [MAIN ] MT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:29:40.752702] [DEBUG] [LOCAL ] MT: Loaded 37095 existing records from primary storage. [2025-09-16T19:29:40.752845] [INFO ] [MAIN ] MT: Loaded 37095 existing records. [2025-09-16T19:29:40.929560] [WARN ] [MAIN ] MT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:29:40.929801] [INFO ] [MAIN ] MT: Wiping existing primary data before full refresh. [2025-09-16T19:29:40.929850] [INFO ] [LOCAL ] MT: Starting deletion of primary data. [2025-09-16T19:29:40.930631] [DEBUG] [LOCAL ] MT: Deleted primary data file. [2025-09-16T19:29:40.930705] [INFO ] [MAIN ] MT: Starting full primary refresh. [2025-09-16T19:29:41.180177] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-06. [2025-09-16T19:29:41.294574] [INFO ] [REMOTE ] MT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:29:41.731009] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-09-16T19:29:41.974914] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-02. [2025-09-16T19:29:42.223035] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-09-16T19:29:42.774910] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-07. [2025-09-16T19:29:43.574911] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-07. [2025-09-16T19:29:43.711761] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-09-16T19:29:44.244988] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T19:29:44.374922] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-08. [2025-09-16T19:29:44.579032] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-09-16T19:29:45.174931] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-07. [2025-09-16T19:29:45.737610] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-09-16T19:29:45.943879] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-09-16T19:29:45.974847] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-08. [2025-09-16T19:29:46.774928] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-08. [2025-09-16T19:29:46.819496] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-09-16T19:29:47.574903] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-09. [2025-09-16T19:29:47.825738] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-09-16T19:29:48.374930] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-09. [2025-09-16T19:29:48.617191] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-09-16T19:29:48.621697] [INFO ] [REMOTE ] MSM: Full fetch complete (38121 records). [2025-09-16T19:29:48.621801] [DEBUG] [VALIDATOR ] MSM: Validating primary data. [2025-09-16T19:29:48.996651] [DEBUG] [VALIDATOR ] MSM: All expected trading days are present. [2025-09-16T19:29:48.996744] [DEBUG] [VALIDATOR ] MSM: Primary data validation passed. [2025-09-16T19:29:48.997504] [INFO ] [MAIN ] MSM: Fetched primary data is valid. [2025-09-16T19:29:48.997551] [DEBUG] [LOCAL ] MSM: Saving primary data. [2025-09-16T19:29:48.998594] [DEBUG] [LOCAL ] MSM: Trimmed 38121 records to 25497 (keeping 150 days of data) [2025-09-16T19:29:49.105955] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T19:29:49.127542] [INFO ] [REMOTE ] MSIF: Full fetch complete (14376 records). [2025-09-16T19:29:49.128498] [DEBUG] [VALIDATOR ] MSIF: Validating primary data. [2025-09-16T19:29:49.184505] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-09. [2025-09-16T19:29:49.367749] [DEBUG] [LOCAL ] MSM: Primary data saved (25497 records). [2025-09-16T19:29:49.410528] [INFO ] [MAIN ] MSM: Full history saved. [2025-09-16T19:29:49.432419] [INFO ] [MAIN ] Update Progress: 1411/2369 symbols completed. MSM took 20.5s. Estimated remaining: 1h 41m 41s. Rate: 9.4 symbols/min [2025-09-16T19:29:49.725892] [DEBUG] [VALIDATOR ] MSIF: All expected trading days are present. [2025-09-16T19:29:49.753745] [DEBUG] [VALIDATOR ] MSIF: Primary data validation passed. [2025-09-16T19:29:49.757831] [INFO ] [MAIN ] MSIF: Fetched primary data is valid. [2025-09-16T19:29:49.757947] [DEBUG] [LOCAL ] MSIF: Saving primary data. [2025-09-16T19:29:49.757266] [INFO ] [MAIN ] MTAL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:29:49.759063] [DEBUG] [LOCAL ] MSIF: Trimmed 14376 records to 10694 (keeping 150 days of data) [2025-09-16T19:29:49.828260] [DEBUG] [LOCAL ] MSIF: Primary data saved (10694 records). [2025-09-16T19:29:49.828436] [INFO ] [MAIN ] MSIF: Full history saved. [2025-09-16T19:29:49.828616] [INFO ] [MAIN ] Update Progress: 1412/2369 symbols completed. MSIF took 23.6s. Estimated remaining: 1h 41m 31s. Rate: 9.4 symbols/min [2025-09-16T19:29:49.947718] [DEBUG] [LOCAL ] MTAL: Loaded 27153 existing records from primary storage. [2025-09-16T19:29:49.953570] [INFO ] [MAIN ] MTAL: Loaded 27153 existing records. [2025-09-16T19:29:50.001998] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-03. [2025-09-16T19:29:50.127433] [INFO ] [MAIN ] MTB: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:29:50.228964] [WARN ] [MAIN ] MTAL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:29:50.229396] [INFO ] [MAIN ] MTAL: Wiping existing primary data before full refresh. [2025-09-16T19:29:50.229469] [INFO ] [LOCAL ] MTAL: Starting deletion of primary data. [2025-09-16T19:29:50.238707] [DEBUG] [LOCAL ] MTAL: Deleted primary data file. [2025-09-16T19:29:50.238880] [INFO ] [MAIN ] MTAL: Starting full primary refresh. [2025-09-16T19:29:50.370777] [DEBUG] [LOCAL ] MTB: Loaded 34251 existing records from primary storage. [2025-09-16T19:29:50.376620] [INFO ] [MAIN ] MTB: Loaded 34251 existing records. [2025-09-16T19:29:50.588526] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-09-16T19:29:50.657272] [INFO ] [REMOTE ] MSGS: Full fetch complete (17781 records). [2025-09-16T19:29:50.678454] [DEBUG] [VALIDATOR ] MSGS: Validating primary data. [2025-09-16T19:29:50.870005] [WARN ] [MAIN ] MTB: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:29:50.919561] [INFO ] [REMOTE ] MTAL: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:29:50.923277] [INFO ] [REMOTE ] MTAL: Fetching primary data for 2025-02. [2025-09-16T19:29:50.923407] [INFO ] [MAIN ] MTB: Wiping existing primary data before full refresh. [2025-09-16T19:29:50.923496] [INFO ] [LOCAL ] MTB: Starting deletion of primary data. [2025-09-16T19:29:50.934244] [DEBUG] [LOCAL ] MTB: Deleted primary data file. [2025-09-16T19:29:50.934354] [INFO ] [MAIN ] MTB: Starting full primary refresh. [2025-09-16T19:29:51.526096] [DEBUG] [VALIDATOR ] MSGS: All expected trading days are present. [2025-09-16T19:29:51.538019] [DEBUG] [VALIDATOR ] MSGS: Primary data validation passed. [2025-09-16T19:29:51.548316] [INFO ] [MAIN ] MSGS: Fetched primary data is valid. [2025-09-16T19:29:51.556985] [DEBUG] [LOCAL ] MSGS: Saving primary data. [2025-09-16T19:29:51.578224] [INFO ] [REMOTE ] MTB: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:29:51.578858] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-02. [2025-09-16T19:29:51.578579] [DEBUG] [LOCAL ] MSGS: Trimmed 17781 records to 12394 (keeping 150 days of data) [2025-09-16T19:29:51.655910] [DEBUG] [LOCAL ] MSGS: Primary data saved (12394 records). [2025-09-16T19:29:51.656057] [INFO ] [MAIN ] MSGS: Full history saved. [2025-09-16T19:29:51.656148] [INFO ] [MAIN ] Update Progress: 1413/2369 symbols completed. MSGS took 36.5s. Estimated remaining: 1h 41m 21s. Rate: 9.4 symbols/min [2025-09-16T19:29:51.836664] [INFO ] [MAIN ] MTD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:29:51.857637] [INFO ] [REMOTE ] MTAL: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-09-16T19:29:51.868910] [DEBUG] [LOCAL ] MTD: Loaded 11010 existing records from primary storage. [2025-09-16T19:29:51.869125] [INFO ] [MAIN ] MTD: Loaded 11010 existing records. [2025-09-16T19:29:52.048261] [WARN ] [MAIN ] MTD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:29:52.048733] [INFO ] [MAIN ] MTD: Wiping existing primary data before full refresh. [2025-09-16T19:29:52.048849] [INFO ] [LOCAL ] MTD: Starting deletion of primary data. [2025-09-16T19:29:52.049262] [DEBUG] [LOCAL ] MTD: Deleted primary data file. [2025-09-16T19:29:52.049340] [INFO ] [MAIN ] MTD: Starting full primary refresh. [2025-09-16T19:29:52.220639] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-03 (2.22 seconds). [2025-09-16T19:29:52.380016] [INFO ] [REMOTE ] MTAL: Fetching primary data for 2025-03. [2025-09-16T19:29:52.445684] [INFO ] [REMOTE ] MTD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:29:52.889560] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-09-16T19:29:53.174943] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-02. [2025-09-16T19:29:53.315792] [INFO ] [REMOTE ] MTAL: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T19:29:53.767909] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-09-16T19:29:53.974902] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-03. [2025-09-16T19:29:54.774943] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-04. [2025-09-16T19:29:54.910586] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-09-16T19:29:55.574915] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-04. [2025-09-16T19:29:56.374956] [INFO ] [REMOTE ] MTAL: Fetching primary data for 2025-04. [2025-09-16T19:29:56.597519] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-09-16T19:29:56.901527] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-09-16T19:29:57.174916] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-05. [2025-09-16T19:29:57.459837] [INFO ] [REMOTE ] MTAL: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-09-16T19:29:57.974950] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-05. [2025-09-16T19:29:58.006068] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-09-16T19:29:58.774964] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-06. [2025-09-16T19:29:59.574878] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-03. [2025-09-16T19:29:59.608815] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-09-16T19:29:59.648546] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-09-16T19:30:00.374917] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-06. [2025-09-16T19:30:01.174875] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-07. [2025-09-16T19:30:01.384936] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-09-16T19:30:01.909613] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-09-16T19:30:01.974882] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-08. [2025-09-16T19:30:02.032720] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-09-16T19:30:02.774888] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-04. [2025-09-16T19:30:02.886239] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-09-16T19:30:03.574908] [INFO ] [REMOTE ] MTAL: Fetching primary data for 2025-05. [2025-09-16T19:30:04.374947] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-07. [2025-09-16T19:30:04.490489] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-09-16T19:30:05.113984] [INFO ] [REMOTE ] MTAL: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T19:30:05.175045] [INFO ] [REMOTE ] MTAL: Fetching primary data for 2025-06. [2025-09-16T19:30:05.974892] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-09. [2025-09-16T19:30:06.242537] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-09-16T19:30:06.468720] [INFO ] [REMOTE ] MTAL: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-09-16T19:30:06.774891] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-08. [2025-09-16T19:30:06.918505] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-09-16T19:30:06.920186] [INFO ] [REMOTE ] MTD: Full fetch complete (16600 records). [2025-09-16T19:30:06.920283] [DEBUG] [VALIDATOR ] MTD: Validating primary data. [2025-09-16T19:30:07.282824] [DEBUG] [VALIDATOR ] MTD: All expected trading days are present. [2025-09-16T19:30:07.282908] [DEBUG] [VALIDATOR ] MTD: Primary data validation passed. [2025-09-16T19:30:07.283312] [INFO ] [MAIN ] MTD: Fetched primary data is valid. [2025-09-16T19:30:07.283369] [DEBUG] [LOCAL ] MTD: Saving primary data. [2025-09-16T19:30:07.283925] [DEBUG] [LOCAL ] MTD: Trimmed 16600 records to 11088 (keeping 150 days of data) [2025-09-16T19:30:07.357273] [DEBUG] [LOCAL ] MTD: Primary data saved (11088 records). [2025-09-16T19:30:07.357434] [INFO ] [MAIN ] MTD: Full history saved. [2025-09-16T19:30:07.357558] [INFO ] [MAIN ] Update Progress: 1414/2369 symbols completed. MTD took 15.7s. Estimated remaining: 1h 41m 21s. Rate: 9.4 symbols/min [2025-09-16T19:30:07.537169] [INFO ] [MAIN ] MTDR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:30:07.580988] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-05. [2025-09-16T19:30:07.583663] [DEBUG] [LOCAL ] MTDR: Loaded 37997 existing records from primary storage. [2025-09-16T19:30:07.583766] [INFO ] [MAIN ] MTDR: Loaded 37997 existing records. [2025-09-16T19:30:07.800548] [WARN ] [MAIN ] MTDR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:30:07.800843] [INFO ] [MAIN ] MTDR: Wiping existing primary data before full refresh. [2025-09-16T19:30:07.800892] [INFO ] [LOCAL ] MTDR: Starting deletion of primary data. [2025-09-16T19:30:07.801766] [DEBUG] [LOCAL ] MTDR: Deleted primary data file. [2025-09-16T19:30:07.801816] [INFO ] [MAIN ] MTDR: Starting full primary refresh. [2025-09-16T19:30:08.152351] [INFO ] [REMOTE ] MTDR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:30:08.374912] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-02. [2025-09-16T19:30:08.602246] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-09-16T19:30:08.933298] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-09-16T19:30:09.174947] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-06. [2025-09-16T19:30:09.974931] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-09. [2025-09-16T19:30:09.988974] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-09-16T19:30:10.665791] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-09-16T19:30:10.774897] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-07. [2025-09-16T19:30:11.574904] [INFO ] [REMOTE ] MTAL: Fetching primary data for 2025-07. [2025-09-16T19:30:11.826211] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-09-16T19:30:11.832291] [INFO ] [REMOTE ] MT: Full fetch complete (55393 records). [2025-09-16T19:30:11.832397] [DEBUG] [VALIDATOR ] MT: Validating primary data. [2025-09-16T19:30:12.216528] [DEBUG] [VALIDATOR ] MT: All expected trading days are present. [2025-09-16T19:30:12.216655] [DEBUG] [VALIDATOR ] MT: Primary data validation passed. [2025-09-16T19:30:12.217488] [INFO ] [MAIN ] MT: Fetched primary data is valid. [2025-09-16T19:30:12.217532] [DEBUG] [LOCAL ] MT: Saving primary data. [2025-09-16T19:30:12.218853] [DEBUG] [LOCAL ] MT: Trimmed 55393 records to 37445 (keeping 150 days of data) [2025-09-16T19:30:12.393834] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-03. [2025-09-16T19:30:12.455652] [DEBUG] [LOCAL ] MT: Primary data saved (37445 records). [2025-09-16T19:30:12.455818] [INFO ] [MAIN ] MT: Full history saved. [2025-09-16T19:30:12.456777] [INFO ] [MAIN ] Update Progress: 1415/2369 symbols completed. MT took 31.9s. Estimated remaining: 1h 41m 14s. Rate: 9.4 symbols/min [2025-09-16T19:30:12.631361] [INFO ] [MAIN ] MTG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:30:12.670462] [DEBUG] [LOCAL ] MTG: Loaded 37536 existing records from primary storage. [2025-09-16T19:30:12.670610] [INFO ] [MAIN ] MTG: Loaded 37536 existing records. [2025-09-16T19:30:12.857841] [WARN ] [MAIN ] MTG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:30:12.858172] [INFO ] [MAIN ] MTG: Wiping existing primary data before full refresh. [2025-09-16T19:30:12.858281] [INFO ] [LOCAL ] MTG: Starting deletion of primary data. [2025-09-16T19:30:12.859166] [DEBUG] [LOCAL ] MTG: Deleted primary data file. [2025-09-16T19:30:12.859217] [INFO ] [MAIN ] MTG: Starting full primary refresh. [2025-09-16T19:30:13.020872] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-09-16T19:30:13.177321] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-08. [2025-09-16T19:30:13.281854] [INFO ] [REMOTE ] MTG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:30:13.304490] [INFO ] [REMOTE ] MTAL: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-09-16T19:30:13.974932] [INFO ] [REMOTE ] MTAL: Fetching primary data for 2025-08. [2025-09-16T19:30:14.248674] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-09-16T19:30:14.773408] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-09-16T19:30:14.782396] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-02. [2025-09-16T19:30:15.231693] [INFO ] [REMOTE ] MTAL: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-09-16T19:30:15.574924] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-09. [2025-09-16T19:30:16.257067] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T19:30:16.374936] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-04. [2025-09-16T19:30:17.177961] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-03. [2025-09-16T19:30:17.180243] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-09-16T19:30:17.186417] [INFO ] [REMOTE ] MTB: Full fetch complete (51317 records). [2025-09-16T19:30:17.186553] [DEBUG] [VALIDATOR ] MTB: Validating primary data. [2025-09-16T19:30:17.565827] [DEBUG] [VALIDATOR ] MTB: All expected trading days are present. [2025-09-16T19:30:17.565915] [DEBUG] [VALIDATOR ] MTB: Primary data validation passed. [2025-09-16T19:30:17.566795] [INFO ] [MAIN ] MTB: Fetched primary data is valid. [2025-09-16T19:30:17.566846] [DEBUG] [LOCAL ] MTB: Saving primary data. [2025-09-16T19:30:17.567908] [DEBUG] [LOCAL ] MTB: Trimmed 51317 records to 34580 (keeping 150 days of data) [2025-09-16T19:30:17.798713] [DEBUG] [LOCAL ] MTB: Primary data saved (34580 records). [2025-09-16T19:30:17.798882] [INFO ] [MAIN ] MTB: Full history saved. [2025-09-16T19:30:17.799011] [INFO ] [MAIN ] Update Progress: 1416/2369 symbols completed. MTB took 28.0s. Estimated remaining: 1h 41m 7s. Rate: 9.4 symbols/min [2025-09-16T19:30:17.969242] [INFO ] [MAIN ] MTH: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:30:17.974973] [INFO ] [REMOTE ] MTAL: Fetching primary data for 2025-09. [2025-09-16T19:30:18.004722] [DEBUG] [LOCAL ] MTH: Loaded 32137 existing records from primary storage. [2025-09-16T19:30:18.004895] [INFO ] [MAIN ] MTH: Loaded 32137 existing records. [2025-09-16T19:30:18.185056] [WARN ] [MAIN ] MTH: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:30:18.185355] [INFO ] [MAIN ] MTH: Wiping existing primary data before full refresh. [2025-09-16T19:30:18.185426] [INFO ] [LOCAL ] MTH: Starting deletion of primary data. [2025-09-16T19:30:18.186157] [DEBUG] [LOCAL ] MTH: Deleted primary data file. [2025-09-16T19:30:18.186211] [INFO ] [MAIN ] MTH: Starting full primary refresh. [2025-09-16T19:30:18.543352] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-09-16T19:30:18.625490] [INFO ] [REMOTE ] MTH: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:30:18.774908] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-02. [2025-09-16T19:30:18.932011] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-09-16T19:30:19.574930] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-04. [2025-09-16T19:30:20.076176] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-09-16T19:30:20.375020] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-05. [2025-09-16T19:30:20.905055] [INFO ] [REMOTE ] MTAL: Incremental fetch complete for 2025-09 (2.93 seconds). [2025-09-16T19:30:20.910508] [INFO ] [REMOTE ] MTAL: Full fetch complete (37829 records). [2025-09-16T19:30:20.910620] [DEBUG] [VALIDATOR ] MTAL: Validating primary data. [2025-09-16T19:30:20.911579] [FATAL] [VALIDATOR ] MTAL: Data is stale - most recent date (2025-09-15) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T19:30:20.911691] [FATAL] [MAIN ] MTAL: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:30:20.911731] [WARN ] [MAIN ] MTAL: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:30:20.911791] [INFO ] [LOCAL ] MTAL: Starting deletion of primary data. [2025-09-16T19:30:20.911852] [DEBUG] [LOCAL ] MTAL: No primary data file found to delete. [2025-09-16T19:30:20.911888] [INFO ] [LOCAL ] MTAL: Starting deletion of derivative data. [2025-09-16T19:30:20.912184] [DEBUG] [LOCAL ] MTAL: Deleted derivative data file. [2025-09-16T19:30:20.912274] [INFO ] [MAIN ] Update Progress: 1417/2369 symbols completed. MTAL took 31.5s. Estimated remaining: 1h 40m 59s. Rate: 9.4 symbols/min [2025-09-16T19:30:21.082385] [INFO ] [MAIN ] MTN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:30:21.117264] [DEBUG] [LOCAL ] MTN: Loaded 28572 existing records from primary storage. [2025-09-16T19:30:21.117413] [INFO ] [MAIN ] MTN: Loaded 28572 existing records. [2025-09-16T19:30:21.183658] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-03. [2025-09-16T19:30:21.300072] [WARN ] [MAIN ] MTN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:30:21.300445] [INFO ] [MAIN ] MTN: Wiping existing primary data before full refresh. [2025-09-16T19:30:21.300532] [INFO ] [LOCAL ] MTN: Starting deletion of primary data. [2025-09-16T19:30:21.301175] [DEBUG] [LOCAL ] MTN: Deleted primary data file. [2025-09-16T19:30:21.301230] [INFO ] [MAIN ] MTN: Starting full primary refresh. [2025-09-16T19:30:21.611238] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-09-16T19:30:21.698429] [INFO ] [REMOTE ] MTN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:30:21.974859] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-05. [2025-09-16T19:30:22.105763] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-09-16T19:30:22.774956] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-02. [2025-09-16T19:30:22.829995] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-09-16T19:30:23.574908] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-06. [2025-09-16T19:30:23.673788] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-09-16T19:30:23.828927] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-09-16T19:30:24.374902] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-03. [2025-09-16T19:30:25.174957] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-06. [2025-09-16T19:30:25.643488] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-09-16T19:30:25.715198] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-06 (2.14 seconds). [2025-09-16T19:30:25.974959] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-07. [2025-09-16T19:30:26.774908] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-04. [2025-09-16T19:30:26.845653] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-09-16T19:30:27.444898] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-09-16T19:30:27.574940] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-07. [2025-09-16T19:30:28.356009] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-09-16T19:30:28.374870] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-05. [2025-09-16T19:30:29.174968] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-08. [2025-09-16T19:30:29.474531] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-09-16T19:30:29.936501] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-09-16T19:30:29.974949] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-04. [2025-09-16T19:30:30.694241] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-09-16T19:30:30.774858] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-09. [2025-09-16T19:30:31.255751] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-09-16T19:30:31.574907] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-08. [2025-09-16T19:30:32.297490] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-09-16T19:30:32.306359] [INFO ] [REMOTE ] MTDR: Full fetch complete (55961 records). [2025-09-16T19:30:32.306497] [DEBUG] [VALIDATOR ] MTDR: Validating primary data. [2025-09-16T19:30:32.379974] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-05. [2025-09-16T19:30:32.691257] [DEBUG] [VALIDATOR ] MTDR: All expected trading days are present. [2025-09-16T19:30:32.691341] [DEBUG] [VALIDATOR ] MTDR: Primary data validation passed. [2025-09-16T19:30:32.692278] [INFO ] [MAIN ] MTDR: Fetched primary data is valid. [2025-09-16T19:30:32.692362] [DEBUG] [LOCAL ] MTDR: Saving primary data. [2025-09-16T19:30:32.693594] [DEBUG] [LOCAL ] MTDR: Trimmed 55961 records to 38375 (keeping 150 days of data) [2025-09-16T19:30:32.929083] [DEBUG] [LOCAL ] MTDR: Primary data saved (38375 records). [2025-09-16T19:30:32.929245] [INFO ] [MAIN ] MTDR: Full history saved. [2025-09-16T19:30:32.930106] [INFO ] [MAIN ] Update Progress: 1418/2369 symbols completed. MTDR took 25.6s. Estimated remaining: 1h 40m 56s. Rate: 9.4 symbols/min [2025-09-16T19:30:33.113629] [INFO ] [MAIN ] MTR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:30:33.119147] [DEBUG] [LOCAL ] MTR: Loaded 1586 existing records from primary storage. [2025-09-16T19:30:33.119238] [INFO ] [MAIN ] MTR: Loaded 1586 existing records. [2025-09-16T19:30:33.183068] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-06. [2025-09-16T19:30:33.302608] [WARN ] [MAIN ] MTR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:30:33.302979] [INFO ] [MAIN ] MTR: Wiping existing primary data before full refresh. [2025-09-16T19:30:33.303130] [INFO ] [LOCAL ] MTR: Starting deletion of primary data. [2025-09-16T19:30:33.303482] [DEBUG] [LOCAL ] MTR: Deleted primary data file. [2025-09-16T19:30:33.303578] [INFO ] [MAIN ] MTR: Starting full primary refresh. [2025-09-16T19:30:33.608866] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-08 (2.03 seconds). [2025-09-16T19:30:33.743728] [INFO ] [REMOTE ] MTR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:30:33.974885] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-09. [2025-09-16T19:30:34.053758] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-09-16T19:30:34.744530] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T19:30:34.774842] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-07. [2025-09-16T19:30:35.531496] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-09-16T19:30:35.541121] [INFO ] [REMOTE ] MTG: Full fetch complete (55869 records). [2025-09-16T19:30:35.541374] [DEBUG] [VALIDATOR ] MTG: Validating primary data. [2025-09-16T19:30:35.580182] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-06. [2025-09-16T19:30:35.950344] [DEBUG] [VALIDATOR ] MTG: All expected trading days are present. [2025-09-16T19:30:35.950428] [DEBUG] [VALIDATOR ] MTG: Primary data validation passed. [2025-09-16T19:30:35.951340] [INFO ] [MAIN ] MTG: Fetched primary data is valid. [2025-09-16T19:30:35.951387] [DEBUG] [LOCAL ] MTG: Saving primary data. [2025-09-16T19:30:35.952624] [DEBUG] [LOCAL ] MTG: Trimmed 55869 records to 37903 (keeping 150 days of data) [2025-09-16T19:30:36.194807] [DEBUG] [LOCAL ] MTG: Primary data saved (37903 records). [2025-09-16T19:30:36.195020] [INFO ] [MAIN ] MTG: Full history saved. [2025-09-16T19:30:36.195735] [INFO ] [MAIN ] Update Progress: 1419/2369 symbols completed. MTG took 23.7s. Estimated remaining: 1h 40m 48s. Rate: 9.4 symbols/min [2025-09-16T19:30:36.369327] [INFO ] [MAIN ] MTRN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:30:36.379343] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-02. [2025-09-16T19:30:36.406827] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-09-16T19:30:36.428142] [DEBUG] [LOCAL ] MTRN: Loaded 13585 existing records from primary storage. [2025-09-16T19:30:36.428279] [INFO ] [MAIN ] MTRN: Loaded 13585 existing records. [2025-09-16T19:30:36.609336] [WARN ] [MAIN ] MTRN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:30:36.609581] [INFO ] [MAIN ] MTRN: Wiping existing primary data before full refresh. [2025-09-16T19:30:36.609633] [INFO ] [LOCAL ] MTRN: Starting deletion of primary data. [2025-09-16T19:30:36.610182] [DEBUG] [LOCAL ] MTRN: Deleted primary data file. [2025-09-16T19:30:36.610235] [INFO ] [MAIN ] MTRN: Starting full primary refresh. [2025-09-16T19:30:36.876539] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-09-16T19:30:36.984848] [INFO ] [REMOTE ] MTRN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:30:37.174888] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-03. [2025-09-16T19:30:37.213615] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-09-16T19:30:37.532868] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-09-16T19:30:37.974963] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-02. [2025-09-16T19:30:38.774971] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-04. [2025-09-16T19:30:38.844619] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-09-16T19:30:39.072732] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-09-16T19:30:39.574931] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-03. [2025-09-16T19:30:40.374898] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-05. [2025-09-16T19:30:40.644065] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-09-16T19:30:40.736078] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-09-16T19:30:41.174920] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-07. [2025-09-16T19:30:41.974895] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-08. [2025-09-16T19:30:42.742635] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-09-16T19:30:42.774862] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-08. [2025-09-16T19:30:43.336902] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-09-16T19:30:43.574913] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-04. [2025-09-16T19:30:44.070945] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-09-16T19:30:44.374918] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-09. [2025-09-16T19:30:44.617965] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-09-16T19:30:45.174910] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-05. [2025-09-16T19:30:45.974952] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-06. [2025-09-16T19:30:46.091761] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-09-16T19:30:46.097220] [INFO ] [REMOTE ] MTH: Full fetch complete (47968 records). [2025-09-16T19:30:46.097377] [DEBUG] [VALIDATOR ] MTH: Validating primary data. [2025-09-16T19:30:46.508467] [DEBUG] [VALIDATOR ] MTH: All expected trading days are present. [2025-09-16T19:30:46.509254] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-09-16T19:30:46.509396] [DEBUG] [VALIDATOR ] MTH: Primary data validation passed. [2025-09-16T19:30:46.514980] [INFO ] [MAIN ] MTH: Fetched primary data is valid. [2025-09-16T19:30:46.515635] [DEBUG] [LOCAL ] MTH: Saving primary data. [2025-09-16T19:30:46.516615] [DEBUG] [LOCAL ] MTH: Trimmed 47968 records to 32475 (keeping 150 days of data) [2025-09-16T19:30:46.721882] [DEBUG] [LOCAL ] MTH: Primary data saved (32475 records). [2025-09-16T19:30:46.722019] [INFO ] [MAIN ] MTH: Full history saved. [2025-09-16T19:30:46.722135] [INFO ] [MAIN ] Update Progress: 1420/2369 symbols completed. MTH took 28.9s. Estimated remaining: 1h 40m 44s. Rate: 9.4 symbols/min [2025-09-16T19:30:46.786230] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-06. [2025-09-16T19:30:46.888788] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-09-16T19:30:46.906507] [INFO ] [MAIN ] MTUS: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:30:46.928523] [DEBUG] [LOCAL ] MTUS: Loaded 20687 existing records from primary storage. [2025-09-16T19:30:46.928695] [INFO ] [MAIN ] MTUS: Loaded 20687 existing records. [2025-09-16T19:30:47.104468] [WARN ] [MAIN ] MTUS: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:30:47.104714] [INFO ] [MAIN ] MTUS: Wiping existing primary data before full refresh. [2025-09-16T19:30:47.104771] [INFO ] [LOCAL ] MTUS: Starting deletion of primary data. [2025-09-16T19:30:47.105300] [DEBUG] [LOCAL ] MTUS: Deleted primary data file. [2025-09-16T19:30:47.105345] [INFO ] [MAIN ] MTUS: Starting full primary refresh. [2025-09-16T19:30:47.461568] [INFO ] [REMOTE ] MTUS: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:30:47.574888] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-09. [2025-09-16T19:30:47.742635] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-09-16T19:30:48.374964] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-07. [2025-09-16T19:30:48.732096] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-09-16T19:30:48.737170] [INFO ] [REMOTE ] MTN: Full fetch complete (42420 records). [2025-09-16T19:30:48.737323] [DEBUG] [VALIDATOR ] MTN: Validating primary data. [2025-09-16T19:30:48.915066] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-09-16T19:30:49.114197] [DEBUG] [VALIDATOR ] MTN: All expected trading days are present. [2025-09-16T19:30:49.114281] [DEBUG] [VALIDATOR ] MTN: Primary data validation passed. [2025-09-16T19:30:49.115084] [INFO ] [MAIN ] MTN: Fetched primary data is valid. [2025-09-16T19:30:49.115129] [DEBUG] [LOCAL ] MTN: Saving primary data. [2025-09-16T19:30:49.116082] [DEBUG] [LOCAL ] MTN: Trimmed 42420 records to 28880 (keeping 150 days of data) [2025-09-16T19:30:49.178499] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-07. [2025-09-16T19:30:49.307290] [DEBUG] [LOCAL ] MTN: Primary data saved (28880 records). [2025-09-16T19:30:49.307445] [INFO ] [MAIN ] MTN: Full history saved. [2025-09-16T19:30:49.307550] [INFO ] [MAIN ] Update Progress: 1421/2369 symbols completed. MTN took 28.4s. Estimated remaining: 1h 40m 35s. Rate: 9.4 symbols/min [2025-09-16T19:30:49.474331] [INFO ] [MAIN ] MTW: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:30:49.494940] [DEBUG] [LOCAL ] MTW: Loaded 19697 existing records from primary storage. [2025-09-16T19:30:49.495071] [INFO ] [MAIN ] MTW: Loaded 19697 existing records. [2025-09-16T19:30:49.668437] [WARN ] [MAIN ] MTW: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:30:49.668728] [INFO ] [MAIN ] MTW: Wiping existing primary data before full refresh. [2025-09-16T19:30:49.668789] [INFO ] [LOCAL ] MTW: Starting deletion of primary data. [2025-09-16T19:30:49.669348] [DEBUG] [LOCAL ] MTW: Deleted primary data file. [2025-09-16T19:30:49.669397] [INFO ] [MAIN ] MTW: Starting full primary refresh. [2025-09-16T19:30:49.989410] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-08. [2025-09-16T19:30:50.028013] [INFO ] [REMOTE ] MTW: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:30:50.195761] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T19:30:50.385714] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-09-16T19:30:50.774905] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-02. [2025-09-16T19:30:51.574917] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-02. [2025-09-16T19:30:51.941406] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-09-16T19:30:52.374898] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-03. [2025-09-16T19:30:52.911134] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-09-16T19:30:53.174929] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-03. [2025-09-16T19:30:53.757506] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-09-16T19:30:53.974979] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-04. [2025-09-16T19:30:54.277808] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-09-16T19:30:54.774945] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-09. [2025-09-16T19:30:55.269219] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-09-16T19:30:55.294940] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-09-16T19:30:55.295771] [INFO ] [REMOTE ] MTR: Full fetch complete (1924 records). [2025-09-16T19:30:55.295892] [DEBUG] [VALIDATOR ] MTR: Validating primary data. [2025-09-16T19:30:55.580141] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-05. [2025-09-16T19:30:55.661191] [DEBUG] [VALIDATOR ] MTR: All expected trading days are present. [2025-09-16T19:30:55.661411] [DEBUG] [VALIDATOR ] MTR: Primary data validation passed. [2025-09-16T19:30:55.661530] [INFO ] [MAIN ] MTR: Fetched primary data is valid. [2025-09-16T19:30:55.661569] [DEBUG] [LOCAL ] MTR: Saving primary data. [2025-09-16T19:30:55.661997] [DEBUG] [LOCAL ] MTR: Trimmed 1924 records to 1590 (keeping 150 days of data) [2025-09-16T19:30:55.672787] [DEBUG] [LOCAL ] MTR: Primary data saved (1590 records). [2025-09-16T19:30:55.672876] [INFO ] [MAIN ] MTR: Full history saved. [2025-09-16T19:30:55.672969] [INFO ] [MAIN ] Update Progress: 1422/2369 symbols completed. MTR took 22.7s. Estimated remaining: 1h 40m 29s. Rate: 9.4 symbols/min [2025-09-16T19:30:55.847596] [INFO ] [MAIN ] MTX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:30:55.874378] [DEBUG] [LOCAL ] MTX: Loaded 17395 existing records from primary storage. [2025-09-16T19:30:55.874577] [INFO ] [MAIN ] MTX: Loaded 17395 existing records. [2025-09-16T19:30:56.058609] [WARN ] [MAIN ] MTX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:30:56.058876] [INFO ] [MAIN ] MTX: Wiping existing primary data before full refresh. [2025-09-16T19:30:56.058922] [INFO ] [LOCAL ] MTX: Starting deletion of primary data. [2025-09-16T19:30:56.059566] [DEBUG] [LOCAL ] MTX: Deleted primary data file. [2025-09-16T19:30:56.059613] [INFO ] [MAIN ] MTX: Starting full primary refresh. [2025-09-16T19:30:56.384298] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-04. [2025-09-16T19:30:56.422572] [INFO ] [REMOTE ] MTX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:30:56.969140] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-09-16T19:30:57.174921] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-02. [2025-09-16T19:30:57.706166] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-09-16T19:30:57.974909] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-05. [2025-09-16T19:30:58.235142] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T19:30:58.774916] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-06. [2025-09-16T19:30:59.430518] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-09-16T19:30:59.574968] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-06. [2025-09-16T19:30:59.943546] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-09-16T19:31:00.374957] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-08. [2025-09-16T19:31:01.174921] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-07. [2025-09-16T19:31:01.253667] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-09-16T19:31:01.470036] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-09-16T19:31:01.974916] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-09. [2025-09-16T19:31:02.425166] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-09-16T19:31:02.774885] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-07. [2025-09-16T19:31:02.982627] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-09-16T19:31:02.985013] [INFO ] [REMOTE ] MTRN: Full fetch complete (18914 records). [2025-09-16T19:31:02.985092] [DEBUG] [VALIDATOR ] MTRN: Validating primary data. [2025-09-16T19:31:03.353511] [DEBUG] [VALIDATOR ] MTRN: All expected trading days are present. [2025-09-16T19:31:03.353636] [DEBUG] [VALIDATOR ] MTRN: Primary data validation passed. [2025-09-16T19:31:03.354172] [INFO ] [MAIN ] MTRN: Fetched primary data is valid. [2025-09-16T19:31:03.354219] [DEBUG] [LOCAL ] MTRN: Saving primary data. [2025-09-16T19:31:03.354875] [DEBUG] [LOCAL ] MTRN: Trimmed 18914 records to 13726 (keeping 150 days of data) [2025-09-16T19:31:03.442736] [DEBUG] [LOCAL ] MTRN: Primary data saved (13726 records). [2025-09-16T19:31:03.442880] [INFO ] [MAIN ] MTRN: Full history saved. [2025-09-16T19:31:03.442960] [INFO ] [MAIN ] Update Progress: 1423/2369 symbols completed. MTRN took 27.2s. Estimated remaining: 1h 40m 23s. Rate: 9.4 symbols/min [2025-09-16T19:31:03.583101] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-03. [2025-09-16T19:31:03.613972] [INFO ] [MAIN ] MTZ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:31:03.652603] [DEBUG] [LOCAL ] MTZ: Loaded 31459 existing records from primary storage. [2025-09-16T19:31:03.652796] [INFO ] [MAIN ] MTZ: Loaded 31459 existing records. [2025-09-16T19:31:03.832485] [WARN ] [MAIN ] MTZ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:31:03.832790] [INFO ] [MAIN ] MTZ: Wiping existing primary data before full refresh. [2025-09-16T19:31:03.832838] [INFO ] [LOCAL ] MTZ: Starting deletion of primary data. [2025-09-16T19:31:03.833519] [DEBUG] [LOCAL ] MTZ: Deleted primary data file. [2025-09-16T19:31:03.833566] [INFO ] [MAIN ] MTZ: Starting full primary refresh. [2025-09-16T19:31:04.204987] [INFO ] [REMOTE ] MTZ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:31:04.227057] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-09-16T19:31:04.374905] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-02. [2025-09-16T19:31:04.722563] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-09-16T19:31:05.174904] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-04. [2025-09-16T19:31:05.636507] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-09-16T19:31:05.974951] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-08. [2025-09-16T19:31:06.316073] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-09-16T19:31:06.774917] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-03. [2025-09-16T19:31:07.151176] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T19:31:07.574905] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-09. [2025-09-16T19:31:08.374887] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-05. [2025-09-16T19:31:08.571325] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-09-16T19:31:08.619764] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-09-16T19:31:08.623348] [INFO ] [REMOTE ] MTUS: Full fetch complete (31780 records). [2025-09-16T19:31:08.623439] [DEBUG] [VALIDATOR ] MTUS: Validating primary data. [2025-09-16T19:31:09.018480] [DEBUG] [VALIDATOR ] MTUS: All expected trading days are present. [2025-09-16T19:31:09.018561] [DEBUG] [VALIDATOR ] MTUS: Primary data validation passed. [2025-09-16T19:31:09.019284] [INFO ] [MAIN ] MTUS: Fetched primary data is valid. [2025-09-16T19:31:09.019332] [DEBUG] [LOCAL ] MTUS: Saving primary data. [2025-09-16T19:31:09.020129] [DEBUG] [LOCAL ] MTUS: Trimmed 31780 records to 20861 (keeping 150 days of data) [2025-09-16T19:31:09.146082] [DEBUG] [LOCAL ] MTUS: Primary data saved (20861 records). [2025-09-16T19:31:09.146218] [INFO ] [MAIN ] MTUS: Full history saved. [2025-09-16T19:31:09.146306] [INFO ] [MAIN ] Update Progress: 1424/2369 symbols completed. MTUS took 22.4s. Estimated remaining: 1h 40m 16s. Rate: 9.4 symbols/min [2025-09-16T19:31:09.179963] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-08. [2025-09-16T19:31:09.330589] [INFO ] [MAIN ] MUA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:31:09.341049] [DEBUG] [LOCAL ] MUA: Loaded 8367 existing records from primary storage. [2025-09-16T19:31:09.341182] [INFO ] [MAIN ] MUA: Loaded 8367 existing records. [2025-09-16T19:31:09.534202] [WARN ] [MAIN ] MUA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:31:09.534615] [INFO ] [MAIN ] MUA: Wiping existing primary data before full refresh. [2025-09-16T19:31:09.534772] [INFO ] [LOCAL ] MUA: Starting deletion of primary data. [2025-09-16T19:31:09.540224] [DEBUG] [LOCAL ] MUA: Deleted primary data file. [2025-09-16T19:31:09.548401] [INFO ] [MAIN ] MUA: Starting full primary refresh. [2025-09-16T19:31:09.555963] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-09-16T19:31:09.932893] [INFO ] [REMOTE ] MUA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:31:09.974867] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-06. [2025-09-16T19:31:10.305237] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-09-16T19:31:10.774913] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-02. [2025-09-16T19:31:11.127313] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-09-16T19:31:11.574904] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-07. [2025-09-16T19:31:11.817382] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T19:31:12.374871] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-03. [2025-09-16T19:31:12.790041] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-09-16T19:31:13.139307] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-09-16T19:31:13.174905] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-08. [2025-09-16T19:31:13.974921] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-04. [2025-09-16T19:31:14.354324] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-09-16T19:31:14.774942] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-09. [2025-09-16T19:31:14.901696] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-09-16T19:31:15.574886] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-04. [2025-09-16T19:31:15.865055] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-09-16T19:31:15.869459] [INFO ] [REMOTE ] MTX: Full fetch complete (24921 records). [2025-09-16T19:31:15.869596] [DEBUG] [VALIDATOR ] MTX: Validating primary data. [2025-09-16T19:31:16.261869] [DEBUG] [VALIDATOR ] MTX: All expected trading days are present. [2025-09-16T19:31:16.261957] [DEBUG] [VALIDATOR ] MTX: Primary data validation passed. [2025-09-16T19:31:16.262521] [INFO ] [MAIN ] MTX: Fetched primary data is valid. [2025-09-16T19:31:16.262565] [DEBUG] [LOCAL ] MTX: Saving primary data. [2025-09-16T19:31:16.263408] [DEBUG] [LOCAL ] MTX: Trimmed 24921 records to 17516 (keeping 150 days of data) [2025-09-16T19:31:16.370634] [DEBUG] [LOCAL ] MTX: Primary data saved (17516 records). [2025-09-16T19:31:16.370782] [INFO ] [MAIN ] MTX: Full history saved. [2025-09-16T19:31:16.370865] [INFO ] [MAIN ] Update Progress: 1425/2369 symbols completed. MTX took 20.7s. Estimated remaining: 1h 40m 11s. Rate: 9.4 symbols/min [2025-09-16T19:31:16.382262] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-05. [2025-09-16T19:31:16.548258] [INFO ] [MAIN ] MUC: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:31:16.563253] [DEBUG] [LOCAL ] MUC: Loaded 14348 existing records from primary storage. [2025-09-16T19:31:16.563404] [INFO ] [MAIN ] MUC: Loaded 14348 existing records. [2025-09-16T19:31:16.743121] [WARN ] [MAIN ] MUC: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:31:16.743362] [INFO ] [MAIN ] MUC: Wiping existing primary data before full refresh. [2025-09-16T19:31:16.743409] [INFO ] [LOCAL ] MUC: Starting deletion of primary data. [2025-09-16T19:31:16.743846] [DEBUG] [LOCAL ] MUC: Deleted primary data file. [2025-09-16T19:31:16.743895] [INFO ] [MAIN ] MUC: Starting full primary refresh. [2025-09-16T19:31:17.099261] [INFO ] [REMOTE ] MUC: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:31:17.184203] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-02. [2025-09-16T19:31:17.202887] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-09-16T19:31:17.225369] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-09-16T19:31:17.975026] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-05. [2025-09-16T19:31:18.245128] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-09-16T19:31:18.775019] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-03. [2025-09-16T19:31:19.574966] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-09. [2025-09-16T19:31:19.712757] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-09-16T19:31:20.006471] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-09-16T19:31:20.374952] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-06. [2025-09-16T19:31:20.471681] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-09-16T19:31:20.475022] [INFO ] [REMOTE ] MTW: Full fetch complete (29439 records). [2025-09-16T19:31:20.475121] [DEBUG] [VALIDATOR ] MTW: Validating primary data. [2025-09-16T19:31:20.896498] [DEBUG] [VALIDATOR ] MTW: All expected trading days are present. [2025-09-16T19:31:20.896581] [DEBUG] [VALIDATOR ] MTW: Primary data validation passed. [2025-09-16T19:31:20.897235] [INFO ] [MAIN ] MTW: Fetched primary data is valid. [2025-09-16T19:31:20.897283] [DEBUG] [LOCAL ] MTW: Saving primary data. [2025-09-16T19:31:20.898100] [DEBUG] [LOCAL ] MTW: Trimmed 29439 records to 19869 (keeping 150 days of data) [2025-09-16T19:31:21.013880] [DEBUG] [LOCAL ] MTW: Primary data saved (19869 records). [2025-09-16T19:31:21.014177] [INFO ] [MAIN ] MTW: Full history saved. [2025-09-16T19:31:21.014314] [INFO ] [MAIN ] Update Progress: 1426/2369 symbols completed. MTW took 31.7s. Estimated remaining: 1h 40m 3s. Rate: 9.4 symbols/min [2025-09-16T19:31:21.183274] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-06. [2025-09-16T19:31:21.185856] [INFO ] [MAIN ] MUE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:31:21.196851] [DEBUG] [LOCAL ] MUE: Loaded 9308 existing records from primary storage. [2025-09-16T19:31:21.196975] [INFO ] [MAIN ] MUE: Loaded 9308 existing records. [2025-09-16T19:31:21.375582] [WARN ] [MAIN ] MUE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:31:21.375851] [INFO ] [MAIN ] MUE: Wiping existing primary data before full refresh. [2025-09-16T19:31:21.375897] [INFO ] [LOCAL ] MUE: Starting deletion of primary data. [2025-09-16T19:31:21.376260] [DEBUG] [LOCAL ] MUE: Deleted primary data file. [2025-09-16T19:31:21.376309] [INFO ] [MAIN ] MUE: Starting full primary refresh. [2025-09-16T19:31:21.736436] [INFO ] [REMOTE ] MUE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:31:21.973749] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-09-16T19:31:21.982195] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-02. [2025-09-16T19:31:22.024689] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-09-16T19:31:22.698406] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-09-16T19:31:22.774915] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-07. [2025-09-16T19:31:23.574975] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-07. [2025-09-16T19:31:24.374918] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-04. [2025-09-16T19:31:24.428086] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-09-16T19:31:24.457218] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-09-16T19:31:25.174903] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-08. [2025-09-16T19:31:25.496799] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T19:31:25.974890] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-05. [2025-09-16T19:31:26.102587] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-09-16T19:31:26.728053] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-09-16T19:31:26.774866] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-08. [2025-09-16T19:31:27.574944] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-06. [2025-09-16T19:31:28.374909] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-03. [2025-09-16T19:31:28.530772] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-09-16T19:31:28.581776] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-09-16T19:31:28.838097] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-09-16T19:31:29.174890] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-07. [2025-09-16T19:31:29.974893] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-09. [2025-09-16T19:31:30.307807] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-09-16T19:31:30.774908] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-09. [2025-09-16T19:31:31.557466] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-09-16T19:31:31.562666] [INFO ] [REMOTE ] MTZ: Full fetch complete (47573 records). [2025-09-16T19:31:31.562812] [DEBUG] [VALIDATOR ] MTZ: Validating primary data. [2025-09-16T19:31:31.584320] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-04. [2025-09-16T19:31:31.696591] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-09-16T19:31:31.748369] [INFO ] [REMOTE ] MUA: Full fetch complete (13571 records). [2025-09-16T19:31:31.758701] [DEBUG] [VALIDATOR ] MUA: Validating primary data. [2025-09-16T19:31:32.248975] [DEBUG] [VALIDATOR ] MTZ: All expected trading days are present. [2025-09-16T19:31:32.268167] [DEBUG] [VALIDATOR ] MTZ: Primary data validation passed. [2025-09-16T19:31:32.271253] [INFO ] [MAIN ] MTZ: Fetched primary data is valid. [2025-09-16T19:31:32.271578] [DEBUG] [LOCAL ] MTZ: Saving primary data. [2025-09-16T19:31:32.328204] [DEBUG] [LOCAL ] MTZ: Trimmed 47573 records to 31729 (keeping 150 days of data) [2025-09-16T19:31:32.351884] [DEBUG] [VALIDATOR ] MUA: All expected trading days are present. [2025-09-16T19:31:32.352203] [DEBUG] [VALIDATOR ] MUA: Primary data validation passed. [2025-09-16T19:31:32.352373] [INFO ] [MAIN ] MUA: Fetched primary data is valid. [2025-09-16T19:31:32.352593] [DEBUG] [LOCAL ] MUA: Saving primary data. [2025-09-16T19:31:32.353190] [DEBUG] [LOCAL ] MUA: Trimmed 13571 records to 8428 (keeping 150 days of data) [2025-09-16T19:31:32.426390] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-08. [2025-09-16T19:31:32.502479] [DEBUG] [LOCAL ] MUA: Primary data saved (8428 records). [2025-09-16T19:31:32.505670] [INFO ] [MAIN ] MUA: Full history saved. [2025-09-16T19:31:32.506500] [INFO ] [MAIN ] Update Progress: 1427/2369 symbols completed. MUA took 23.4s. Estimated remaining: 1h 40m 0s. Rate: 9.4 symbols/min [2025-09-16T19:31:32.645385] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-09-16T19:31:32.711050] [INFO ] [MAIN ] MUFG: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:31:32.806737] [DEBUG] [LOCAL ] MUFG: Loaded 38767 existing records from primary storage. [2025-09-16T19:31:32.809941] [INFO ] [MAIN ] MUFG: Loaded 38767 existing records. [2025-09-16T19:31:32.809874] [DEBUG] [LOCAL ] MTZ: Primary data saved (31729 records). [2025-09-16T19:31:32.810045] [INFO ] [MAIN ] MTZ: Full history saved. [2025-09-16T19:31:32.842592] [INFO ] [MAIN ] Update Progress: 1428/2369 symbols completed. MTZ took 29.4s. Estimated remaining: 1h 39m 50s. Rate: 9.4 symbols/min [2025-09-16T19:31:33.135745] [WARN ] [MAIN ] MUFG: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:31:33.167861] [INFO ] [MAIN ] MUJ: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:31:33.166776] [INFO ] [MAIN ] MUFG: Wiping existing primary data before full refresh. [2025-09-16T19:31:33.168025] [INFO ] [LOCAL ] MUFG: Starting deletion of primary data. [2025-09-16T19:31:33.169469] [DEBUG] [LOCAL ] MUFG: Deleted primary data file. [2025-09-16T19:31:33.169833] [INFO ] [MAIN ] MUFG: Starting full primary refresh. [2025-09-16T19:31:33.175053] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-05. [2025-09-16T19:31:33.312411] [DEBUG] [LOCAL ] MUJ: Loaded 10188 existing records from primary storage. [2025-09-16T19:31:33.360005] [INFO ] [MAIN ] MUJ: Loaded 10188 existing records. [2025-09-16T19:31:33.713914] [WARN ] [MAIN ] MUJ: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:31:33.731194] [INFO ] [REMOTE ] MUFG: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:31:33.732072] [INFO ] [MAIN ] MUJ: Wiping existing primary data before full refresh. [2025-09-16T19:31:33.733261] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-09-16T19:31:33.733320] [INFO ] [LOCAL ] MUJ: Starting deletion of primary data. [2025-09-16T19:31:33.738385] [DEBUG] [LOCAL ] MUJ: Deleted primary data file. [2025-09-16T19:31:33.738441] [INFO ] [MAIN ] MUJ: Starting full primary refresh. [2025-09-16T19:31:33.983039] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-09. [2025-09-16T19:31:34.094104] [INFO ] [REMOTE ] MUJ: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:31:34.101566] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-09-16T19:31:34.774874] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-02. [2025-09-16T19:31:35.062599] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-09-16T19:31:35.066996] [INFO ] [REMOTE ] MUC: Full fetch complete (21408 records). [2025-09-16T19:31:35.067095] [DEBUG] [VALIDATOR ] MUC: Validating primary data. [2025-09-16T19:31:35.428728] [DEBUG] [VALIDATOR ] MUC: All expected trading days are present. [2025-09-16T19:31:35.428806] [DEBUG] [VALIDATOR ] MUC: Primary data validation passed. [2025-09-16T19:31:35.429251] [INFO ] [MAIN ] MUC: Fetched primary data is valid. [2025-09-16T19:31:35.429290] [DEBUG] [LOCAL ] MUC: Saving primary data. [2025-09-16T19:31:35.429940] [DEBUG] [LOCAL ] MUC: Trimmed 21408 records to 14448 (keeping 150 days of data) [2025-09-16T19:31:35.517288] [DEBUG] [LOCAL ] MUC: Primary data saved (14448 records). [2025-09-16T19:31:35.517421] [INFO ] [MAIN ] MUC: Full history saved. [2025-09-16T19:31:35.517515] [INFO ] [MAIN ] Update Progress: 1429/2369 symbols completed. MUC took 19.1s. Estimated remaining: 1h 39m 41s. Rate: 9.4 symbols/min [2025-09-16T19:31:35.583966] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-02. [2025-09-16T19:31:35.692682] [INFO ] [MAIN ] MUR: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:31:35.731274] [DEBUG] [LOCAL ] MUR: Loaded 38975 existing records from primary storage. [2025-09-16T19:31:35.731410] [INFO ] [MAIN ] MUR: Loaded 38975 existing records. [2025-09-16T19:31:35.906360] [WARN ] [MAIN ] MUR: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:31:35.906613] [INFO ] [MAIN ] MUR: Wiping existing primary data before full refresh. [2025-09-16T19:31:35.906676] [INFO ] [LOCAL ] MUR: Starting deletion of primary data. [2025-09-16T19:31:35.907504] [DEBUG] [LOCAL ] MUR: Deleted primary data file. [2025-09-16T19:31:35.907550] [INFO ] [MAIN ] MUR: Starting full primary refresh. [2025-09-16T19:31:36.254069] [INFO ] [REMOTE ] MUR: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:31:36.300196] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-09-16T19:31:36.374817] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-06. [2025-09-16T19:31:36.428952] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-09-16T19:31:37.174898] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-03. [2025-09-16T19:31:37.405064] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-09-16T19:31:37.945236] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-09-16T19:31:37.974924] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-02. [2025-09-16T19:31:38.774902] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-07. [2025-09-16T19:31:39.586687] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-03. [2025-09-16T19:31:39.608448] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-09-16T19:31:39.630903] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-09-16T19:31:40.374893] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-08. [2025-09-16T19:31:41.174946] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-03. [2025-09-16T19:31:41.284039] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-09-16T19:31:41.645284] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-03 (2.06 seconds). [2025-09-16T19:31:41.974908] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-09. [2025-09-16T19:31:42.726897] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-09-16T19:31:42.729034] [INFO ] [REMOTE ] MUE: Full fetch complete (12557 records). [2025-09-16T19:31:42.729104] [DEBUG] [VALIDATOR ] MUE: Validating primary data. [2025-09-16T19:31:42.784039] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-04. [2025-09-16T19:31:43.125002] [DEBUG] [VALIDATOR ] MUE: All expected trading days are present. [2025-09-16T19:31:43.127794] [DEBUG] [VALIDATOR ] MUE: Primary data validation passed. [2025-09-16T19:31:43.128085] [INFO ] [MAIN ] MUE: Fetched primary data is valid. [2025-09-16T19:31:43.128271] [DEBUG] [LOCAL ] MUE: Saving primary data. [2025-09-16T19:31:43.129958] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-09-16T19:31:43.140833] [DEBUG] [LOCAL ] MUE: Trimmed 12557 records to 9379 (keeping 150 days of data) [2025-09-16T19:31:43.197377] [DEBUG] [LOCAL ] MUE: Primary data saved (9379 records). [2025-09-16T19:31:43.197519] [INFO ] [MAIN ] MUE: Full history saved. [2025-09-16T19:31:43.197636] [INFO ] [MAIN ] Update Progress: 1430/2369 symbols completed. MUE took 22.2s. Estimated remaining: 1h 39m 36s. Rate: 9.4 symbols/min [2025-09-16T19:31:43.374762] [INFO ] [MAIN ] MUSA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:31:43.392999] [DEBUG] [LOCAL ] MUSA: Loaded 18513 existing records from primary storage. [2025-09-16T19:31:43.393124] [INFO ] [MAIN ] MUSA: Loaded 18513 existing records. [2025-09-16T19:31:43.567773] [WARN ] [MAIN ] MUSA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:31:43.568004] [INFO ] [MAIN ] MUSA: Wiping existing primary data before full refresh. [2025-09-16T19:31:43.568057] [INFO ] [LOCAL ] MUSA: Starting deletion of primary data. [2025-09-16T19:31:43.568587] [DEBUG] [LOCAL ] MUSA: Deleted primary data file. [2025-09-16T19:31:43.568631] [INFO ] [MAIN ] MUSA: Starting full primary refresh. [2025-09-16T19:31:43.581224] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-04. [2025-09-16T19:31:43.738877] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-09-16T19:31:43.938979] [INFO ] [REMOTE ] MUSA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:31:44.374932] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-02. [2025-09-16T19:31:45.174865] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-04. [2025-09-16T19:31:45.629843] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-09-16T19:31:45.736527] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-09-16T19:31:45.974928] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-03. [2025-09-16T19:31:46.689433] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-09-16T19:31:46.774893] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-05. [2025-09-16T19:31:47.020445] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-09-16T19:31:47.574961] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-05. [2025-09-16T19:31:48.374901] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-04. [2025-09-16T19:31:48.493816] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-09-16T19:31:48.561131] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-09-16T19:31:49.174884] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-06. [2025-09-16T19:31:49.552816] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-09-16T19:31:49.938098] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-09-16T19:31:49.974871] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-06. [2025-09-16T19:31:50.775004] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-07. [2025-09-16T19:31:51.574879] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-05. [2025-09-16T19:31:51.645448] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-09-16T19:31:51.665091] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-09-16T19:31:52.374885] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-07. [2025-09-16T19:31:52.971491] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-09-16T19:31:53.174897] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-06. [2025-09-16T19:31:53.983632] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-08. [2025-09-16T19:31:53.986691] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-09-16T19:31:54.338710] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-09-16T19:31:54.774924] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-07. [2025-09-16T19:31:54.900964] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-09-16T19:31:55.574948] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-05. [2025-09-16T19:31:56.053791] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-09-16T19:31:56.374943] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-08. [2025-09-16T19:31:57.110822] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-09-16T19:31:57.174872] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-06. [2025-09-16T19:31:57.974953] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-08. [2025-09-16T19:31:58.175570] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-09-16T19:31:58.759241] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-09-16T19:31:58.774845] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-09. [2025-09-16T19:31:58.987494] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-09-16T19:31:59.574944] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-07. [2025-09-16T19:32:00.072439] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T19:32:00.077250] [INFO ] [REMOTE ] MUSA: Full fetch complete (27540 records). [2025-09-16T19:32:00.077381] [DEBUG] [VALIDATOR ] MUSA: Validating primary data. [2025-09-16T19:32:00.380032] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-09. [2025-09-16T19:32:00.453063] [DEBUG] [VALIDATOR ] MUSA: All expected trading days are present. [2025-09-16T19:32:00.453143] [DEBUG] [VALIDATOR ] MUSA: Primary data validation passed. [2025-09-16T19:32:00.453739] [INFO ] [MAIN ] MUSA: Fetched primary data is valid. [2025-09-16T19:32:00.453799] [DEBUG] [LOCAL ] MUSA: Saving primary data. [2025-09-16T19:32:00.454636] [DEBUG] [LOCAL ] MUSA: Trimmed 27540 records to 18704 (keeping 150 days of data) [2025-09-16T19:32:00.573301] [DEBUG] [LOCAL ] MUSA: Primary data saved (18704 records). [2025-09-16T19:32:00.573454] [INFO ] [MAIN ] MUSA: Full history saved. [2025-09-16T19:32:00.573578] [INFO ] [MAIN ] Update Progress: 1431/2369 symbols completed. MUSA took 17.4s. Estimated remaining: 1h 39m 36s. Rate: 9.4 symbols/min [2025-09-16T19:32:00.750163] [INFO ] [MAIN ] MUSE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:00.750387] [DEBUG] [LOCAL ] MUSE: No existing data file found in primary storage. [2025-09-16T19:32:00.750433] [WARN ] [MAIN ] MUSE: No existing data found, performing full refresh instead. [2025-09-16T19:32:00.750578] [INFO ] [MAIN ] MUSE: Wiping existing primary data before full refresh. [2025-09-16T19:32:00.750619] [INFO ] [LOCAL ] MUSE: Starting deletion of primary data. [2025-09-16T19:32:00.750687] [DEBUG] [LOCAL ] MUSE: No primary data file found to delete. [2025-09-16T19:32:00.750721] [INFO ] [MAIN ] MUSE: Starting full primary refresh. [2025-09-16T19:32:01.109515] [INFO ] [REMOTE ] MUSE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:01.174930] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-09. [2025-09-16T19:32:01.613394] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-09-16T19:32:01.974897] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-08. [2025-09-16T19:32:02.120675] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-09-16T19:32:02.124246] [INFO ] [REMOTE ] MUJ: Full fetch complete (13866 records). [2025-09-16T19:32:02.124359] [DEBUG] [VALIDATOR ] MUJ: Validating primary data. [2025-09-16T19:32:02.458543] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-09-16T19:32:02.517987] [INFO ] [REMOTE ] MUFG: Full fetch complete (57164 records). [2025-09-16T19:32:02.525407] [DEBUG] [VALIDATOR ] MUJ: All expected trading days are present. [2025-09-16T19:32:02.525535] [DEBUG] [VALIDATOR ] MUFG: Validating primary data. [2025-09-16T19:32:02.525624] [DEBUG] [VALIDATOR ] MUJ: Primary data validation passed. [2025-09-16T19:32:02.526163] [INFO ] [MAIN ] MUJ: Fetched primary data is valid. [2025-09-16T19:32:02.526288] [DEBUG] [LOCAL ] MUJ: Saving primary data. [2025-09-16T19:32:02.618911] [DEBUG] [LOCAL ] MUJ: Trimmed 13866 records to 10300 (keeping 150 days of data) [2025-09-16T19:32:02.801985] [INFO ] [REMOTE ] MUSE: Fetching primary data for 2025-02. [2025-09-16T19:32:02.968812] [DEBUG] [VALIDATOR ] MUFG: All expected trading days are present. [2025-09-16T19:32:02.971777] [DEBUG] [VALIDATOR ] MUFG: Primary data validation passed. [2025-09-16T19:32:02.973344] [INFO ] [MAIN ] MUFG: Fetched primary data is valid. [2025-09-16T19:32:02.973917] [DEBUG] [LOCAL ] MUFG: Saving primary data. [2025-09-16T19:32:02.977346] [DEBUG] [LOCAL ] MUFG: Trimmed 57164 records to 39152 (keeping 150 days of data) [2025-09-16T19:32:02.981031] [DEBUG] [LOCAL ] MUJ: Primary data saved (10300 records). [2025-09-16T19:32:02.981160] [INFO ] [MAIN ] MUJ: Full history saved. [2025-09-16T19:32:02.981275] [INFO ] [MAIN ] Update Progress: 1432/2369 symbols completed. MUJ took 30.1s. Estimated remaining: 1h 39m 27s. Rate: 9.4 symbols/min [2025-09-16T19:32:03.111997] [DEBUG] [REMOTE ] MUSE: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:32:03.170331] [WARN ] [REMOTE ] MUSE: No data returned for month 2025-02 [2025-09-16T19:32:03.278596] [INFO ] [MAIN ] MUX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:03.408958] [DEBUG] [LOCAL ] MUFG: Primary data saved (39152 records). [2025-09-16T19:32:03.409234] [INFO ] [MAIN ] MUFG: Full history saved. [2025-09-16T19:32:03.409393] [INFO ] [MAIN ] Update Progress: 1433/2369 symbols completed. MUFG took 30.9s. Estimated remaining: 1h 39m 17s. Rate: 9.4 symbols/min [2025-09-16T19:32:03.506391] [DEBUG] [LOCAL ] MUX: Loaded 30886 existing records from primary storage. [2025-09-16T19:32:03.522586] [INFO ] [MAIN ] MUX: Loaded 30886 existing records. [2025-09-16T19:32:03.576010] [INFO ] [REMOTE ] MUSE: Fetching primary data for 2025-03. [2025-09-16T19:32:03.700410] [INFO ] [MAIN ] MVF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:03.773267] [WARN ] [MAIN ] MUX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:32:03.773763] [INFO ] [MAIN ] MUX: Wiping existing primary data before full refresh. [2025-09-16T19:32:03.773832] [INFO ] [LOCAL ] MUX: Starting deletion of primary data. [2025-09-16T19:32:03.779141] [DEBUG] [LOCAL ] MUX: Deleted primary data file. [2025-09-16T19:32:03.779312] [INFO ] [MAIN ] MUX: Starting full primary refresh. [2025-09-16T19:32:03.898067] [DEBUG] [LOCAL ] MVF: Loaded 8769 existing records from primary storage. [2025-09-16T19:32:03.905750] [INFO ] [MAIN ] MVF: Loaded 8769 existing records. [2025-09-16T19:32:03.905668] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-09-16T19:32:04.022726] [INFO ] [REMOTE ] MUSE: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-09-16T19:32:04.286533] [WARN ] [MAIN ] MVF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:32:04.286815] [INFO ] [MAIN ] MVF: Wiping existing primary data before full refresh. [2025-09-16T19:32:04.287064] [INFO ] [LOCAL ] MVF: Starting deletion of primary data. [2025-09-16T19:32:04.302818] [DEBUG] [LOCAL ] MVF: Deleted primary data file. [2025-09-16T19:32:04.313061] [INFO ] [MAIN ] MVF: Starting full primary refresh. [2025-09-16T19:32:04.379136] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-09. [2025-09-16T19:32:04.462883] [INFO ] [REMOTE ] MUX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:04.743314] [INFO ] [REMOTE ] MVF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:05.174910] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-02. [2025-09-16T19:32:05.974880] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-02. [2025-09-16T19:32:06.125810] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-09-16T19:32:06.132342] [INFO ] [REMOTE ] MUR: Full fetch complete (57587 records). [2025-09-16T19:32:06.132448] [DEBUG] [VALIDATOR ] MUR: Validating primary data. [2025-09-16T19:32:06.517547] [DEBUG] [VALIDATOR ] MUR: All expected trading days are present. [2025-09-16T19:32:06.517716] [DEBUG] [VALIDATOR ] MUR: Primary data validation passed. [2025-09-16T19:32:06.518715] [INFO ] [MAIN ] MUR: Fetched primary data is valid. [2025-09-16T19:32:06.518807] [DEBUG] [LOCAL ] MUR: Saving primary data. [2025-09-16T19:32:06.527244] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-09-16T19:32:06.530531] [DEBUG] [LOCAL ] MUR: Trimmed 57587 records to 39364 (keeping 150 days of data) [2025-09-16T19:32:06.648571] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-09-16T19:32:06.777570] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-03. [2025-09-16T19:32:06.815328] [DEBUG] [LOCAL ] MUR: Primary data saved (39364 records). [2025-09-16T19:32:06.815505] [INFO ] [MAIN ] MUR: Full history saved. [2025-09-16T19:32:06.816209] [INFO ] [MAIN ] Update Progress: 1434/2369 symbols completed. MUR took 31.3s. Estimated remaining: 1h 39m 9s. Rate: 9.4 symbols/min [2025-09-16T19:32:07.002260] [INFO ] [MAIN ] MVO: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:07.011883] [DEBUG] [LOCAL ] MVO: Loaded 6388 existing records from primary storage. [2025-09-16T19:32:07.012106] [INFO ] [MAIN ] MVO: Loaded 6388 existing records. [2025-09-16T19:32:07.202378] [WARN ] [MAIN ] MVO: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:32:07.202638] [INFO ] [MAIN ] MVO: Wiping existing primary data before full refresh. [2025-09-16T19:32:07.202706] [INFO ] [LOCAL ] MVO: Starting deletion of primary data. [2025-09-16T19:32:07.203036] [DEBUG] [LOCAL ] MVO: Deleted primary data file. [2025-09-16T19:32:07.203084] [INFO ] [MAIN ] MVO: Starting full primary refresh. [2025-09-16T19:32:07.558487] [INFO ] [REMOTE ] MVO: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:07.574945] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-03. [2025-09-16T19:32:07.677741] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-09-16T19:32:08.374935] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-04. [2025-09-16T19:32:08.973362] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-09-16T19:32:09.121447] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-09-16T19:32:09.174944] [INFO ] [REMOTE ] MUSE: Fetching primary data for 2025-04. [2025-09-16T19:32:09.410284] [INFO ] [REMOTE ] MUSE: Incremental fetch complete for 2025-04 (0.24 seconds). [2025-09-16T19:32:09.974904] [INFO ] [REMOTE ] MUSE: Fetching primary data for 2025-05. [2025-09-16T19:32:10.188627] [INFO ] [REMOTE ] MUSE: Incremental fetch complete for 2025-05 (0.21 seconds). [2025-09-16T19:32:10.774945] [INFO ] [REMOTE ] MUSE: Fetching primary data for 2025-06. [2025-09-16T19:32:11.017664] [INFO ] [REMOTE ] MUSE: Incremental fetch complete for 2025-06 (0.24 seconds). [2025-09-16T19:32:11.574923] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-02. [2025-09-16T19:32:12.293116] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-09-16T19:32:12.374945] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-04. [2025-09-16T19:32:13.174947] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-05. [2025-09-16T19:32:13.847974] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-09-16T19:32:13.948911] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-09-16T19:32:13.974858] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-06. [2025-09-16T19:32:14.774919] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-05. [2025-09-16T19:32:14.802836] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-09-16T19:32:15.574918] [INFO ] [REMOTE ] MUSE: Fetching primary data for 2025-07. [2025-09-16T19:32:15.803680] [INFO ] [REMOTE ] MUSE: Incremental fetch complete for 2025-07 (0.23 seconds). [2025-09-16T19:32:16.001491] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-09-16T19:32:16.374914] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-07. [2025-09-16T19:32:17.174903] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-03. [2025-09-16T19:32:17.246043] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-09-16T19:32:17.755756] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T19:32:17.974941] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-08. [2025-09-16T19:32:18.774902] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-06. [2025-09-16T19:32:18.817946] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-09-16T19:32:19.574932] [INFO ] [REMOTE ] MUSE: Fetching primary data for 2025-08. [2025-09-16T19:32:19.828488] [INFO ] [REMOTE ] MUSE: Incremental fetch complete for 2025-08 (0.25 seconds). [2025-09-16T19:32:20.332064] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-09-16T19:32:20.374858] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-07. [2025-09-16T19:32:21.174934] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-04. [2025-09-16T19:32:21.959877] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-09-16T19:32:21.976615] [INFO ] [REMOTE ] MUSE: Fetching primary data for 2025-09. [2025-09-16T19:32:21.977597] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-09-16T19:32:22.397136] [DEBUG] [REMOTE ] MUSE: Error fetching primary data for 2025-09: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-09-16T19:32:22.397506] [WARN ] [REMOTE ] MUSE: No data returned for month 2025-09 [2025-09-16T19:32:22.397611] [INFO ] [REMOTE ] MUSE: Full fetch complete (95 records). [2025-09-16T19:32:22.397713] [DEBUG] [VALIDATOR ] MUSE: Validating primary data. [2025-09-16T19:32:22.398411] [FATAL] [VALIDATOR ] MUSE: Data is stale - most recent date (2025-08-05) is before last completed trading day (2025-09-16). Symbol may be delisted or halted. [2025-09-16T19:32:22.398494] [FATAL] [MAIN ] MUSE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:32:22.398547] [WARN ] [MAIN ] MUSE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:32:22.398602] [INFO ] [LOCAL ] MUSE: Starting deletion of primary data. [2025-09-16T19:32:22.398692] [DEBUG] [LOCAL ] MUSE: No primary data file found to delete. [2025-09-16T19:32:22.398740] [INFO ] [LOCAL ] MUSE: Starting deletion of derivative data. [2025-09-16T19:32:22.398809] [DEBUG] [LOCAL ] MUSE: No derivative data file found to delete. [2025-09-16T19:32:22.398904] [INFO ] [MAIN ] Update Progress: 1435/2369 symbols completed. MUSE took 21.8s. Estimated remaining: 1h 39m 9s. Rate: 9.4 symbols/min [2025-09-16T19:32:22.583393] [INFO ] [MAIN ] MVT: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:22.592540] [DEBUG] [LOCAL ] MVT: Loaded 7340 existing records from primary storage. [2025-09-16T19:32:22.592718] [INFO ] [MAIN ] MVT: Loaded 7340 existing records. [2025-09-16T19:32:22.776478] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-09. [2025-09-16T19:32:22.776355] [WARN ] [MAIN ] MVT: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:32:22.777832] [INFO ] [MAIN ] MVT: Wiping existing primary data before full refresh. [2025-09-16T19:32:22.777904] [INFO ] [LOCAL ] MVT: Starting deletion of primary data. [2025-09-16T19:32:22.778291] [DEBUG] [LOCAL ] MVT: Deleted primary data file. [2025-09-16T19:32:22.778366] [INFO ] [MAIN ] MVT: Starting full primary refresh. [2025-09-16T19:32:23.128079] [INFO ] [REMOTE ] MVT: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:23.574945] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-08. [2025-09-16T19:32:23.670582] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-09-16T19:32:23.673973] [INFO ] [REMOTE ] MVF: Full fetch complete (13823 records). [2025-09-16T19:32:23.674179] [DEBUG] [VALIDATOR ] MVF: Validating primary data. [2025-09-16T19:32:24.071474] [DEBUG] [VALIDATOR ] MVF: All expected trading days are present. [2025-09-16T19:32:24.071562] [DEBUG] [VALIDATOR ] MVF: Primary data validation passed. [2025-09-16T19:32:24.071993] [INFO ] [MAIN ] MVF: Fetched primary data is valid. [2025-09-16T19:32:24.072046] [DEBUG] [LOCAL ] MVF: Saving primary data. [2025-09-16T19:32:24.072561] [DEBUG] [LOCAL ] MVF: Trimmed 13823 records to 8944 (keeping 150 days of data) [2025-09-16T19:32:24.128663] [DEBUG] [LOCAL ] MVF: Primary data saved (8944 records). [2025-09-16T19:32:24.128810] [INFO ] [MAIN ] MVF: Full history saved. [2025-09-16T19:32:24.128945] [INFO ] [MAIN ] Update Progress: 1436/2369 symbols completed. MVF took 20.7s. Estimated remaining: 1h 38m 59s. Rate: 9.4 symbols/min [2025-09-16T19:32:24.305248] [INFO ] [MAIN ] MWA: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:24.340168] [DEBUG] [LOCAL ] MWA: Loaded 33785 existing records from primary storage. [2025-09-16T19:32:24.340312] [INFO ] [MAIN ] MWA: Loaded 33785 existing records. [2025-09-16T19:32:24.385073] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-02. [2025-09-16T19:32:24.526285] [WARN ] [MAIN ] MWA: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:32:24.526547] [INFO ] [MAIN ] MWA: Wiping existing primary data before full refresh. [2025-09-16T19:32:24.526594] [INFO ] [LOCAL ] MWA: Starting deletion of primary data. [2025-09-16T19:32:24.527377] [DEBUG] [LOCAL ] MWA: Deleted primary data file. [2025-09-16T19:32:24.527425] [INFO ] [MAIN ] MWA: Starting full primary refresh. [2025-09-16T19:32:24.891406] [INFO ] [REMOTE ] MWA: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:24.974194] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-09-16T19:32:25.174905] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-03. [2025-09-16T19:32:25.456923] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-09-16T19:32:25.974919] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-09. [2025-09-16T19:32:25.974804] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-09-16T19:32:26.774906] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-04. [2025-09-16T19:32:27.278768] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-09-16T19:32:27.284844] [INFO ] [REMOTE ] MUX: Full fetch complete (45641 records). [2025-09-16T19:32:27.284943] [DEBUG] [VALIDATOR ] MUX: Validating primary data. [2025-09-16T19:32:27.585277] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-05. [2025-09-16T19:32:27.683191] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-09-16T19:32:27.698078] [DEBUG] [VALIDATOR ] MUX: All expected trading days are present. [2025-09-16T19:32:27.698891] [DEBUG] [VALIDATOR ] MUX: Primary data validation passed. [2025-09-16T19:32:27.699729] [INFO ] [MAIN ] MUX: Fetched primary data is valid. [2025-09-16T19:32:27.699781] [DEBUG] [LOCAL ] MUX: Saving primary data. [2025-09-16T19:32:27.700903] [DEBUG] [LOCAL ] MUX: Trimmed 45641 records to 31233 (keeping 150 days of data) [2025-09-16T19:32:27.889100] [DEBUG] [LOCAL ] MUX: Primary data saved (31233 records). [2025-09-16T19:32:27.889247] [INFO ] [MAIN ] MUX: Full history saved. [2025-09-16T19:32:27.889933] [INFO ] [MAIN ] Update Progress: 1437/2369 symbols completed. MUX took 24.9s. Estimated remaining: 1h 38m 51s. Rate: 9.4 symbols/min [2025-09-16T19:32:28.313986] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T19:32:28.374917] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-06. [2025-09-16T19:32:29.174974] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-05. [2025-09-16T19:32:29.205117] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-09-16T19:32:29.708744] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-09-16T19:32:29.974951] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-02. [2025-09-16T19:32:30.774899] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-06. [2025-09-16T19:32:31.487934] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-09-16T19:32:31.528174] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-09-16T19:32:31.574857] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-07. [2025-09-16T19:32:32.362419] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-09-16T19:32:32.374841] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-08. [2025-09-16T19:32:33.159430] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-09-16T19:32:33.174843] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-03. [2025-09-16T19:32:33.974936] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-07. [2025-09-16T19:32:34.774899] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-09. [2025-09-16T19:32:35.032475] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-09-16T19:32:35.054397] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-09-16T19:32:35.574901] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-08. [2025-09-16T19:32:35.591716] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-09-16T19:32:35.593623] [INFO ] [REMOTE ] MVT: Full fetch complete (10504 records). [2025-09-16T19:32:35.593758] [DEBUG] [VALIDATOR ] MVT: Validating primary data. [2025-09-16T19:32:35.963215] [DEBUG] [VALIDATOR ] MVT: All expected trading days are present. [2025-09-16T19:32:35.963298] [DEBUG] [VALIDATOR ] MVT: Primary data validation passed. [2025-09-16T19:32:35.963572] [INFO ] [MAIN ] MVT: Fetched primary data is valid. [2025-09-16T19:32:35.963637] [DEBUG] [LOCAL ] MVT: Saving primary data. [2025-09-16T19:32:35.964121] [DEBUG] [LOCAL ] MVT: Trimmed 10504 records to 7404 (keeping 150 days of data) [2025-09-16T19:32:36.011996] [DEBUG] [LOCAL ] MVT: Primary data saved (7404 records). [2025-09-16T19:32:36.012114] [INFO ] [MAIN ] MVT: Full history saved. [2025-09-16T19:32:36.012228] [INFO ] [MAIN ] Update Progress: 1438/2369 symbols completed. MVT took 13.6s. Estimated remaining: 1h 38m 46s. Rate: 9.4 symbols/min [2025-09-16T19:32:36.324135] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-09-16T19:32:36.374900] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-04. [2025-09-16T19:32:37.174963] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-09. [2025-09-16T19:32:37.899967] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-09-16T19:32:37.901962] [INFO ] [REMOTE ] MVO: Full fetch complete (10013 records). [2025-09-16T19:32:37.902052] [DEBUG] [VALIDATOR ] MVO: Validating primary data. [2025-09-16T19:32:38.284819] [DEBUG] [VALIDATOR ] MVO: All expected trading days are present. [2025-09-16T19:32:38.284959] [DEBUG] [VALIDATOR ] MVO: Primary data validation passed. [2025-09-16T19:32:38.285219] [INFO ] [MAIN ] MVO: Fetched primary data is valid. [2025-09-16T19:32:38.285272] [DEBUG] [LOCAL ] MVO: Saving primary data. [2025-09-16T19:32:38.285893] [DEBUG] [LOCAL ] MVO: Trimmed 10013 records to 6446 (keeping 150 days of data) [2025-09-16T19:32:38.349304] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-09-16T19:32:38.362225] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-05. [2025-09-16T19:32:38.365143] [DEBUG] [LOCAL ] MVO: Primary data saved (6446 records). [2025-09-16T19:32:38.365299] [INFO ] [MAIN ] MVO: Full history saved. [2025-09-16T19:32:38.365418] [INFO ] [MAIN ] Update Progress: 1439/2369 symbols completed. MVO took 31.5s. Estimated remaining: 1h 38m 37s. Rate: 9.4 symbols/min [2025-09-16T19:32:39.982995] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-09-16T19:32:39.993964] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-06. [2025-09-16T19:32:41.533941] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-09-16T19:32:41.543834] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-07. [2025-09-16T19:32:43.025153] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-09-16T19:32:43.034981] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-08. [2025-09-16T19:32:44.726789] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-09-16T19:32:44.736366] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-09. [2025-09-16T19:32:46.271983] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-09-16T19:32:46.277062] [INFO ] [REMOTE ] MWA: Full fetch complete (51448 records). [2025-09-16T19:32:46.277183] [DEBUG] [VALIDATOR ] MWA: Validating primary data. [2025-09-16T19:32:46.660802] [DEBUG] [VALIDATOR ] MWA: All expected trading days are present. [2025-09-16T19:32:46.660910] [DEBUG] [VALIDATOR ] MWA: Primary data validation passed. [2025-09-16T19:32:46.661749] [INFO ] [MAIN ] MWA: Fetched primary data is valid. [2025-09-16T19:32:46.661797] [DEBUG] [LOCAL ] MWA: Saving primary data. [2025-09-16T19:32:46.663924] [DEBUG] [LOCAL ] MWA: Trimmed 51448 records to 34072 (keeping 150 days of data) [2025-09-16T19:32:46.871623] [DEBUG] [LOCAL ] MWA: Primary data saved (34072 records). [2025-09-16T19:32:46.871781] [INFO ] [MAIN ] MWA: Full history saved. [2025-09-16T19:32:46.871911] [INFO ] [MAIN ] Update Progress: 1440/2369 symbols completed. MWA took 22.7s. Estimated remaining: 1h 38m 32s. Rate: 9.4 symbols/min [2025-09-16T19:32:47.432813] [INFO ] [MAIN ] MX: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:47.455002] [INFO ] [MAIN ] MYD: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:47.558900] [INFO ] [MAIN ] MXE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:47.587272] [INFO ] [MAIN ] MXF: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:47.587666] [DEBUG] [LOCAL ] MXE: No existing data file found in primary storage. [2025-09-16T19:32:47.592980] [WARN ] [MAIN ] MXE: No existing data found, performing full refresh instead. [2025-09-16T19:32:47.593209] [INFO ] [MAIN ] MXE: Wiping existing primary data before full refresh. [2025-09-16T19:32:47.599294] [DEBUG] [LOCAL ] MYD: Loaded 9151 existing records from primary storage. [2025-09-16T19:32:47.599552] [INFO ] [MAIN ] MYD: Loaded 9151 existing records. [2025-09-16T19:32:47.599029] [INFO ] [LOCAL ] MXE: Starting deletion of primary data. [2025-09-16T19:32:47.599684] [DEBUG] [LOCAL ] MXE: No primary data file found to delete. [2025-09-16T19:32:47.604850] [INFO ] [MAIN ] MXE: Starting full primary refresh. [2025-09-16T19:32:47.665062] [DEBUG] [LOCAL ] MX: Loaded 16505 existing records from primary storage. [2025-09-16T19:32:47.741085] [INFO ] [MAIN ] MX: Loaded 16505 existing records. [2025-09-16T19:32:47.757758] [DEBUG] [LOCAL ] MXF: Loaded 4055 existing records from primary storage. [2025-09-16T19:32:47.824105] [INFO ] [MAIN ] MXF: Loaded 4055 existing records. [2025-09-16T19:32:48.112011] [WARN ] [MAIN ] MYD: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:32:48.199330] [INFO ] [MAIN ] MYD: Wiping existing primary data before full refresh. [2025-09-16T19:32:48.234629] [INFO ] [LOCAL ] MYD: Starting deletion of primary data. [2025-09-16T19:32:48.314117] [DEBUG] [LOCAL ] MYD: Deleted primary data file. [2025-09-16T19:32:48.372501] [INFO ] [MAIN ] MYD: Starting full primary refresh. [2025-09-16T19:32:48.421528] [WARN ] [MAIN ] MXF: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:32:48.475759] [INFO ] [MAIN ] MXF: Wiping existing primary data before full refresh. [2025-09-16T19:32:48.475465] [WARN ] [MAIN ] MX: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:32:48.488723] [INFO ] [LOCAL ] MXF: Starting deletion of primary data. [2025-09-16T19:32:48.504943] [INFO ] [MAIN ] MX: Wiping existing primary data before full refresh. [2025-09-16T19:32:48.524870] [INFO ] [LOCAL ] MX: Starting deletion of primary data. [2025-09-16T19:32:48.559687] [DEBUG] [LOCAL ] MXF: Deleted primary data file. [2025-09-16T19:32:48.570183] [INFO ] [MAIN ] MXF: Starting full primary refresh. [2025-09-16T19:32:48.575375] [DEBUG] [LOCAL ] MX: Deleted primary data file. [2025-09-16T19:32:48.615471] [INFO ] [MAIN ] MX: Starting full primary refresh. [2025-09-16T19:32:48.697415] [INFO ] [REMOTE ] MXE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:48.741479] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-02. [2025-09-16T19:32:49.477286] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-09-16T19:32:49.486435] [INFO ] [REMOTE ] MYD: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:49.522191] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-02. [2025-09-16T19:32:49.521869] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-03. [2025-09-16T19:32:49.600345] [INFO ] [REMOTE ] MX: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:49.649143] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-02. [2025-09-16T19:32:49.670009] [INFO ] [REMOTE ] MXF: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:49.671207] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-02. [2025-09-16T19:32:50.054291] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-09-16T19:32:50.055332] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-04. [2025-09-16T19:32:50.243689] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-09-16T19:32:50.245915] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-03. [2025-09-16T19:32:50.325768] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-09-16T19:32:50.329283] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-03. [2025-09-16T19:32:50.415710] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-09-16T19:32:50.421346] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-03. [2025-09-16T19:32:50.494324] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-09-16T19:32:50.495476] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-05. [2025-09-16T19:32:50.819958] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-09-16T19:32:50.821165] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-06. [2025-09-16T19:32:50.828688] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-09-16T19:32:50.831356] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-04. [2025-09-16T19:32:50.914812] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-09-16T19:32:51.193368] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-09-16T19:32:51.375938] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-09-16T19:32:51.574911] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-04. [2025-09-16T19:32:51.709770] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-09-16T19:32:52.374921] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-05. [2025-09-16T19:32:52.457386] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-09-16T19:32:52.980606] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-09-16T19:32:53.174907] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-06. [2025-09-16T19:32:53.774547] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-09-16T19:32:53.974909] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-07. [2025-09-16T19:32:54.422313] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-09-16T19:32:54.774968] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-07. [2025-09-16T19:32:55.436485] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-09-16T19:32:55.574907] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-08. [2025-09-16T19:32:56.064574] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-09-16T19:32:56.374917] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-09. [2025-09-16T19:32:56.874893] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-09-16T19:32:56.875861] [INFO ] [REMOTE ] MXF: Full fetch complete (6935 records). [2025-09-16T19:32:56.875943] [DEBUG] [VALIDATOR ] MXF: Validating primary data. [2025-09-16T19:32:57.180024] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-05. [2025-09-16T19:32:57.243910] [DEBUG] [VALIDATOR ] MXF: All expected trading days are present. [2025-09-16T19:32:57.244004] [DEBUG] [VALIDATOR ] MXF: Primary data validation passed. [2025-09-16T19:32:57.244277] [INFO ] [MAIN ] MXF: Fetched primary data is valid. [2025-09-16T19:32:57.244322] [DEBUG] [LOCAL ] MXF: Saving primary data. [2025-09-16T19:32:57.244878] [DEBUG] [LOCAL ] MXF: Trimmed 6935 records to 4061 (keeping 150 days of data) [2025-09-16T19:32:57.271144] [DEBUG] [LOCAL ] MXF: Primary data saved (4061 records). [2025-09-16T19:32:57.271298] [INFO ] [MAIN ] MXF: Full history saved. [2025-09-16T19:32:57.271405] [INFO ] [MAIN ] Update Progress: 1441/2369 symbols completed. MXF took 10.4s. Estimated remaining: 1h 38m 28s. Rate: 9.4 symbols/min [2025-09-16T19:32:57.440108] [INFO ] [MAIN ] MYE: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:32:57.458602] [DEBUG] [LOCAL ] MYE: Loaded 17475 existing records from primary storage. [2025-09-16T19:32:57.458748] [INFO ] [MAIN ] MYE: Loaded 17475 existing records. [2025-09-16T19:32:57.639860] [WARN ] [MAIN ] MYE: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:32:57.640104] [INFO ] [MAIN ] MYE: Wiping existing primary data before full refresh. [2025-09-16T19:32:57.640151] [INFO ] [LOCAL ] MYE: Starting deletion of primary data. [2025-09-16T19:32:57.640920] [DEBUG] [LOCAL ] MYE: Deleted primary data file. [2025-09-16T19:32:57.640977] [INFO ] [MAIN ] MYE: Starting full primary refresh. [2025-09-16T19:32:57.993233] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-08. [2025-09-16T19:32:57.997224] [INFO ] [REMOTE ] MYE: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:32:58.059241] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-09-16T19:32:58.367241] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-09-16T19:32:58.774914] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-02. [2025-09-16T19:32:59.574872] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-09. [2025-09-16T19:32:59.925127] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-09-16T19:33:00.132455] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-09-16T19:33:00.133444] [INFO ] [REMOTE ] MXE: Full fetch complete (1819 records). [2025-09-16T19:33:00.133523] [DEBUG] [VALIDATOR ] MXE: Validating primary data. [2025-09-16T19:33:00.339108] [WARN ] [VALIDATOR ] MXE: Validation failed - insufficient historical data. Have 148 unique days, need 150 days [2025-09-16T19:33:00.339362] [FATAL] [MAIN ] MXE: Fetched primary data is invalid. Refusing to save it. [2025-09-16T19:33:00.339433] [WARN ] [MAIN ] MXE: Deleting existing primary and derivative data due to validation failure. [2025-09-16T19:33:00.339499] [INFO ] [LOCAL ] MXE: Starting deletion of primary data. [2025-09-16T19:33:00.339574] [DEBUG] [LOCAL ] MXE: No primary data file found to delete. [2025-09-16T19:33:00.339612] [INFO ] [LOCAL ] MXE: Starting deletion of derivative data. [2025-09-16T19:33:00.339673] [DEBUG] [LOCAL ] MXE: No derivative data file found to delete. [2025-09-16T19:33:00.339739] [INFO ] [MAIN ] Update Progress: 1442/2369 symbols completed. MXE took 13.5s. Estimated remaining: 1h 38m 20s. Rate: 9.4 symbols/min [2025-09-16T19:33:00.380071] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-04. [2025-09-16T19:33:00.518471] [INFO ] [MAIN ] MYI: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:33:00.535722] [DEBUG] [LOCAL ] MYI: Loaded 13412 existing records from primary storage. [2025-09-16T19:33:00.535867] [INFO ] [MAIN ] MYI: Loaded 13412 existing records. [2025-09-16T19:33:00.712890] [WARN ] [MAIN ] MYI: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:33:00.713183] [INFO ] [MAIN ] MYI: Wiping existing primary data before full refresh. [2025-09-16T19:33:00.713228] [INFO ] [LOCAL ] MYI: Starting deletion of primary data. [2025-09-16T19:33:00.713763] [DEBUG] [LOCAL ] MYI: Deleted primary data file. [2025-09-16T19:33:00.713811] [INFO ] [MAIN ] MYI: Starting full primary refresh. [2025-09-16T19:33:01.064292] [INFO ] [REMOTE ] MYI: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:33:01.174891] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-02. [2025-09-16T19:33:01.502875] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-09-16T19:33:01.974922] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-05. [2025-09-16T19:33:02.212094] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-09-16T19:33:02.774867] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-06. [2025-09-16T19:33:03.094688] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-09-16T19:33:03.542212] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-09-16T19:33:03.574842] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-07. [2025-09-16T19:33:04.374886] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-03. [2025-09-16T19:33:04.495104] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-09-16T19:33:05.174920] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-06. [2025-09-16T19:33:05.232578] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-09-16T19:33:05.861114] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-09-16T19:33:05.974895] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-07. [2025-09-16T19:33:06.774931] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-03. [2025-09-16T19:33:06.991116] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-09-16T19:33:07.574902] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-08. [2025-09-16T19:33:08.056334] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-09-16T19:33:08.374908] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-08. [2025-09-16T19:33:08.931697] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-09-16T19:33:09.174900] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-09. [2025-09-16T19:33:09.974951] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-04. [2025-09-16T19:33:10.326863] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-09-16T19:33:10.329807] [INFO ] [REMOTE ] MX: Full fetch complete (24706 records). [2025-09-16T19:33:10.329887] [DEBUG] [VALIDATOR ] MX: Validating primary data. [2025-09-16T19:33:10.694724] [DEBUG] [VALIDATOR ] MX: All expected trading days are present. [2025-09-16T19:33:10.694815] [DEBUG] [VALIDATOR ] MX: Primary data validation passed. [2025-09-16T19:33:10.695404] [INFO ] [MAIN ] MX: Fetched primary data is valid. [2025-09-16T19:33:10.695458] [DEBUG] [LOCAL ] MX: Saving primary data. [2025-09-16T19:33:10.696162] [DEBUG] [LOCAL ] MX: Trimmed 24706 records to 16635 (keeping 150 days of data) [2025-09-16T19:33:10.776210] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-04. [2025-09-16T19:33:10.797538] [DEBUG] [LOCAL ] MX: Primary data saved (16635 records). [2025-09-16T19:33:10.797661] [INFO ] [MAIN ] MX: Full history saved. [2025-09-16T19:33:10.797755] [INFO ] [MAIN ] Update Progress: 1443/2369 symbols completed. MX took 23.9s. Estimated remaining: 1h 38m 16s. Rate: 9.4 symbols/min [2025-09-16T19:33:10.970144] [INFO ] [MAIN ] MYN: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:33:10.978688] [DEBUG] [LOCAL ] MYN: Loaded 7468 existing records from primary storage. [2025-09-16T19:33:10.978873] [INFO ] [MAIN ] MYN: Loaded 7468 existing records. [2025-09-16T19:33:11.159709] [WARN ] [MAIN ] MYN: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:33:11.159958] [INFO ] [MAIN ] MYN: Wiping existing primary data before full refresh. [2025-09-16T19:33:11.160005] [INFO ] [LOCAL ] MYN: Starting deletion of primary data. [2025-09-16T19:33:11.160282] [DEBUG] [LOCAL ] MYN: Deleted primary data file. [2025-09-16T19:33:11.160329] [INFO ] [MAIN ] MYN: Starting full primary refresh. [2025-09-16T19:33:11.519702] [INFO ] [REMOTE ] MYN: Fetching primary data covering 150 trading days from 2025-02-11 to 2025-09-16 (requires 8 months of data pull). [2025-09-16T19:33:11.537077] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-09-16T19:33:11.574887] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-05. [2025-09-16T19:33:11.948373] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-09-16T19:33:12.374871] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-02. [2025-09-16T19:33:12.682457] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-09-16T19:33:12.919222] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-09-16T19:33:13.174884] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-03. [2025-09-16T19:33:13.974928] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-05. [2025-09-16T19:33:13.983898] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-09-16T19:33:14.774915] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-06. [2025-09-16T19:33:15.574944] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-04. [2025-09-16T19:33:15.924113] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-09-16T19:33:16.356069] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-09-16T19:33:16.374864] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-05. [2025-09-16T19:33:17.099927] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-09-16T19:33:17.174929] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-07. [2025-09-16T19:33:17.974901] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-06. [2025-09-16T19:33:18.759248] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-09-16T19:33:18.774865] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-07. [2025-09-16T19:33:19.527853] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-09-16T19:33:19.574870] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-08. [2025-09-16T19:33:20.362519] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-09-16T19:33:20.374848] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-09. [2025-09-16T19:33:21.170106] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-09-16T19:33:21.172021] [INFO ] [REMOTE ] MYN: Full fetch complete (11039 records). [2025-09-16T19:33:21.172089] [DEBUG] [VALIDATOR ] MYN: Validating primary data. [2025-09-16T19:33:21.540620] [DEBUG] [VALIDATOR ] MYN: All expected trading days are present. [2025-09-16T19:33:21.540724] [DEBUG] [VALIDATOR ] MYN: Primary data validation passed. [2025-09-16T19:33:21.541149] [INFO ] [MAIN ] MYN: Fetched primary data is valid. [2025-09-16T19:33:21.541214] [DEBUG] [LOCAL ] MYN: Saving primary data. [2025-09-16T19:33:21.541789] [DEBUG] [LOCAL ] MYN: Trimmed 11039 records to 7569 (keeping 150 days of data) [2025-09-16T19:33:21.596240] [DEBUG] [LOCAL ] MYN: Primary data saved (7569 records). [2025-09-16T19:33:21.596399] [INFO ] [MAIN ] MYN: Full history saved. [2025-09-16T19:33:21.596494] [INFO ] [MAIN ] Update Progress: 1444/2369 symbols completed. MYN took 10.8s. Estimated remaining: 1h 38m 12s. Rate: 9.4 symbols/min [2025-09-16T19:33:21.789049] [INFO ] [MAIN ] NABL: Starting incremental primary update from 2025-09-08 to 2025-09-16. [2025-09-16T19:33:21.828290] [DEBUG] [LOCAL ] NABL: Loaded 29090 existing records from primary storage. [2025-09-16T19:33:21.828488] [INFO ] [MAIN ] NABL: Loaded 29090 existing records. [2025-09-16T19:33:22.130782] [WARN ] [MAIN ] NABL: Insufficient historical data. Have data from 2025-04-21, but need from 2025-02-11 for 150 day retention. Performing full refresh. [2025-09-16T19:33:22.132059] [INFO ] [MAIN ] NABL: Wiping existing primary data before full refresh. [2025-09-16T19:33:22.132131] [INF