Problem: Stored Procedures in both staging and production dbs were causing data columns for both act and fcst wx stations to be switched(cloudp% to wdir, wdir to wind speed, wind speed to cloud%). The source file looked correct.
Diagnosis: When debugging to a local db (with no history), the columns were not switched. Must be in the DB.
Solution:
- Delete SPs:
- go to the db in question in SSMS
- go to the Programmability folder
- go to Store Procedures
- locate the SPs that correspond to the affected tables and delete them
- Run the ingest task again and verify the columns and newly ingested data look right
- Clean the remaining affected data:
- determine when the error started
- open DataUtility,
- get the affected tables within the affected time span
- fix the data
- re-ingest the new table with DataUtility