I have stream of alien csv files which I want to be staged as-is to database for further processing. Basically known things are:
- filename
- maximum amount of columns any file can have
- the used separator
So how to have all load related metadata so that there is no need to check logs or investigate alien csv-files after they have been imported, but just sql the data?
Example of additional columns per import could be like:
- sqlcl - load parameters used, e.g. : column_names off, delimiter ;, enclosure left “, enclosure right ”, encoding UTF8 .. etc. whatever was used to make the load to happen.
- in the target table populate during import extra columns to help processing the data later on:
- filename
- csv_row_number
- imported_sysdate
- .. any other suggestions for clever fields helping later on ?
rgrds Paavo