Processing huge amount of data using PLSQL
HI
Before I post the question here are a few essentials I would like to point out:
1) I have to use PL/SQL as the tool though many in here will suggest to use a different tool
2) The data requirement is such that I have to stick to it. Cant help it.
So here goes the problem.
I have to create a text file and the requirement goes as below:
1) It will have a 3 X 50 X 12 X 4 X 2 X 146 X 1 large table. The resultant will be the number of rows in the table which will have 7 columns. the figures that you see will be the number of unique values in the columns. For eg col 1 will have 3 unique values, column 2 will have 50 and so on. This table will be static and wont change.
2) I have to cross join this table with another table which has just one column and has the year month for the past 38 months. So the resultant table will be a 3 X 50 X 12 X 4 X 2 X 146 X 1 X 38 rows.
3) Once I have this data I have to pass the values in the columns as input parameter to a function to produce another column. The functions accesses data across a db link. So the function will be called 3 X 50 X 12 X 4 X 2 X 146 X 1 X 38 number of times.
As you would have guessed the problem is performance. Its close to like 86 million records. I thought of some options like CTAS (Create table as select) but thats pretty much it. I just need pointers to look in which direction rather than a solution itself (which is pretty crass if am asking for it).
Thanks in advance
Cheers
Sid