Using cursors for long-running processes
389473Aug 27 2009 — edited Aug 27 2009Greetings all,
I know I could probably just google this and/or run some tests, but if it wouldn't be an imposition, I'd just like to hear the answers with a "personal touch".
I'm writing a PL/SQL process that performs tasks against rows of a table that may end up having about a quarter of a million rows. Each row will take about one second to process, so we're talking about something that's going to run for about 60 hours straight. There is a very high probability that the rows matching the criteria for the against which the process should run will change over those 60 hours. In as few words as possible, the process runs against all rows in the table with an "active" status, and I've also given the users a "priority" flag that is intended to let them identify which objects need to run ASAP vs. those that can wait. New active rows could be added, rows which were active at the time of execution could be deleted by the time processing gets to them etc.
Is a cursor the wrong solution for this type of architecture? I was thinking that I should do the following:
1. Add a "processed" flag to the table I'm traversing
2. Clear it a the beginning of a run
3. Use a loop where each time through the loop I query the first row matching the criteria whose processed flag is NOT set
4. Process it
5. Set its processed flag
6. Go back to 3 and exit the loop when no more objects match the criteria and do not have the flag set
By the way, in future phases of the project we will probably add servers to the architecture and/or rewrite the code to take advantage of multithreading and table partitioning to guarantee that it runs overnight. We just don't have time or budget right now. The users understand that their data may not be ready available and need to look for the processed flag to know whether or not it's been updated.
So knowing that for now we have something that's going to run for 60 hours, is my approach better than just using a cursor? Is there an even better option I'm not thinking of?
Thanks
DTXCF