Rsync on archive logs resulting in sporadic corruption on receiver
902604Nov 28 2011 — edited Nov 29 2011I have 2 databases running 10g, one is the main database and the other a standby. I am using rsync to copy archive logs from the active to the standby and apply the logs to the standby. Once in a while (a few times recently but otherwise maybe once every 6 months) the standby database cannot apply the log with error "ORA-00353: log corruption near block".
I have compared the corrupt archive log with the main database, and I can clearly see a small difference. In the standby (corrupt) version there are some blocks of NUL, and on the original version there is clearly data there. The NUL blocks are in different places in the file, not the end, so it is not a case of oracle appending data after the copy.
Md5sum confirms that the files are different, and so does rsync but only after I force it to check using the -c option. After syncing with rsync -c, the log can be applied successfully. I do not want to make -c a default option as it causes a lot more IO.
Has anyone seen this before? Any idea what's going on? It seems like part of the log is changed after it's written. Does oracle do more than simply append to the archive log files?
Oracle 10g
RHEL Linux 5 x64
rsync 3.0.6