XML Vs CSV Performance for a large data file
Problem background
==================
My application running on SUN Solaris downloads large amount of statisitics data (up to 1 million counters from each device) from hundreds of devices over a T1 line every 15 minutes. This data is written to different tables of a database (Oracle9i). The data can then be viewed from a reporting client that has extensive filtering and sorting capability.
Question
========
I am thinking of using either XML or Comma Separated (CSV) format for representing the statistics data. I am assuming that the XML file will be at least 6-7 times the size of the CSV file due to the tags in XML.
How would you compare the performance of XML and CSV while:
- parsing the data file
- storing the data in the DB (using indexes)