Unicode Issue
546700Nov 17 2006 — edited Nov 18 2006I am facing a problem while fetching the data from Oracle 10g running under Solaris..
In my current application,Peoplesoft generates employee details as a CSV file. The same file is used as a data source for an external table.
The data generated by people soft contains Euporian languages and the file is generated in Unicode format. The external table is unable to read the data in a proper format. When I give select * from I am able to see some junk values.
I am using the following script for external table:
CREATE TABLE EXT_EMPLOYEE_TEST (
EMP_ID VARCHAR2 (15),
FIRST_NAME VARCHAR2 (50),
STR_EOF VARCHAR2 (3) )
ORGANIZATION EXTERNAL
(TYPE ORACLE_LOADER
DEFAULT DIRECTORY INPROCESS_DIR
ACCESS PARAMETERS
(RECORDS DELIMITED BY NEWLINE CHARACTERSET ALT32UTF8
badfile 'unicode_test.bad'
logfile 'unicode_log.log'
fields terminated by '|'
lrtrim
missing field values are null
(
EMP_ID,
FIRST_NAME,
STR_EOF))
LOCATION (INPROCESS_DIR:'EMPLOYEE_TEST_FILE.CSV')
)
REJECT LIMIT UNLIMITED
NOPARALLEL;
The file 'EMPLOYEE_TEST_FILE.CSV' will have the following text.
Copy the text in a notepad and save as EMPLOYEE_TEST_FILE.CSV with Unicode format. (I am giving the text as there is no option to attach the file)
11534 | СТОИЛОВ ПОЛЯ ГАНЧЕВА | EOF
30941 | 行 色家电新 白色家电新 | EOF
11517 | BLÁHOVÁ DW LDE | EOF
Note: The employee names are in different languages as Bulgarian, Polish and Chinese.
My Ground Work
I have tried in many ways changing the character set, etc even then I am not able to view the proper data either at back end or through front end.
1. The problem can also be with file transfer.
I am trying to transfer the CSV file from Windows box (My local Machine) to UNIX box (Oracle server, where the InProcess_Dir is located) through UNIX command (mput). This command transfers the file in ANSII mode; hence the data you see in the Database is nothing but the ANSII values.
I have tried using Binary mode; even then I have the same problem.
2. As I thought the problem can be through file transfer, I tried to work on the same through Windows box. I created a dummy database in Oracle home running through Windows box. I copied the same file in InProcess_dir and tried to create the external table. I see the same result (i.e. if I run the script with the same file saved as Utf-8, or ASII I see junk values and if I save the file as Unicode and run the script not records are displayed.)
Kindly give a solution for this problem. I will be happy if I can see the actual data using this External Table Concept.
Please help me in this regard, I shall be very thankful to you.