I have DataStage 7.5 in Windows 2003 server .. the problem is:
I have a jobs that writes data to hashed file, hashed file format is the defaults of Type 30 (Dynamic). One job deletes then create the file, then some other jobs append records to the same file. One day I found that the data folder for this file instead of containing 2 files (data and metadata), it contains a single file for each record with file name equal to record key. so if the file should hold 5 million records , its data folder contains 5 million files !!!
I tried to delete all files manually (this required more than 4 hours) then recompile the job and run it again for 100 records only, again it generated 100 files !!! It only work correct again if file path changes which is not always possible.
I need a solution to correct this up normal job behavior without changing the file path.
Also I need a fast way to delete 5 million files in a single folder.
Pinned topic Hash file generates single file for each record in data fold
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
Updated on 2005-12-27T14:58:07Z at 2005-12-27T14:58:07Z by SystemAdmin
Re: Hash file generates single file for each record in data2005-06-05T05:03:28ZThis is the accepted answer. This is the accepted answer.You don't have a hashed file. A dynamic hashed file is a folder containing exactly three files, called DATA.30, OVER.30 and .Type30.
If someone has deleted any of these, or added any other file to the folder, then it can not be opened as a hashed file.
What you need to do is to delete every file except the three named above, or delete the folder itself and its file dictionary (a file called D_hashedfile) and, if the hashed file was created in Account (rather than in Directory), its VOC entry, then re-create the hashed file.
Re: Hash file generates single file for each record in data2005-12-27T14:07:10ZThis is the accepted answer. This is the accepted answer.just to throw a little info in, I had the same problem, and I think it came from having one process reading the hash file while another was trying to re-create it. i'm guessing a read-lock kept the delete from working and then the folder/files became out of synch. deleting the folder and D_ file would resolve it on the next run....
Re: Hash file generates single file for each record in data2005-12-27T14:58:07ZThis is the accepted answer. This is the accepted answer.This can also happen if you have the "Create File" and "Clear File Before Writing" options selected at the same time. When you run the job, it will delete part of the .30 files, but leave some intact because they are locked.