DMF import giving error as duplicates in staging

Hi, I ma trying to import text file through data management framework getting below error

Failed to insert record into staging table. The keys of the record are XXXXX, YYYYY, DEFINITIONGROUP, EXECUTIONID, PARTITION, ZZZZ. Duplicate records must be removed from the file prior to import.-The duplicate key value is (5637144576, 4013_CompartmentInfo_1 (1)_5102022082128, 4013_CompartmentInfo_1 (1)_5102022082128-2022-05-10T08:21:29-8519CD4C054D4BFB99920D0DCE12B, , , Jan 1 1900 12:00AM)

But when I check staging table duplicate records not available, the record that I am importing absolutely unique one.
The entity is custom entity. please suggest if any workaround. I have tried ith only one record but still same error …and the recid that is showing in the error is not available in staging table… Please suggest.

I can see that partition column added to key which is not available in index of staging table

What fields do you have in the unique index (or indexes) on the staging table?

Hello Martin

It has DefinitionGroup, ExecutionId and other three columns from my target table as staging Index.

But if we see the error message it has partition as well , I suspect that is causing issue, could you pls suggest if any workaround

It seems to me that the problem is with empty values. I assume that the first three values of
5637144576, 4013_CompartmentInfo_1 (1)_5102022082128, 4013_CompartmentInfo_1 (1)_5102022082128-2022-05-10T08:21:29-8519CD4C054D4BFB99920D0DCE12B, , , Jan 1 1900 12:00AM
are Partition, DefinitionGroup and ExcecutionId - and all the remaining are empty.
Don’t you have some empty lines in your file?

Hi Martin I am trying with VerticalBarseperator file and it has only one record ,example as shown below

Column1|Column2|Column3|Column4|Column5|Column6|Column6
000015|2|xyz|08/01/2022 00:00:00|07/10/2022 00:00:00|m3|1.55

As I said, the error message complains about empty values, not about your values above.
Check out for empty lines. Also, don’t you have any code in postGetStagingData()?

Hello Martin, Thank you very much for your help
issue got resolved after cleaning the text file which means it is issue with empty values in file

Thank you very much once again