Imorting a 30 MB CSV file in ax 2009 using X++ not working...?

Finance and Operations/AX Developer Forum
1

Hi All,

I am trying to update CustTable records by importing a CSV file which is 30 MB in size and almost have 500,000 records in it thru X++ code, and it is running forever. If I did the same with 100,000 records CSV file it is taking just 15 mins.

Let me know your thoughts…

Thanks In Advance…!

2

It’s difficult to comment your code if it’s not shown here. It probably could be rewritten for better scalability.

Or it may be easier just to split the file to several smaller ones and import them separately.

3

Hi Martin,

can you let me know if is there any way to split the 30 MB CSV file automatically by X++ instead of splitting the CSV file mannually, Since the whole process should be done by a Batch Job.

Thanks in advance…!

4

If it’s not an one-off task and you have to write code for it, you can process each line individually, read blocks of lines to create several batch tasks or anything you want, so you probably get better options than physically splitting the file.

If you decide to split the file, it’s straightforward - read lines from the source file, write them to a target file and if the target file is big enough, use another target file. You can use TextIO class, System.IO.StreamReader/StreamWriter or anything else capable to read and write text files. (I guess that writing a simple .NET library using IO streams would be the best approach from performance perspective.)