I am working with aggregated EDI X12 837, 834, 274, and 271 files. I have yet to see a 1GB file however have been told I can expect to see them in the future. The largest file I've seen so far was about 72 MB and it contained around 182,000 transactions/messages.
I've been informed the maximum number of transactions per file is 9,999,999.
There is typically one interchange in each file (ISA - IEA) with one group (GS - GE) containing (10s of 1000s) of transactions/messages (ST - SE) in the envelope.
So something like :
I have been using the library like so:
var x12str = Console.In.ReadToEnd();
Byte strAsBytes = new System.Text.UTF8Encoding().GetBytes(x12str);
var interchange = INTERCHANGE.LoadFrom(new System.IO.MemoryStream(strAsBytes));
foreach (var group in interchange.GROUPS)
foreach (var message in group.MESSAGES)
var typedMessage = message.DeserializeItem<M_837>();
However it loads the entire contents of the file into memory (and in the case of the 72 megabyte file it uses a ton of memory making multiple copies of the data) when really I only need to process (make a few flat delimited lines) from one transaction at a
time. So, I have written a preprocessor that splits the file into more manageable chunks of ~4MB each or 10000 transactions per file (whichever comes first).