1

Topic: MSXML Creation of the big files

There is a file of a similar format

<root>
<table1>
<table1_row>
<field11> Value11 </field11>
<field12> Value12 </field12>
............
<field1n> Value1n </field1n>
</table1_row>
<table1_row>
...........
</table1_row>
............
</table1>
<table2>
.......
</table2>
..........
<tableN>
.......
</tableN>
</root>

Procedure of adding of a new line of the table

procedure TXMLDataSet. Post;
var
LNewNode: IXMLDOMNode;
Li: Integer;
LError: IXMLDOMParseError;
begin
LNewNode: = FXMLDoc.createNode (NODE_ELEMENT, FRowElementName, Namespace);
for Li: = 0 to Fields. Count - 1 do
FFields. Post (LNewNode);
FParentNode.appendChild (LNewNode);
LError: = FXMLDoc.validateNode (LNewNode);
Win32Check (LError.errorCode);
end;

Problem: the operating time of this procedure strongly depends on that how many already nodes is interposed in FParentNode. At 15000 lines the operating time makes 15 msec on one start (begins with 0). Since there is a parallel insertion to three tables the total operating time appears rather considerable. The first 30 % of all operation are fulfilled somewhere minutes for 30, and remaining 70 % - hour for 4.
Actually a question - that such to twist? A cycle of an insertion of fields

for Li: = 0 to Fields. Count - 1 do
FFields. Post (LNewNode);

it is always fulfilled for constant time (~0), and runtime appendChild and validateNode grows linearly, depending on a record count.
If the code slightly to modify

LNewNode: = FXMLDoc.createNode (NODE_ELEMENT, FRowElementName, Namespace);
FParentNode.appendChild (LNewNode);
for Li: = 0 to Fields. Count - 1 do
FFields. Post (LNewNode);
LError: = FXMLDoc.validateNode (LNewNode);

that almost imperceptible growth of time  (is received from 0 that 2 msec on 17000 records). Hardly  on appendChild (from 0 to 5) and wild growth on adding of fields (to 15 msec on 17000 records). As a result of such code works on 5-7 msec longer, than initial.
It tried to be played with fragments (IXMLDOMDocumentFragment) and to interpose into 1 fragment on 1000 records but then fell off . If record to interpose in the beginning in , there , and then to transfer to a fragment it turned out essential a procorf on speed.
In general, any ideas (are welcomed up to parcer change). But the new parcer should be able  on XSD to the circuit. Including, to support  one node
In application runtime of a basic variant
Yours faithfully, Vasilisk

2

Re: MSXML Creation of the big files

And here is how the modified variant looks

3

Re: MSXML Creation of the big files

Usage of fragments on 1000 nodes.  it is disconnected, fragments are interposed into the document right after creations. On the schedule append lines in a fragment. Who explains me the nature of these steps? Just on 1000 records

4

Re: MSXML Creation of the big files

I would try for each of tables to form the XML document, after the termination of filling of all tables to unite them in one document. Thus, if the insertion goes parallely to three tables simultaneously - would try to space apart this operation on three various flows with waiting of end all three in a principal flow.

5

Re: MSXML Creation of the big files

Very Pavlovny wrote:

after the termination of filling of all tables to unite them in one document

There was such thought. But whether occupies join process the exactly same time, as an initial insertion?

Very Pavlovny wrote:

would try to space apart this operation on three various flows

There not that it is parallel. There master master-detajl-detajl. Though yes - it is possible to try

6

Re: MSXML Creation of the big files

[quote = _ Vasilisk _] Who explains me the nature of these steps?

Changed sampling of the timer with 1 to 0.1 and steps disappeared stable growth