The separate copy of allocator state in parser was meant to increase parsing
performance by reducing aliasing/indirection, but benchmarks against the
current source don't indicate that this is worthwhile.
Removing this simplifies the code slightly and makes it possible to move
compact hash table to the allocator.
This adds about 40 cycles for parsing <?xml version='1.0'?> declaration and
about 70 cycles for parsing <?xml version='1.0' encoding='utf-8'?>, as
measured on a Core i7, which should be negligible for all documents.
Fixes#16.
Previously the page size was defining the data size, and due to additional
headers (+ recently removed allocation padding) the actual allocation was a bit
bigger.
The problem is that some allocators round 2^N+k allocations to 2^N+M, which can
result in noticeable waste of space. Specifically, on 64-bit OSX allocating the
previous page size (32k+40) resulted in 32k+512 allocation, thereby wasting 472
bytes, or 1.4%.
Now we have the allocation size specified exactly and just recompute the available
data size, which can in small space savings depending on the allocator.
When using format_raw the space in the empty tag (<node />) is the only
character that does not have to be there; so format_raw almost results in
a minimal XML but not quite.
It's pretty unlikely that this is crucial for any users - the formatting
change should be benign, and it's better to improve format_raw than to add
yet another flag.
Fixes#87.
Unify the implementations by automatically deducing the unsigned type from its
signed counterpart. That allows us to use a templated function instead of
duplicating code.
This utilizes the fact that pages are of limited size so we can store offset
from the object to the page in a few bits - we currently use 24 although that's
excessive given that pages are limited to ~512k.
This has several benefits:
- Pages do not have to be 64b aligned any more - this simplifies allocation flow
and frees up 40-50 bytes from xml_document::_memory.
- Header now has 8 bits available for metadata for both compact and default mode
which makes it possible to store type as-is (allowing easy type extension and
removing one add/sub operation from type checks).
- One extra bit is easily available for future metadata extension (in addition
to the bit for type encoding that could be reclaimed if necessary).
- Allocators that return 4b-aligned memory on 64-bit platforms work fine if
misaligned reads are supported.
The downside is that there is one or two extra instructions on the allocation
path. This does not seem to hurt parsing performance.
This change fixes an important ordering issue - if element node has a PCDATA
child *after* other elements, it's impossible to tell which order the children
were in.
Since the goal of PCDATA embedding is to save memory when it's the only child,
only apply the optimization to the first child. This seems to fix all
roundtripping issues so the only caveat is that the DOM structure is different.
This is a bit awkward since preserving correct indentation structure requires
a bit of extra work, and the closing tag has to be written by _start function
to correctly process the rest of the tree.
When this flag is true, PCDATA value is saved to the parent element instead of
allocating a new node.
This prevents some documents from round-tripping since it loses information,
but can provide a significant memory reduction and parsing speedup for some
documents.
Apparently some MinGW distributions have a compiler that's recent enough to
support C++11 but limits.h header that incorrectly omits LLONG limits in
strict ANSI mode, since it guards the definitions with:
#if !defined(__STRICT_ANSI__) && defined(__GNUC__)
We can just define these symbols ourselves in this specific case.
Fixes#66.
Since they don't contribute to the resulting value just skip them before
parsing. This matches the behavior of strtol/strtoll and results in more
intuitive behavior.
Node type enum is not used as an array index anywhere else; the code is not
very readable and the value of this "optimization" is questionable.
The conditions are arranged so that in all normal cases the first comparison
returns true anyway.
The minneg argument is supposed to be the absolute value of the minimum negative
representable number. In case of two-complement arithmetic, it's the same as the
value itself but it's better to be explicit and negate the argument.
Instead of functions with different names (e.g. decode_utf8_block), split
utf_decoder class into multiple classes with ::process static function.
This makes it easier to share code for decoding different encodings.
Instead of calling xml_document public functions just call implementation of
load_buffer_inplace_own. This makes it so we only call reset() once during
load_file/load.
This makes conversion significantly faster and removes more CRT dependencies;
in particular, to support long long pugixml only requires the type itself (and
the division operator...).
New implementation is up to 3x faster on short decimal numbers.
Note that unlike the old implementation, new implementation correctly handles
overflow and underflow and clamps the value to the representable range. This
means that there are some behavior changes - e.g. previously as_uint on "-1"
would return INT_MAX instead of 0.
In addition to CRT issues, for platforms with 64-bit long old implementation
incorrectly truncated from long to int or unsigned int, so even if CRT clamped
the values the result would have been incorrect.
This reduces the amount of non-standard C++ functionality pugixml may be using
by avoiding sprintf with %lld; additionally this implementation is significantly
faster (4-5x) than sprintf, mostly due to avoiding format string parsing and
stream setup that commonly happens in CRT implementations.
This comes at the expense of requiring long long division/remainder operations
if PUGIXML_USE_LONG_LONG is defined which will surely bite me one day.
Change the expression to reference the array element indirectly. The memory
block can be bigger than the structure so it's invalid to use static data[]
size for bounds checking.
To be more precise, the memory block is now aligned to be able to reliably
allocate objects with both double and pointer fields. If there is a platform
with a 4-byte double and a 4-byte pointer, the memory block alignment there will
stay the same after this change.
Fixes#48.
Apparently Clang 3.7 implements C++ DR 1748 that makes placement new with null
pointer undefined behavior. Which renders all C++ programs that rely on this
invalid. Which includes pugixml.
This is not very likely to happen in the wild because the allocations that are
subject to this in pugixml are relatively small, but tests break because of
this.
Fix the issue by adding null pointer checks (that are completely redundant in
all current compilers except Clang 3.7 but it's not like there is another
option).
Work around a name lookup bug by pulling auto_deleter name in the local
scope. We could also move auto_deleter to pugi:: namespace, but that
pollutes it unnecessarily for other compilers.
Extra argument 'hint' is used to start the attribute lookup; if the attribute
is not found the lookup is restarted from the beginning of the attriubte list.
This allows to optimize attribute lookups if you need to get many attributes
from the node and can make assumptions about the likely ordering. The code is
correct regardless of the order, but it is faster than using vanilla lookups
if the order matches the calling order.
Fixes#30.
Now compact_string matches compact_pointer_parent.
Turns out PUGI__UNLIKELY is good at reordering conditions but usually does not
really affect performance. Since MSVC should treat "if" branches as taken and
does not support branch probabilities, don't use them if we don't need to.
Instead of checking if the object being removed allocated a marker, mark the
marker block as deleted immediately upon allocation. This simplifies the logic
and prevents extra markers from being inserted if we allocate/deallocate the
same node indefinitely.
Also change marker pointer type to uint32_t*.
When we deallocate nodes/attributes that allocated the marker we have to
adjust the size accordingly, and dismiss the marker in case it gets
overwritten with something else...
This temporarily increases the node size to 16 bytes - we'll bring it back.
It allows us to remove the horrible node_pi hack and to reduce the amount of
changes against master. This comes at the price of not decreasing basline
xml_node_struct size.
The compact xml_node_struct is also increased by this change but a followup
change will reduce *both* xml_attribute_struct and xml_node_struct (to 8/12
bytes).
We used this in two cases - to get the page pointer and to test flags.
We now use PUGI__GETPAGE for getting the page pointer and operator& to test
flags - this makes getting node type significantly faster since it does not
require page pointer reconstruction.
Clarify the offset applied when encoding the pointer difference.
Make decoding diff slightly more clear - no effect on performance.
Adjust branch weighting in compact_string encoding - 0.5% faster.
Use uint16_t in compact_pointer_parent - 2% faster.
Make sure compact_hash_table::rehash() is not inlined - that way reserve() is
inlined so the fast path has no extra function calls.
Also use subtraction instead of multiplication when checking capacity.
xpath_query, xpath_node_set and xpath_variable_set are now moveable.
This is a nice performance optimization for variable/node sets, and enables
storing xpath_query in containers without using pointers (it's only possible
now since the query is not copyable).
xpath_variable_set is essentially an associative container; it's about time it
became copyable.
Implementation is slightly tricky due to out of memory handling. Both copy ctor
and assignment operator have strong exception guarantee (even if exceptions are
disabled! which translates to "roll back on allocation errors").
If xml_writer::write throws an exception while being called from flush(), the
exception is thrown from destructor. Clang in C++11 mode calls std::terminate
in this case.
Fix code style and revert redundant parameters/whitespace changes.
Also remove format_each_attribute_on_new_line - we're only introducing one
extra formatting flag. The flag implies format_indent but does not include its
bitmask.
Also add a few more tests.
Fixes#14.
Ensure that all the necessary cleanup is performed in case the allocation fails
with an exception - files are closed, buffers are reclaimed, etc.
Any test that triggers a simulated out-of-memory condition is ran once again
with a throwing allocation function. Unobserved std::bad_alloc count as test
failures and require CHECK_ALLOC_FAIL macro.
Fixes#17.
Previously attributes that were copied with their node used string sharing,
but standalone attributes that were copied using xml_node::*_copy(xml_attribute)
were not.
as_utf8_end was used with std::string, where writing an extra zero-terminating
character should *probably* always work (at least if size is positive) but is
not ideal.
The only place that needed to zero-terminate was convert_path_heap.
When parsing XPath variables, we need to perform a heap allocation; if it
fails, an xpath_exception instead of bad_alloc used to be thrown.
Now we throw the exception of a correct type so that xpath_exception means
'parsing error'.
Previously we omitted extra whitespace for single PCDATA/CDATA children, but in
mixed content there was extra indentation before/after text nodes.
One of the problems with that is that the text that you saved is not exactly
the same as the parsing result using default flags (parse_trim_pcdata helps).
Another problem is that parse-format cycles do not have a fixed point for mixed
content - the result expands indefinitely. Some XML libraries, like Python
minidom, have the same issue, but this is definitely a problem.
Pretty-printing mixed content is hard. It seems that the only other sensible
choice is to switch mixed content nodes to raw formatting. In a way the code in
this change is a weaker version of that - it removes indentation around text
nodes but still keeps it around element siblings/children.
Thus we can switch to mixed-raw formatting at some point later, which will be
a superset of the current behavior.
To do this we have to either switch at the first text node (.NET XmlDocument
does that), or scan the children of each element for a possible text node and
switch before we output the first child.
The former behavior seems non-intuitive (and a bit broken); unfortunately, the
latter behavior can cost up to 20% of the output time for trees *without* mixed
content.
Fixes#13.
Since all string allocations are pointer-aligned to avoid aligning more
frequent node allocations, we can rely on that in string encoding.
Encoding page offset and block size in sizeof(void*) units increases the
maximum memory page size from 64k to 256k on 32-bit and 512k on 64-bit
platforms.
Fixes#35.
The implementations generated a string with an internal null terminator; this
went unnoticed since unit test string verification did not perform string
equality check properly (it compared XPath string result as a C-string, thus
stopping at the first null terminator).
Fixes#36.
Make float/double round-trip
This change also adds xml_text::set and xml_attribute::set_value overloads for float so that float is only printed using just enough digits to represent float, instead of enough digits to represent double.
It's sufficient to define PUGIXML_HEADER_ONLY anywhere now, source is included
automatically.
This is a second attempt; this time it includes a workaround for QMake bug
that caused it to generate incorrect Makefile.
Unfortunately, standard headers on MinGW32 insist on undefining off64_t
and _wfopen extensions if __STRICT_ANSI__ is true (e.g. C++11 mode). This
leads to compilation errors since b7a1fec started to use _wfopen in strict
mode. That change erroneously checked GCC version - however, the version
itself is irrelevant; the actual criteria is whether mingw64 runtime is
used.
off64_t is not useful on MinGW32 since we only need it to open large files
on 64-bit platforms; unfortunately, the lack of _wfopen means we won't be
able to support wide-char paths on Windows for MinGW32.
Fixes#24.
Since MinGW 4.5 does not define these functions if __STRICT_ANSI__ is defined
(in case of _wfopen it defines it inconsistently between stdio.h and wchar.h)
use the baseline functions for MinGW 4.5 and earlier.
Fixes#23.
Since copying no longer relies on child insertion we have to also reserve
space in the hash table for the allocator so that pointer manipulations are
guaranteed to succeed.
node_copy_string relied on the fact that target node had an empty name and
value. Normally this is a safe assumption (and a good one to make since it
makes copying faster), however it was not checked and there was one case when
it did not hold.
Since we're reusing the logic for inserting nodes, newly inserted declaration
nodes had the name set automatically to xml, which in our case violates the
assumption and is counter-productive since we'll override the name right after
setting it.
For now the best solution is to do the same insertion manually - that results
in some code duplication that we can refactor later (same logic is partially
shared by _move variants anyway so on a level duplicating is not that bad).
Remove redundant this-> from type() call (argument used to be called type,
but it's now type_).
Use _root member directly when possible instead of calling internal_object.
This should completely eliminate the confusion between load and load_file.
Of course, for compatibility reasons we have to preserve the old variant -
it will be deprecated in a future version and subsequently removed.
