To be more precise, the memory block is now aligned to be able to reliably
allocate objects with both double and pointer fields. If there is a platform
with a 4-byte double and a 4-byte pointer, the memory block alignment there will
stay the same after this change.
Fixes#48.
Apparently Clang 3.7 implements C++ DR 1748 that makes placement new with null
pointer undefined behavior. Which renders all C++ programs that rely on this
invalid. Which includes pugixml.
This is not very likely to happen in the wild because the allocations that are
subject to this in pugixml are relatively small, but tests break because of
this.
Fix the issue by adding null pointer checks (that are completely redundant in
all current compilers except Clang 3.7 but it's not like there is another
option).
Work around a name lookup bug by pulling auto_deleter name in the local
scope. We could also move auto_deleter to pugi:: namespace, but that
pollutes it unnecessarily for other compilers.
Extra argument 'hint' is used to start the attribute lookup; if the attribute
is not found the lookup is restarted from the beginning of the attriubte list.
This allows to optimize attribute lookups if you need to get many attributes
from the node and can make assumptions about the likely ordering. The code is
correct regardless of the order, but it is faster than using vanilla lookups
if the order matches the calling order.
Fixes#30.
Now compact_string matches compact_pointer_parent.
Turns out PUGI__UNLIKELY is good at reordering conditions but usually does not
really affect performance. Since MSVC should treat "if" branches as taken and
does not support branch probabilities, don't use them if we don't need to.
Instead of checking if the object being removed allocated a marker, mark the
marker block as deleted immediately upon allocation. This simplifies the logic
and prevents extra markers from being inserted if we allocate/deallocate the
same node indefinitely.
Also change marker pointer type to uint32_t*.
When we deallocate nodes/attributes that allocated the marker we have to
adjust the size accordingly, and dismiss the marker in case it gets
overwritten with something else...
This temporarily increases the node size to 16 bytes - we'll bring it back.
It allows us to remove the horrible node_pi hack and to reduce the amount of
changes against master. This comes at the price of not decreasing basline
xml_node_struct size.
The compact xml_node_struct is also increased by this change but a followup
change will reduce *both* xml_attribute_struct and xml_node_struct (to 8/12
bytes).
We used this in two cases - to get the page pointer and to test flags.
We now use PUGI__GETPAGE for getting the page pointer and operator& to test
flags - this makes getting node type significantly faster since it does not
require page pointer reconstruction.
Clarify the offset applied when encoding the pointer difference.
Make decoding diff slightly more clear - no effect on performance.
Adjust branch weighting in compact_string encoding - 0.5% faster.
Use uint16_t in compact_pointer_parent - 2% faster.
Make sure compact_hash_table::rehash() is not inlined - that way reserve() is
inlined so the fast path has no extra function calls.
Also use subtraction instead of multiplication when checking capacity.
xpath_query, xpath_node_set and xpath_variable_set are now moveable.
This is a nice performance optimization for variable/node sets, and enables
storing xpath_query in containers without using pointers (it's only possible
now since the query is not copyable).
xpath_variable_set is essentially an associative container; it's about time it
became copyable.
Implementation is slightly tricky due to out of memory handling. Both copy ctor
and assignment operator have strong exception guarantee (even if exceptions are
disabled! which translates to "roll back on allocation errors").
If xml_writer::write throws an exception while being called from flush(), the
exception is thrown from destructor. Clang in C++11 mode calls std::terminate
in this case.
Fix code style and revert redundant parameters/whitespace changes.
Also remove format_each_attribute_on_new_line - we're only introducing one
extra formatting flag. The flag implies format_indent but does not include its
bitmask.
Also add a few more tests.
Fixes#14.
Ensure that all the necessary cleanup is performed in case the allocation fails
with an exception - files are closed, buffers are reclaimed, etc.
Any test that triggers a simulated out-of-memory condition is ran once again
with a throwing allocation function. Unobserved std::bad_alloc count as test
failures and require CHECK_ALLOC_FAIL macro.
Fixes#17.
Previously attributes that were copied with their node used string sharing,
but standalone attributes that were copied using xml_node::*_copy(xml_attribute)
were not.
as_utf8_end was used with std::string, where writing an extra zero-terminating
character should *probably* always work (at least if size is positive) but is
not ideal.
The only place that needed to zero-terminate was convert_path_heap.
When parsing XPath variables, we need to perform a heap allocation; if it
fails, an xpath_exception instead of bad_alloc used to be thrown.
Now we throw the exception of a correct type so that xpath_exception means
'parsing error'.
Previously we omitted extra whitespace for single PCDATA/CDATA children, but in
mixed content there was extra indentation before/after text nodes.
One of the problems with that is that the text that you saved is not exactly
the same as the parsing result using default flags (parse_trim_pcdata helps).
Another problem is that parse-format cycles do not have a fixed point for mixed
content - the result expands indefinitely. Some XML libraries, like Python
minidom, have the same issue, but this is definitely a problem.
Pretty-printing mixed content is hard. It seems that the only other sensible
choice is to switch mixed content nodes to raw formatting. In a way the code in
this change is a weaker version of that - it removes indentation around text
nodes but still keeps it around element siblings/children.
Thus we can switch to mixed-raw formatting at some point later, which will be
a superset of the current behavior.
To do this we have to either switch at the first text node (.NET XmlDocument
does that), or scan the children of each element for a possible text node and
switch before we output the first child.
The former behavior seems non-intuitive (and a bit broken); unfortunately, the
latter behavior can cost up to 20% of the output time for trees *without* mixed
content.
Fixes#13.
Since all string allocations are pointer-aligned to avoid aligning more
frequent node allocations, we can rely on that in string encoding.
Encoding page offset and block size in sizeof(void*) units increases the
maximum memory page size from 64k to 256k on 32-bit and 512k on 64-bit
platforms.
Fixes#35.
The implementations generated a string with an internal null terminator; this
went unnoticed since unit test string verification did not perform string
equality check properly (it compared XPath string result as a C-string, thus
stopping at the first null terminator).
Fixes#36.
Make float/double round-trip
This change also adds xml_text::set and xml_attribute::set_value overloads for float so that float is only printed using just enough digits to represent float, instead of enough digits to represent double.
It's sufficient to define PUGIXML_HEADER_ONLY anywhere now, source is included
automatically.
This is a second attempt; this time it includes a workaround for QMake bug
that caused it to generate incorrect Makefile.
Unfortunately, standard headers on MinGW32 insist on undefining off64_t
and _wfopen extensions if __STRICT_ANSI__ is true (e.g. C++11 mode). This
leads to compilation errors since b7a1fec started to use _wfopen in strict
mode. That change erroneously checked GCC version - however, the version
itself is irrelevant; the actual criteria is whether mingw64 runtime is
used.
off64_t is not useful on MinGW32 since we only need it to open large files
on 64-bit platforms; unfortunately, the lack of _wfopen means we won't be
able to support wide-char paths on Windows for MinGW32.
Fixes#24.
Since MinGW 4.5 does not define these functions if __STRICT_ANSI__ is defined
(in case of _wfopen it defines it inconsistently between stdio.h and wchar.h)
use the baseline functions for MinGW 4.5 and earlier.
Fixes#23.
Since copying no longer relies on child insertion we have to also reserve
space in the hash table for the allocator so that pointer manipulations are
guaranteed to succeed.
node_copy_string relied on the fact that target node had an empty name and
value. Normally this is a safe assumption (and a good one to make since it
makes copying faster), however it was not checked and there was one case when
it did not hold.
Since we're reusing the logic for inserting nodes, newly inserted declaration
nodes had the name set automatically to xml, which in our case violates the
assumption and is counter-productive since we'll override the name right after
setting it.
For now the best solution is to do the same insertion manually - that results
in some code duplication that we can refactor later (same logic is partially
shared by _move variants anyway so on a level duplicating is not that bad).
Remove redundant this-> from type() call (argument used to be called type,
but it's now type_).
Use _root member directly when possible instead of calling internal_object.
This should completely eliminate the confusion between load and load_file.
Of course, for compatibility reasons we have to preserve the old variant -
it will be deprecated in a future version and subsequently removed.
Previously push_back implementation was too big to inline; now the common case
(no realloc) is small and realloc variant is explicitly marked as no-inline.
This is similar to xml_allocator::allocate_memory/allocate_memory_oob and
makes some XPath queries 5% faster.
In some cases constant overhead on step evaluation is important - i.e. for
queries that evaluate a simple step in a predicate expression. Eliminating
a redundant function call thus can prove worthwhile.
This change makes some queries (e.g. //*[not(*)]) 4% faster.
Since page size can be customized let's do a special validation check for
compact encoding. Right now it's redundant since page size is limited by
64k in alloc_string, but that may change in the future.
This allows us to add pi value to restore target support for PI nodes without
increasing the memory usage for other nodes.
Right now the PI node has a separate header that's used for allocated bit;
this allows us to reduce header bitcount in the future.
Previously setting a large page size (i.e. 1M) would cause dynamic string
allocation to assert spuriously. A page size of 64K guarantees that all
offsets fit into 16 bits.
Split number/boolean filtering logic into two functions. This creates an
extra copy of a remove_if-like algorithm, but moves the type check out of
the loop and results in better organized filtering code.
Consolidate test-based dispatch into apply_predicate (which is now a member
function).
This lets us do fewer null pointer checks (making printing 2% faster with -O3)
and removes a lot of function calls (making printing 20% faster with -O0).
To get more benefits from constant predicate/filter optimization we rewrite
[position()=expr] predicates into [expr] for numeric expressions. Right now
the rewrite is only for entire expressions - it may be beneficial to split
complex expressions like [position()=constant and expr] into [constant][expr]
but that is more complicated.
last() does not depend on the node set contents so is "constant" as far as
our optimization is concerned so we can evaluate it once.
Numeric and boolean constant expressions in filters are different in that
to evaluate numeric expressions we need a sorted order, but to evaluate
boolean expressions we don't. The previously implemented optimization adds
an extra sorting step for constant boolean filters that will be more expensive
than redundant computations.
Since constant booleans are sort of an edge case, don't do this optimization.
This allows us to simplify apply_predicate_const to only handle numbers.
Now expression is always _right for filter/predicate nodes to make optimize()
simpler. Additionally we now use predicate metadata to make is_posinv_step()
faster.
This introduces a weak ordering dependency in rewrite rules to optimize() -
classification has to be performed before other optimizations.
If a filter/predicate expression is a constant, we don't need to evaluate it
for every nodeset element - we can evaluate it once and pick the right element
or keep/discard the entire collection.
If the expression is 1, we can early out on first node when evaluating the
node set - queries like following::item[1] are now significantly faster.
Additionally this change refactors filters/predicates to have additional
metadata describing the expression type in _test field that is filled during
optimization.
Note that predicate_constant selection right now is very simple (but captures
most common use cases except for maybe [last()]).
A page can fail to allocate during attribute creation; this case was not
previously handled.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1080 99668b35-9821-0410-8761-19e4c4f06640
When removing a node or attribute, we know that the parent has at least one
node/attribute so a null pointer check is redundant.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1078 99668b35-9821-0410-8761-19e4c4f06640
If the requested evaluation mode is not _all, we can use this mode for the
last predicate/filter expression and exit early.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1073 99668b35-9821-0410-8761-19e4c4f06640
Using pointers instead of node/attribute objects allows us to use knowledge
about the tree to guarantee that pointers are not null. This results in
less null checks (10-20% speedup with optimizations enabled) and less
function calls (5x speedup with optimizations disabled).
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1072 99668b35-9821-0410-8761-19e4c4f06640
Some steps relied on step_push rejecting null inputs; this is no longer
the case. Additionally stepping now more rigorously filters null inputs.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1069 99668b35-9821-0410-8761-19e4c4f06640
Sometimes when evaluating the node set we don't need the entire set and
only need the first element in docorder or any element. In the absence of
iterator support we can still use this information to short-circuit
traversals.
This does not have any effect on straightforward node collection queries,
but frequently improves performance of complex queries with predicates
etc. XMark benchmark gets 15x faster with some queries enjoying 100x
speedup on 10 Mb dataset due to a significant complexity improvement.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1067 99668b35-9821-0410-8761-19e4c4f06640
select_node is shorter and mistyping nodes as node or vice versa should
not lead to any issues since return types are substantially different.
select_single_node method still works and will be deprecated with an
attribute and removed at some point.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1065 99668b35-9821-0410-8761-19e4c4f06640
This method is equivalent to xml_node::select_single_node. This makes
select_single_node faster in certain cases by avoiding an allocation and -
more importantly - paves the way for future step optimizations.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1064 99668b35-9821-0410-8761-19e4c4f06640
Use descendant-or-self::node() transformation for self, descendant and
descendant-or-self axis. Self axis should be semi-frequent; descendant
axes should not really be used with // but if they ever are the complexity
of the step becomes quadratic so it's better to optimize this if possible.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1063 99668b35-9821-0410-8761-19e4c4f06640
When looking for an attribute by name, finding the first attribute means
we can stop looking since attribute names are unique. This makes some
queries faster by 40%.
Another very common pattern in XPath queries is finding an attribute with
a specified value using a predicate (@name = 'value'). While we perform an
optimal amount of traversal in that case, there is a substantial overhead
with evaluating the nodes, saving and restoring the stack state, pushing
the attribute node into a set, etc. Detecting this pattern allows us to
use optimized code, resulting in up to 2x speedup for some queries.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1061 99668b35-9821-0410-8761-19e4c4f06640
The actual condition for the optimization is invariance from context list
-- this includes both position() and last().
Instead of splitting the posinv concept just include last() into
non-posinv expressions - this requires sorting for boolean predicates that
depend on last() and do not depend on position(). These cases should be
very rare.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1060 99668b35-9821-0410-8761-19e4c4f06640
Comment value can not contain the string "--" or end with "-". Since
comments do not support escaping, we're handling this by adding an extra
space after the first "-". A string of "-" thus turns into "- - - ...".
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1058 99668b35-9821-0410-8761-19e4c4f06640
Insert is now unsafe - since we don't have a way to handle rehash()
failures transparently we need to reserve space beforehand. Reserve is now
called before every tree-mutating operations and it guarantees that we
can perform 16 arbitrary pointer mutations after that.
This fixes all test crashes with compact mode.
All ad-hoc attribute operations are now implemented as explicit low-level
functions, and xml_node just uses them. Additionally extract commonly used
is_attribute_of and move detaching of node from append_node to remove_node
- append_node now only works on detached nodes (small increase in parsing
performance).
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1056 99668b35-9821-0410-8761-19e4c4f06640
Parent pointers need to be able to reach everywhere within a page to
minimize shared parent pointer reuse unless it's absolutely necessary.
This reduces parent hash utilization on all test cases to <1%.
Rename compact_parent to compact_shared_parent.
Split the implementation into a generic one with adjustable range and a
special implementation for parent (may need to use 2 bytes on that one
later).
Optimize compact_string and compact_pointer to use minimal amount of math
and move slow hash paths into no-inline functions so that compiler can
inline the fast-paths.
Merge compact_pointer_generic and compact_pointer_forward and optimize.
To make this possible name and value in the node structure had to be
merged into one contents field. Not sure what to do with node_pi, since it
is the only type that required both.
The storage uses one hash table for fallbacks and simple difference
encoding for node and string pointers.
This is a work in progress implementation - while node pointers seem to
work properly, string encoding is inefficient and parent pointers could
use more tuning.
No performance or compatibility work has been done either.
When xpath_string is heap-allocated we always know the length of the
string at some point - it is now stored in the object. This reduces
redundant string length calculations and makes string_value() much faster
in case it has to concatenate strings.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1053 99668b35-9821-0410-8761-19e4c4f06640
translate() with constant arguments now uses a 128-byte table and a table
lookup instead of searching characters in the source string. The table is
generated during query optimization.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1052 99668b35-9821-0410-8761-19e4c4f06640
Node ancestor search now terminates early if ancestor is found before the
document root (only happens if nodes were at the same depth).
Sibling search now steps synchronously for left and right nodes to avoid
worst-case performance when we go in the wrong direction and have to scan
a big list (this comes at the cost of average performance since in the
best case we do 2x more operations).
Node comparison is now done using node pointers to elide some null
comparisons since the structure of the search guarantees that they are
handled properly.
All of the above results in ~2x faster document order comparison on
complex documents.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1050 99668b35-9821-0410-8761-19e4c4f06640
XPath evaluation frequently produces sequences that are sorted but are not
tagged as such (area for improvement...). Doing a linear scan before
sorting is cheap and results in tremendous speedup for already sorted
sequences (especially if document_buffer_order optimization does not
apply).
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1049 99668b35-9821-0410-8761-19e4c4f06640
While gcc and clang can eliminate dependency on s in the inner loop of
PUGI__SCANWHILE_UNROLL, MSVC emits a series of register increments.
Rewriting the code to explicitly remove the dependency keeps similar
codegen on gcc/clang but improves codegen on MSVC for a 10% performance
boost.
Also use unrolled scanning in text_output_escaped (2% faster).
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1048 99668b35-9821-0410-8761-19e4c4f06640
The page no longer contains 'data' field to use sizeof everywhere instead
of offsetof/sizeof inconsistency (that is required because some compilers
don't recognize offsetof as compile-time constant).
The page no longer contains 'memory' field that is now encoded as an
offset byte before the page - this allows us to save one pointer from the
static page in the document to keep the size the same as in v1.2 (binary
compatibility).
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1046 99668b35-9821-0410-8761-19e4c4f06640
Use the same flag that is used for marking name/value in nodes/attributes
as shared. This reduces document structure size and makes some amount of
sense (although admittedly is a bit of a hack).
We need to bring document _memory size back down to 192 bytes and this is
the first step.
git-svn-id: https://pugixml.googlecode.com/svn/trunk@1045 99668b35-9821-0410-8761-19e4c4f06640
