In order to try to deeply understand java streams and spliterators, I have some subtle questions about spliterator characteristics:
Stream.of() (Stream.of() without args)
Stream.empty(): SUBSIZED, SIZED
Stream.of(): SUBSIZED, IMMUTABLE, SIZED, ORDERED
Stream.empty() doesn’t have the same characteristics of
Stream.of()? Note that it has impacts when using in conjunction with Stream.concat() (specially not having
ORDERED). I would say that
Stream.empty() should have not just IMMUTABLE and ORDERED but also DISTINCT and NONNULL. Also it make sense
Stream.of() with only one argument having DISTICT.
LongStream.of() not having NONNULL
Just noticed that NONNULL is not available in
NONNULL a main characteristics of all
LongRange.range(,): SUBSIZED, IMMUTABLE, NONNULL, SIZED, ORDERED, SORTED, DISTINCT
LongStream.range(,).boxed(): SUBSIZED, SIZED, ORDERED
.boxed() loses all these characteristics? It shouldn’t lose any.
I understand that
.mapToObj() can lose the NONNULL, IMMUTABLE and DISTICT, but
.boxed()… doesn’t make sense.
.peek() loses IMMUTABLE and NONNULL
LongStream.of(1): SUBSIZED, IMMUTABLE, NONNULL, SIZED, …
LongStream.of(1).peek(): SUBSIZED, SIZED, …
.peek() loses these characteristics?
.peek shouldn’t really lose any.
.limit() loses SUBSIZED, IMMUTABLE, NONNULL, SIZED
Just notice that these operations loses SUBSIZED, IMMUTABLE, NONNULL, SIZED. Why? If the size is available, then it’s easy to calculate the final size as well.
.filter() loses IMMUTABLE, NONNULL
Just notice that this operations loses as well SUBSIZED, IMMUTABLE, NONNULL, SIZED. It make sense to lose SUBSIZED and SIZED, but the others two doesn’t make sense. Why?
I will appreciate if someone that understands deeply the spliterator could bring some clarity. Thanks.
I have to admit that I had difficulties too when I first tried to find out the actual meaning of the characteristics and had the feeling that their meaning was not clearly settled during the implementation phase of Java 8 and is used inconsistently for that reason.
Characteristic value signifying that the element source cannot be structurally modified; that is, elements cannot be added, replaced, or removed, so such changes cannot occur during traversal.
It’s strange to see “replaced” in this list, which is usually not considered a structural modification when speaking of a
List or an array and consequently, stream and spliterator factories accepting an array (that is not cloned) report
If we interpret this more generously as “as long as not observable by the client”, there is no significant difference to
CONCURRENT, as in either case some elements will be reported to the client without any way to recognize whether they were added during traversal or whether some were unreported due to removal, as there is no way to rewind a spliterator and compare.
The specification continues:
A Spliterator that does not report
CONCURRENTis expected to have a documented policy (for example throwing
ConcurrentModificationException) concerning structural interference detected during traversal.
And that’s the only relevant thing, a spliterator reporting either,
CONCURRENT, is guaranteed to never throw a
ConcurrentModificationException. Of course,
SIZED semantically, but that has no consequence to the client code.
In fact, these characteristics are not used for anything in the Stream API, hence, using them inconsistently would never get noticed somewhere.
This is also the explanation why every intermediate operation has the effect of clearing the
NONNULL characteristics: the Stream implementation doesn’t use them and its internal classes representing the stream state do not maintain them.
NONNULL is not used anywhere, so it’s absence for certain streams has no effect. I could track down the
LongStream.of(…) issue down to the internal use of
Arrays.spliterator(long, int, int) which delegates to
Spliterators.spliterator(long array, int fromIndex, int toIndex, int additionalCharacteristics):
The returned spliterator always reports the characteristics
SUBSIZED. The caller may provide additional characteristics for the spliterator to report. (For example, if it is known the array will not be further modified, specify
IMMUTABLE; if the array data is considered to have an encounter order, specify
ORDERED). The method
Arrays.spliterator(long, int, int)can often be used instead, which returns a spliterator that reports
Note (again) the inconsistent use of the
IMMUTABLE characteristic. It is again treated like having to guaranty the absence of any modification, while at the same time,
Arrays.spliterator and in turn
LongStream.of(…) will report the
IMMUTABLE characteristic, even by specification, without being able to guaranty that the caller won’t modify their array. Unless we consider setting an element not to be a structural modification, but then, the entire distinction becomes nonsensical again, as arrays can’t be structurally modified.
And it clearly specified no
NONNULLcharacteristic. While it is obvious that primitive values can’t be
null, and the
Spliterator.Abstract<Primitive>Spliterator classes invariably inject a
NONNULL characteristic, the spliterator returned by
Spliterators.spliterator(long,int,int,int) does not inherit from
The bad thing is, this can’t be fixed without changing the specification, the good thing is, it has no consequences anyway.
So if we ignore any issues with
NONNULL, which have no consequences, we have
limit. This is a well known issue, result of the way
limit have been implemented by the Stream API. Other implementations are imaginable. This also applies to the combination of an infinite stream with a
limit, which should have a predictable size, but given the current implementation, has not.
Stream.empty(). It sounds reasonable that an empty stream does not impose constraints on the result order. But
Stream.concat(…)’s behavior of releasing the order when only one input has no order, is questionable. Note that being too aggressive regarding ordering is nothing new, see this Q&A regarding a behavior that was considered intentional first, but then has been fixed as late as Java 8, update 60. Perhaps,
Stream.concat should have been discussed right at this point of time too…
The behavior of
.boxed() is easy to explain. When it has been implemented naively like
.mapToObj(Long::valueOf), it will simply loose all the knowledge, as
mapToObj can not assume that the result is still sorted or distinct. But this has been fixed with Java 9. There,
SUBSIZED|SIZED|ORDERED|SORTED|DISTINCT characteristics, maintaining all characteristics that have relevance to the implementation.