Walk me through writing your own Collector. What do its four functions do, and what do the characteristics promise the pipeline?
A collector is a supplier of a mutable container, an accumulator that folds one element into it, a combiner that merges two containers, and a finisher. The combiner runs only in parallel, which is why an untested combiner is the usual defect, and the characteristics tell the pipeline what it may skip or reorder.
What the interviewer is scoring
- Does the candidate explain why accumulating into a mutable container is wrong with reduce and right with collect
- Whether the combiner is identified as parallel-only, and therefore as the part their sequential tests never ran
- That CONCURRENT is described as one shared container rather than as a synonym for faster
- Can they say what declaring IDENTITY_FINISH commits them to
- Whether they reach for an existing composed collector before writing one at all
Answer
Mutable reduction, and why reduce is the wrong tool
reduce performs a functional reduction: it expects an associative function that produces a new value from two values, and it is free to call that function in any grouping. Accumulating into a mutable container with reduce therefore fails in a specific way. The accumulating function must not modify its argument, because the framework may pass the same partial result into more than one call, so people write a version that copies the container per element instead, which is correct and quadratic.
collect exists for precisely this case. It performs a mutable reduction, where the result is built up by mutating containers the pipeline itself created and owns, and it knows how many containers exist and who may touch each one. A Collector is the description of that process, which is why writing one means answering four questions rather than supplying one function.
The four functions and where each one runs
The supplier creates a fresh, empty container. In a sequential pipeline it is called once; in a parallel pipeline it is called once per split, so it must return a new container each time and never a shared one.
The accumulator folds a single element into a container. Each container is confined to one thread at a time, which is what lets the accumulator mutate freely without synchronisation.
The combiner takes two containers and merges them, returning a container holding the contents of both. In a sequential pipeline it is never called. That single fact accounts for most broken custom collectors: the code passes every test, then produces wrong answers only when the pipeline goes parallel, and only sometimes, because it depends on how the source split.
The finisher converts the accumulation container to the result type — sorting it, wrapping it unmodifiable, extracting one value out of it. If the container is already the result, you declare IDENTITY_FINISH and the pipeline skips the step.
// Longest string per initial letter, built as one pass.
Collector<String, Map<Character, String>, Map<Character, String>> longestByInitial =
Collector.of(
HashMap::new,
(map, s) -> map.merge(s.charAt(0), s,
(a, b) -> b.length() > a.length() ? b : a),
// Runs only in parallel: merges two partial maps with the same rule.
(left, right) -> {
right.forEach((k, v) -> left.merge(k, v,
(a, b) -> b.length() > a.length() ? b : a));
return left;
},
Collector.Characteristics.IDENTITY_FINISH);
Note that the accumulator's rule and the combiner's rule are the same rule applied at two granularities. When they disagree, the sequential and parallel answers differ, and that is exactly the bug that survives review because both halves look plausible in isolation.
What the characteristics promise
IDENTITY_FINISH asserts that the accumulation type and the result type are the same and that the identity function is an acceptable finisher, letting the pipeline hand your container straight back. Declaring it when the container is a mutable HashMap you intended to wrap is how a caller ends up able to mutate what you thought was a finished result.
UNORDERED asserts that the collection operation does not depend on encounter order, which frees the pipeline from preserving it and can avoid a merge that would otherwise have to respect order. Declaring it on a collector building a List is wrong, because a list has order that callers will observe; declaring it on one building a HashSet is correct, because the container has no order to preserve.
CONCURRENT is the strongest and most misread. It asserts that the accumulator is safe to call from multiple threads on the same container, so the pipeline may use one shared container instead of one per split and skip combining entirely. That is only a legal optimisation for an unordered pipeline, so it takes effect when the collector is also UNORDERED or the source is unordered. It is not a performance hint; it is a thread-safety claim about your accumulator, and making it falsely produces data races rather than a slow result. This is the distinction between groupingBy and groupingByConcurrent: the latter accumulates into one concurrent map and therefore gives up encounter order within each group.
Where custom collectors go wrong
Beyond the untested combiner, three failures recur. A combiner that is not associative gives answers that depend on how the workload split, which reads as flakiness. A supplier that returns a shared or captured container turns every split into a race, and it happens most often when someone refactors a collector into a field to "avoid allocation". And a stateful accumulator that keeps information outside the container — a counter in the enclosing scope, a running total in a field — breaks as soon as there is more than one container, since the state was never part of what the pipeline knew to merge.
The test that catches all three is cheap: run the same collection sequentially and in parallel over a source large enough to split, and assert the results are equal. Do it over a few different input sizes, because whether a source splits at all depends on its size and its spliterator.
Usually you do not need one
Before writing a collector, check whether composition gives you the same thing. collectingAndThen bolts a finisher onto an existing collector, which covers most of the "collect then wrap or sort" cases. mapping, filtering and flatMapping adapt a downstream collector, the latter two added in Java 9, and they compose inside groupingBy so a grouped-and-transformed result needs no new type. teeing, added in Java 12, feeds the stream to two collectors and merges their results, which handles the class of problem where people reach for a custom collector to compute two aggregates in one pass.
A custom collector earns its place when the accumulation container is genuinely domain-specific — a histogram with fixed buckets, a reservoir sample, an interval tree — because then the merge rule is real logic that no composition expresses. Reaching for one to avoid two lines of composition means writing a combiner, and therefore owning a parallel-correctness argument, in exchange for nothing.
Likely follow-ups
- Why does Collectors.toMap require a merge function where groupingBy does not?
- What has to be true of your combiner for a parallel collect to produce the same result as a sequential one?
- How does groupingByConcurrent differ from groupingBy in what it guarantees about ordering?
- When does a collector need a finisher rather than exposing its accumulation container directly?
Related questions
- Streams are lazy until a terminal operation runs. Why does that matter, and when would you make one parallel?mediumAlso on streams and collectors4 min
- Why would you stream a large file rather than read it into memory, and what is backpressure?mediumAlso on streams4 min
- A list of a million small objects uses far more memory than the data inside them. Where is it going?mediumSame kind of round: concept5 min
- A promise rejects and nothing is awaiting it. What does Node do?hardSame kind of round: concept4 min
- You kick off background work from a controller action and it throws once the response has gone out. What is happening?mediumSame kind of round: coding5 min
- Find the cheapest route between two nodes in a weighted directed graph. Then tell me what breaks if one edge has a negative weight.mediumSame kind of round: coding4 min
- Sorting a list throws IllegalArgumentException saying the comparison method violates its general contract. What did the comparator get wrong?mediumSame kind of round: concept5 min
- Why would you bind configuration with @ConfigurationProperties rather than @Value, and how do you make a bad value fail at startup instead of at the first request?mediumSame kind of round: concept4 min