Debugging a collection iterator

Need Help
1

I have a collection (internally a FilterIterator), that starts with 3 elements and has a rejection that reduces that to 2. But running a foreach on it processes only a single element. I’m trying to debug this, but at a loss as to where I can put a useful breakpoint.

I’m using CakePHP 3.8.6, and PHPStorm. If I put a breakpoint on the foreach line and step into it, it goes into the reject function of the CollectionTrait, where it’s going to call my callback function, and the call stack indicates that this is called from the standard CallbackFilterIterator. Other breakpoints can find me inside the __debugInfo function.

The collection does know that it’s got 2 items in it, and if I add toArray() in the foreach, then it iterates over all the elements. (I am using this to work around this particular issue in the short term, but I don’t like not knowing what’s actually wrong, because it throws every other iteration on collections everywhere in my application into question.) Adding ->rewind() before the foreach has no effect. Uses of the same collection elsewhere in the code seem just fine. (Note: Not strictly the same object, but a separate collection generated by the exact same code on the exact same input, so should be the same for all intents and purposes.)

This really should be rock-solid, I’d have thought. I can’t think of what might be breaking this. But even more frustrating, I can’t seem to debug what’s going wrong. :frowning:

2

Is it possible to put some dummy step into the process that you can break on?

$x = 1; //a meaning less statement that can serve as a break point

This ‘safe’ stepping stone in the river might allow you to then look around at the other values and figure out what is happening.

3

I can step through the contents of the foreach loop. All the variables are set as I expect for the first pass through. When it gets to the end of the loop, it simply doesn’t go back to the start of the loop, it carries on to whatever is after it.

4

Breaking inside the loop isn’t useful, in other words. I need to break somewhere in the iterator where it’s deciding whether there are more items to iterate over.

5

And there is no way to place the stepping-stone break in the iterator callback? Or is breaking in there not allowed at all?

Or that’s still not where you need to see?

6

Seems that the underlying iterator is a base PHP class which cannot be stepped into, like substr, etc.

7

Hmm…

Isn’t your FilterIterator a class you’ve written that extends php’s FileIterator? with your specific implementation of accept()?

An alternative would be to use the cake Collections->filter(). You should be able to step into the callable you provide to satisfy that… At least I know I routinely debug from inside those callables

8

It’s not my FilterIterator class, it’s Cake’s, which is the result of using the stock filter call. The data in the collection is, as far as I can tell, all correct, as is my callable. It’s the next function that I think would be instructive to step into, but Cake’s FilterIterator doesn’t re-implement that, so the actual implementation (from the PHP base class) is presumably written in C.

9

Im not sure if I fully understand what are you doing.

Cant you check printing with debug() on first line of the filter and last line (check filtered/not filtered) to see if all three items are being processed at least.

Can you share a reduced sample snippet to see any errors/bugs?

10

My filter function is being called. If I put a breakpoint in it, it is hit three times, and it correctly returns true for two of them. I have multiple other places where this exact code is working just fine (the collection is being created in an accessor on an entity). In this one place, I can very simply write:

foreach ($items as $item) {
    $x = 1;
}

and when stepping through, the $x = 1 line will only be hit once, while:

foreach ($items->toArray() as $item) {
    $x = 1;
}

will hit it twice.

11

Does phpstorm have configured to show the values of the collection automatically? can you disable that?

It may have something like issue #8438

It may explain why it works with toArray instead of the collection

12

This also breaks if I’m not debugging it, and PHPStorm isn’t involved at all. That’s why I started trying to debug it in the first place.

13

Well, it has to be the iterator, the data or the interaction of these two specific players…

Is it only one specific set of three particular elements that causes the problem. Can you send other sets through and have them work?

If it’s just these elements then we have to examine them for problems.

If nothing works in the filter… Can you write the code using a Reduce instead?

14

I’ve finally had some more time to track this down a bit more. There are specifics that I don’t think should be relevant to the situation, but it turns out are. The collection in question is returned by an accessor function on an entity. Let’s say I have X hasMany Y, so $x->ys is an array. $x->special_ys returns a collection of just the Y’s that I’m interested in, created using ->reject().

If I use $x->special_ys, then all is well in all cases.

If I assign $ys = $x->special_ys and then iterate over $ys, it gives me only the first one. But $ys->count() gives the right count, and $ys->extract('field') returns that field for each Y in the collection, and $ys->toArray() does return an array with all of the Ys in it.

I have tried changing my accessor to use filter with a reversed condition instead of reject, no change in behaviour.

I can certainly use toArray in this particular case. And it’s just generating a bit of output, so I can probably restructure it to use extract with a closure instead of my own foreach loop. But I still worry that there are more such uses lurking somewhere in these few hundred thousand lines of code…

15

Further detail, if I remove an earlier call to that accessor, then it works correctly in this place. So it’s perhaps something about the unwrap function of Cake’s filterIterator class. I will see about writing this up as an issue on GitHub.

16

I have narrowed it down still further. It’s something to do with inspecting the collection while iterating it. Stepping through in a debugger will trigger the erroneous behaviour, as will trying to count the elements. So some of what I wrote above may well not actually be true symptoms, but rather artifacts of me debugging it.

Here’s a very simple bit of code that will demonstrate the problem:

$x = collection([1, 2, 3, 4, 5])->reject(function ($i) { return $i > 3; });
foreach ($x as $i) {
	$x->count();
	pr($i);
}

For me, this generates just “1” as the output. If I remove the count call, it shows 1 2 3.

Maybe others on newer versions of Cake can double-check this? I am using 3.8.6 at the moment.