Skip to content

Cheat to speedup using nqp::iterator+nqp::shift on a list#1809

Open
MasterDuke17 wants to merge 1 commit into
MoarVM:mainfrom
MasterDuke17:hackish_speedup_for_shifting_iterators
Open

Cheat to speedup using nqp::iterator+nqp::shift on a list#1809
MasterDuke17 wants to merge 1 commit into
MoarVM:mainfrom
MasterDuke17:hackish_speedup_for_shifting_iterators

Conversation

@MasterDuke17

Copy link
Copy Markdown
Contributor

We can assume the backing array is in a good state so reach in directly during iteration.

time MVM_SPESH_BLOCKING=1 nqp -e 'my $l := nqp::list(); my int $p := 0; while $p++ < 10_000 { nqp::push($l, $p) }; my int $j := 10000; while $j-- { my $iter := nqp::iterator($l); nqp::while($iter, my $a := nqp::shift($iter)) }' drops from ~1.46s to ~1.25s and time MVM_SPESH_BLOCKING=1 nqp -e 'my $l := nqp::list_s(); my int $p := 0; while $p++ < 10_000 { nqp::push_s($l, $p) }; my int $j := 10000; while $j-- { my $iter := nqp::iterator($l); nqp::while($iter, my $a := nqp::shift_s($iter)) }' drops from ~2.01s to ~1.83s.

@MasterDuke17

Copy link
Copy Markdown
Contributor Author

I noticed this when looking at Raku/nqp#685.

@niner

niner commented May 19, 2024

Copy link
Copy Markdown
Contributor

Why can we assume that the array is in a good state?

@MasterDuke17

Copy link
Copy Markdown
Contributor Author

Why can we assume that the array is in a good state?

Well, I believe we’re allowed to act as if it’s in a good state. Modifying an array during iteration is not allowed AIUI. Also, the iterator keeps track of the index compared to the size of the array.

@lizmat

lizmat commented Jul 21, 2025

Copy link
Copy Markdown
Contributor

Is this PR still good? If so, what would be against merging it?

@raiph

raiph commented Jul 21, 2025

Copy link
Copy Markdown

Modifying an array during iteration is not allowed AIUI.

I couldn't tell if this comment was purely about nqp level commitments or also about Raku level commitments.

For Raku, I'll say this. Shortly after 6.d shipped, jnthn commented under an SO answer by Vadim Belman. Based on my existing belief that modification during iteration was fine, plus Vadim's answer, plus jnthn's comment under it, I'd say it looks like it is "officially" allowed in Raku 6.d.

For nqp, I'll say this. The doc for nqp::iterator says (with my added EMPHASIS):

You can also use nqp::iterator() to iterate over a list, which is functionally equivalent to cloning the list and not having to know which form of shift to have to use.

my $list := nqp::list('a', 'b', 'c');
my $iter := nqp::iterator($list);

while $iter {
    say(nqp::shift($iter));
}

Using iterator this way proved to be not very well optimizable, so this form will probably be DEPRECATED in the future. Depending on circumstances one could use clone instead, or use an index variable...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants