WIP: ENH: Enhance ImageVisual with polar capabilities, add example by kmuehlbauer · Pull Request #2133 · vispy/vispy

kmuehlbauer · 2021-07-06T15:34:01Z

This PullRequest enhances the ImageVisual to display a polar representation of itself via kwarg polar (eg. polar=(dir, loc, origin))

added property polar to work on existing ImageVisuals
method impostor is needed
dir - direction ("cw", "ccw", clockwise/counterclockwise)
loc (location of theta=0, eg "N", "E" etc. )
origin (which point is taken to map theta/range, eg. "UL", "LR")

Todo: Add tests, add imap-functionality to remap coordinates.

Context:

As asked on SO (https://stackoverflow.com/questions/68177737/how-to-convert-visuals-image-to-polar-using-vispy) and the chat on gitter (https://gitter.im/vispy/vispy?at=60e1e0194e325e6132bc5cf9) the normal transform system can't be used (AFAICT) to create what we want. We do not have access to all these image attributes (size, direction, location, origin) within the transform chain. So I decided to put this directly into ImageVisual. That way we leave the transform chain alone and can do all other kinds of transforms. Not sure, if this the best approach, but it works quite nicely. See also the attached example.

kmuehlbauer · 2021-07-06T15:35:00Z

If anyone has ideas how to test this, please let me know.

kmuehlbauer · 2021-07-07T12:56:37Z

almarklein

Nice work! I made a few comments. I'm not a heavy user of polar plots, so I mostly looked at the technical aspects.

kmuehlbauer · 2021-07-08T14:30:39Z

Thanks @almarklein. I need to rework the polar transform shader again, since in some conditions theta is running out of[0, 2 * pi] ...

kmuehlbauer · 2021-07-09T14:08:26Z

I'm still thinking how to test this...

kmuehlbauer · 2021-07-11T13:25:55Z

@requires_application()
@pytest.mark.parametrize('ori', ["UL", "UR", "LR", "LL"])
@pytest.mark.parametrize('dir', ["cw", "ccw"])
@pytest.mark.parametrize('loc', ["N", "NW", "W", "SW", "S", "SE", "E", "NE"])
def test_image_polar(ori, dir, loc):
    """Test image visual"""
    csize = 32
    with TestingCanvas(size=(csize, csize), bgcolor='w') as c:
        if ori in ["UL", "LR"]:
            tr1 = (STTransform(scale=(3.0, 3.0, 1), translate=(csize / 2, csize / 2, 0)))
        else:
            tr1 = (
                STTransform(scale=(1.0, 1.0, 1), translate=(csize / 2, csize / 2, 0)))
        shape = (csize // 4, csize // 2) + ()
        np.random.seed(379823)
        data = np.random.randint(0, 255, size=shape, dtype=np.uint8)
        image = Image(data,
                      polar=dict(dir=dir, loc=loc, ori=ori),
                      cmap="grays",
                      method='impostor',
                      interpolation="nearest",
                      parent=c.scene)
        image.transform = tr1
        assert_image_approved(c.render(), f"visuals/polar_images/image_polar_{ori}_{dir}_{loc}.png")

This creates 64 32x32 px grayscale images, each approx 850 bytes. That sums up to roughly 54kB. Would this work for https://github.com/vispy/test-data? Then I could add those files to the test-repo.

almarklein · 2021-07-12T10:01:16Z

This creates 64 32x32 px grayscale images, each approx 850 bytes. That sums up to roughly 54kB. Would this work for https://github.com/vispy/test-data? Then I could add those files to the test-repo.

Sounds good to me - 54k is very modest.

djhoese · 2021-07-12T15:37:31Z

I have mixed feeling about this PR. In the end these updates are working with coordinates. In this PR they are texture coordinates, but in the original PolarTransform they are vertex coordinates. My first issue is that this update to the ImageVisual doesn't fix anything about the PolarTransform and for users who may want to use it in other Visuals. Maybe this being done in the ImageVisual is necessary to do this "right" and get the most accurate polar display with the most flexibility. If so, I'm fine with that. Could some or all of this logic be moved to the PolarTransform and have the user control these direction, location, and origin parameters in PolarTransform.__init__?

Could origin also be accomplished by providing a negative scale value for the X or Y axis in an STTransform?

Does the impostor method need to be used here because operating on vertex coordinates will produce bad results? Like drawing a straight line between two vertices instead of along a polar "path"?

kmuehlbauer · 2021-07-13T05:27:00Z

My first issue is that this update to the ImageVisual doesn't fix anything about the PolarTransform and for users who may want to use it in other Visuals.

True.

Maybe this being done in the ImageVisual is necessary to do this "right" and get the most accurate polar display with the most flexibility. If so, I'm fine with that.

Also true. This machinery does only work on ImageVisual, where we can extract the needed coordinates directly from the texture.

Could some or all of this logic be moved to the PolarTransform and have the user control these direction, location, and origin parameters in PolarTransform.__init__?

I'm not sure if there is a way, but in vispy code it is written, that non-linear transforms work best with impostor mode. This is also the mode which get's chosen if method="auto". I've tried to work this out using PolarTransform but it didn't find a way.

Could origin also be accomplished by providing a negative scale value for the X or Y axis in an STTransform?

That might work. I did similar before using MatrixTransform (eg. rotation), but something still blocked that way of solving.

Does the impostor method need to be used here because operating on vertex coordinates will produce bad results? Like drawing a straight line between two vertices instead of along a polar "path"?

That's the impression one get, if PolarTransform is applied to ImageVisual if subdivide is chosen as method.

I still have the feeling, that this should work the transform-way but either I can't really envision it to myself or I have other problems in understanding.

kmuehlbauer · 2021-07-13T14:05:49Z

@djhoese Thanks for asking these questions. They keep me going 😀 I've made some progress with subdivide but don't know how exactly this works (my bad).

djhoese · 2021-07-13T19:16:42Z

Well that looks good...right? Did the code get ugly? Does it still have to be done in the fragment shader?

kmuehlbauer · 2021-07-13T19:26:14Z

@djhoese No, it's plain simple in the vertex shader. But it's not configurable atm. I'll try to show some code tomorrow.

kmuehlbauer · 2021-07-14T12:36:37Z

import numpy as np
import vispy.app
from vispy import gloo
from vispy import visuals
from vispy.visuals.transforms import STTransform, PolarTransform, MatrixTransform


directions = ["cw", "ccw"]
dir_index = 0

locations = ['N', 'NW', 'W', 'SW', 'S', 'SE', 'E', 'NE']
loc_index = 0

origins = ["top", "bottom"]
ori_index = 0

polar = [True, False]
pol_index = 0

csize = 800


def cycle_state(states, index):
    new_index = (index + 1) % len(states)
    return states[new_index], new_index


class Canvas(vispy.app.Canvas):
    def __init__(self):

        vispy.app.Canvas.__init__(self, keys='interactive', size=(csize, csize))

        # Create image
        xmax = csize // 4
        xres = xmax / 360
        ymax = csize // 2
        yres = ymax / 100

        image = np.ones((xmax, ymax), dtype=np.uint8)
        print(image.shape)



        grad = np.linspace(0, 255, ymax, dtype=np.uint8)[None, :]
        image *= grad

        bl_y1_start, bl_y1_stop = int(yres * 80), int(yres * 90)
        bl_y2_start, bl_y2_stop = int(yres * 95), int(yres * 100)
        wt_y1_start, wt_y1_stop = int(yres * 5), int(yres * 10)
        wt_y2_start, wt_y2_stop = int(yres * 90), int(yres * 95)

        image[:, wt_y1_start:wt_y1_stop] = 255
        image[:, wt_y2_start:wt_y2_stop] = 255
        image[:, bl_y1_start:bl_y1_stop] = 0
        image[:, bl_y2_start:bl_y2_stop] = 0
        image[int(xres*10):int(xres*80)] = 255
        image[int(xres*340):int(xres*350)] = 255


        self.image = visuals.ImageVisual(image,
                                         cmap="grays",
                                         grid=(1, 360),
                                         interpolation="nearest",
                                         method='subdivide')

        tr1 = (STTransform(scale=(1.0, 1.0, 1), translate=(csize/2, csize/2, 0)))
        self.image.transform = tr1

        self.visuals = [self.image]
        self.title = (f"Direction (d): {directions[dir_index]} - "
                      f"Location (l): {locations[loc_index]} - "
                      f"Origin (o): {origins[ori_index]} - "
                      f"Polar (p): {polar[pol_index]}"
                      )
        self.show()

    def on_draw(self, ev):
        gloo.clear(color='w', depth=True)
        for vis in self.visuals:
            vis.draw()

    def on_resize(self, event):
        # Set canvas viewport and reconfigure visual transforms to match.
        vp = (0, 0, self.physical_size[0], self.physical_size[1])
        self.context.set_viewport(*vp)
        for vis in self.visuals:
            vis.transforms.configure(canvas=self, viewport=vp)

    def on_key_press(self, event):
        global dir_index, loc_index, ori_index, pol_index
        if event.key == 'd':
            dir0, dir_index = cycle_state(directions, dir_index)
        elif event.key == 'l':
            loc0, loc_index = cycle_state(locations, loc_index)
        elif event.key == 'o':
            ori0, ori_index = cycle_state(origins, ori_index)
        elif event.key == "p":
            pol0, pol_index = cycle_state(polar, pol_index)
        else:
            pass

        self.title = (f"Direction (d): {directions[dir_index]} - "
                      f"Location (l): {locations[loc_index]} - "
                      f"Origin (o): {origins[ori_index]} - "
                      f"Polar (p): {polar[pol_index]}")

        # direction
        dir0 = (-1. if directions[dir_index] == 'cw' else 1.)
        # location
        locmap = {
            'N': np.pi * 0.0,
            'NW': np.pi * 0.25,
            'W': np.pi * 0.5,
            'SW': np.pi * 0.75,
            'S': np.pi * 1.0,
            'SE': np.pi * 1.25,
            'E': np.pi * 1.5,
            'NE': np.pi * 1.75}
        loc0 = locmap[locations[loc_index]] / (np.pi * 2)
        # origin
        srcmap = {
            "top": 0,
            "bottom": 1,
        }
        ori0 = srcmap[origins[ori_index]]
        if polar[pol_index]:
            tr = (
                    # move image to center and scale
                    STTransform(scale=(1.0, 1.0), translate=(400, 400))

                    # 0
                    # just plain simple polar transform
                    * PolarTransform()
            
                    # 1
                    # pre scale image to work with polar transform
                    # PolarTransform does not work without this
                    * STTransform(scale=(2 * np.pi / self.image.size[0], 1.0))

                    # 2
                    # origin switch via translate.y, fix translate.y
                    * STTransform(translate=(self.image.size[0] * (ori0 % 2) * 0.5,
                                             -self.image.size[1] * (ori0 % 2)))

                    # 3
                    # location change via translate.x
                    * STTransform(translate=(self.image.size[0] * (loc0 - 0.25), 0.0))

                    # 4
                    # direction switch via inverting scale.x
                    * STTransform(scale=(dir0, 1.0))
            )
        else:
            tr = (
                    STTransform(scale=(1.0, 1.0), translate=(400, 400))

                    # origin switch via translate.y, fix translate.y
                    * STTransform(translate=(self.image.size[0] * (ori0 % 2) * 0.5,
                                             -self.image.size[1] * (ori0 % 2)))

                    # location change via translate.x
                    * STTransform(translate=(self.image.size[0] * (loc0 - 0.25), 0.0))

                    # direction switch via inverting scale.x
                    * STTransform(scale=(dir0, 1.0))
            )

        self.visuals[0].transform = tr
        self.update()

if __name__ == '__main__':
    win = Canvas()
    import sys
    if sys.flags.interactive != 1:
        vispy.app.run()

kmuehlbauer · 2021-07-14T12:51:45Z

The above now works with subdivide and the transforms in the vertex shader under the following constraints:

Initialize ImageVisual with kwarggrid=(1, N), where N is a reasonable high number (eg. 360)
Before PolarTransform several STTransform have to be applied in order to move the vertexes into the needed locations
These STTransform depend on the size of the ImageVisual, so the whole procedure can't just live as a standalone transform
Only top and bottom can be collapsed to center.

It looks like we can't source out everything to PolarTransform since knowledge of the image size is needed. So this has to live in ImageVisual or in a subclassed PolarImageVisual.

I'll not have time to give this more love until early august, so if 0.8 is due before, just remove it from the milestone.

djhoese · 2021-07-14T14:48:08Z

Very nice @kmuehlbauer! How do you want to go forward with this? Do you think this PR is needed?

Does the above example work fine with impostor still?

If we want to advertise the PolarTransform as an option like in the example above maybe we can update the docstring for it with examples and mention these "gotchas" of the ImageVisual including these chains of transforms.

kmuehlbauer · 2021-07-14T15:03:50Z

@djhoese No, the PR is probably not needed anymore.

I haven't tested with impostor yet. I was too happy to have subdivide running.

I'll clean up this mess here tomorrow and work along your suggestion.

djhoese · 2021-07-14T15:07:17Z

Amazing! Nice job. Can't wait to see the final code.

kmuehlbauer · 2021-07-15T09:05:16Z

@djhoese Unfortunately this does not work with impostor as it works with subdivide.

I'll not have time to give this more love until early august, so I will set this to draft here and remove the milestone. I've answered the SO question with the current findings (https://stackoverflow.com/questions/68177737/how-to-convert-visuals-image-to-polar-using-vispy).

kmuehlbauer · 2021-08-18T06:32:52Z

@djhoese @almarklein I'll pick this up again in a few days.

rougier · 2021-08-24T14:48:13Z

Just for the record, here are some examples of polar transforms with matplotlib. The idea is just to see which one can be implemented with this PR (it's mostly a matter of parameters):

import numpy as np
import matplotlib.pyplot as plt

R = np.random.uniform(1, 5, 250)
T = np.random.uniform(0, 45/180*np.pi, 250)

fig = plt.figure(figsize=(10,4), dpi=100)

ax = plt.subplot(1, 3, 1, projection="polar")
ax.scatter(T, R, 1, "black", zorder=10)
ax.set_rmin(1)                # Minimum radius
ax.set_rmax(5)                # Maximum radius
ax.set_rorigin(1)             # Origin position
ax.set_rticks(1+np.arange(5)) # Maximum radius
ax.set_thetamin(0)            # Minimum angle (degrees)
ax.set_thetamax(45)           # Maximum radius (degrees)
ax.set_theta_offset(0)        # Origin orientation

ax = plt.subplot(1, 3, 2, projection="polar")
ax.scatter(T, R, 1, "black", zorder=10)
ax.set_rmin(1) 
ax.set_rmax(5) 
ax.set_rorigin(0)
ax.set_rticks([1,5])
ax.set_thetamin(0)
ax.set_thetamax(180)
ax.set_theta_offset(0)
ax.set_xticks([])

ax = plt.subplot(1, 3, 3, projection="polar")
ax.scatter(T, R, 1, "black", zorder=10)
ax.set_rmin(1)
ax.set_rmax(5) 
ax.set_rorigin(-10)
ax.set_rticks([1,2,3,4,5])
ax.set_thetamin(0)
ax.set_thetamax(45)
ax.set_theta_offset(0.5 * 3*np.pi/4)
ax.set_xticks([0, np.pi/4])

plt.show();

And another one:

import numpy as np
import matplotlib.pyplot as plt


X,Y = np.meshgrid(np.linspace(-3, 3, 200),
                  np.linspace(-3, 3, 200))
Z =(1-X/2+X**5+Y**3)*np.exp(-X**2-Y**2)

fig = plt.figure(figsize=(10,10), dpi=300)

# Regular imshow
ax = plt.subplot(1, 3, 1)
ax.imshow(Z)
ax.set_xticks([])
ax.set_yticks([])

# Full polar "imshow" (pcolormesh)
ax = plt.subplot(1, 3, 2, projection="polar")
ax.pcolormesh(np.linspace(0, 2*np.pi, 200),
              np.linspace(0, 1, 200), Z[::-1,::-1], shading='auto')
ax.set_rticks([])
ax.set_xticks([])

# Partial polar "imshow" (pcolormesh)
ax = plt.subplot(1, 3, 3, projection="polar")
ax.pcolormesh(np.linspace(0, np.pi/2, 200),
              np.linspace(0, 1, 200), Z[::-1,::-1], shading='auto')
ax.set_rticks([])
ax.set_xticks([])
ax.set_rmin(0); ax.set_rmax(1)
ax.set_rorigin(-.5)
ax.set_thetamin(0)
ax.set_thetamax(90)

plt.show();

ViNOJ-DAViS · 2021-09-03T11:03:49Z

Hi,
I am unable to have same polar effect for an scene.visuals.Image
How can I achieve same polar effect similar to ImageVisual

djhoese · 2021-09-03T14:39:29Z

@ViNOJ-DAViS Image and ImageVisual are the same Visual code. They both even have a .transforms property although I forget every difference involved here. To get the functionality that @kmuehlbauer has demonstrated in these comments, you should theoretically only have to add an STTransform or two to the chain of transforms being made to handle the "Visual" <-> "Screen" coordinate conversion. If you'd like more help on this please file a new issue and reference this one. Be sure to include a minimal example of your code and what you've tried and what isn't working. Thanks.

ENH: Enhance ImageVisual woth polar capabilities, add example

6b89946

WIP: rework shader code, adapt naming nomenclature, improve example

7d43900

kmuehlbauer added 2 commits July 7, 2021 15:17

WIP: minor fixes ImageVisual/GridlineVisual

f4e702b

WIP: flakify

1144e2e

almarklein reviewed Jul 8, 2021

View reviewed changes

Comment thread vispy/visuals/image.py Outdated

Comment thread vispy/visuals/image.py Outdated

Comment thread vispy/visuals/image.py Outdated

kmuehlbauer added 3 commits July 9, 2021 15:38

WIP: working version, need to simplify now

6e22c64

WIP: working version, simplifying

c2fbd17

WIP: working version, simplifying

5fd8915

kmuehlbauer requested review from almarklein and djhoese July 11, 2021 13:26

kmuehlbauer self-assigned this Jul 12, 2021

kmuehlbauer added component: visuals type: enhancement type: example labels Jul 12, 2021

kmuehlbauer added this to the Version 0.8.0 milestone Jul 12, 2021

kmuehlbauer removed this from the Version 0.8.0 milestone Jul 15, 2021

kmuehlbauer changed the title ~~ENH: Enhance ImageVisual with polar capabilities, add example~~ WIP: ENH: Enhance ImageVisual with polar capabilities, add example Jul 15, 2021

kmuehlbauer marked this pull request as draft July 15, 2021 09:06

ViNOJ-DAViS mentioned this pull request Sep 3, 2021

Polar Transformation not working for scene.visuals.Image #2210

Closed

Conversation

kmuehlbauer commented Jul 6, 2021

Uh oh!

kmuehlbauer commented Jul 6, 2021

Uh oh!

kmuehlbauer commented Jul 7, 2021

Uh oh!

almarklein left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kmuehlbauer commented Jul 8, 2021

Uh oh!

kmuehlbauer commented Jul 9, 2021

Uh oh!

kmuehlbauer commented Jul 11, 2021

Uh oh!

almarklein commented Jul 12, 2021

Uh oh!

djhoese commented Jul 12, 2021

Uh oh!

kmuehlbauer commented Jul 13, 2021

Uh oh!

kmuehlbauer commented Jul 13, 2021

Uh oh!

djhoese commented Jul 13, 2021

Uh oh!

kmuehlbauer commented Jul 13, 2021

Uh oh!

kmuehlbauer commented Jul 14, 2021

Uh oh!

kmuehlbauer commented Jul 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

djhoese commented Jul 14, 2021

Uh oh!

kmuehlbauer commented Jul 14, 2021

Uh oh!

djhoese commented Jul 14, 2021

Uh oh!

kmuehlbauer commented Jul 15, 2021

Uh oh!

kmuehlbauer commented Aug 18, 2021

Uh oh!

rougier commented Aug 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ViNOJ-DAViS commented Sep 3, 2021

Uh oh!

djhoese commented Sep 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kmuehlbauer commented Jul 14, 2021 •

edited

Loading

rougier commented Aug 24, 2021 •

edited

Loading