Introduce PixelFormat API
#15
Conversation
| extension Colorspace { | ||
| /* Note: Despite the unconventional bitmap configuration settings, the resulting snapshots produce correctly rendered Lottie frames. | ||
|
|
||
| Typically, ARGB should use a big endian format since the red value follows the alpha at the smallest memory address. | ||
| Conversely, ABGR should utilize a little endian format with `premultipliedFirst` due to blue being the first color channel following alpha. | ||
|
|
||
| However, results from the snapshots indicate correct visual output, suggesting a potential anomaly in iOS’s handling of memory format. | ||
| Verification confirms that ThorVG is outputting buffers in the correct ARGB/ABGR format, pointing to an iOS-specific issue in interpreting these formats. | ||
| */ | ||
| var bitmapInfo: CGBitmapInfo { | ||
| switch self { | ||
| case .argb: | ||
| return [.byteOrder32Little, CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue)] | ||
| case .abgr: | ||
| return [.byteOrder32Big, CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue)] | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Just a note here: we've found a weird bug with iOS in aligning the pixel format when trying to create a CGBitmapInfo for our CGContext when taking screenshots. I will continue to investigate this further and create a bug report with Apple if neccessary.
There was a problem hiding this comment.
The Core Graphics logic seems fine to me. For our use case, you can think of the CGBitmapInfo as a way to interpret the color information stored in the buffer. Since we are using RGB, here’s how the different values of CGImageAlphaInfo will behave for .byteOrder32Little:
none: RGB
first: ARGB
last: RGBA
alphaOnly: A
noneSkipFirst: _RGB // _ means to ignore whatever value is in there
noneSkipLast: RGB_
For .byteOrder32Big, the result will be the same values as before, but reversed.
So, to summarize, if you want to create a CGBitmapInfo for a particular pixel format (other than only A), you can follow these steps:
- Are the RGB components in that order (R first and B last) in the pixel format? If yes, use
.byteOrder32Little, otherwise use.byteOrder32Big. - If the RGB components are not in that same order in the pixel format, reverse the pixel format.
- Is the alpha at the start of the pixel format obtained in step 2? If yes, use
.premultipliedFirst, otherwise use.premultipliedLast.
There was a problem hiding this comment.
Thanks for such a detailed comment @rlaguilar. I really appreciate how you take the time to walk us through how these conventions work.
I think the core thing I was missing was that iOS follows the little-endian format. Which means that reading a buffer of color values would require us to read it in .byteOrder32Little to get the RGB order. Which now makes more sense to me.
And I love the steps! Will save those as a guide that I will definitely need to come back to in the future. 😅
Have removed the comment 👍
| extension Colorspace { | ||
| /* Note: Despite the unconventional bitmap configuration settings, the resulting snapshots produce correctly rendered Lottie frames. | ||
|
|
||
| Typically, ARGB should use a big endian format since the red value follows the alpha at the smallest memory address. | ||
| Conversely, ABGR should utilize a little endian format with `premultipliedFirst` due to blue being the first color channel following alpha. | ||
|
|
||
| However, results from the snapshots indicate correct visual output, suggesting a potential anomaly in iOS’s handling of memory format. | ||
| Verification confirms that ThorVG is outputting buffers in the correct ARGB/ABGR format, pointing to an iOS-specific issue in interpreting these formats. | ||
| */ | ||
| var bitmapInfo: CGBitmapInfo { | ||
| switch self { | ||
| case .argb: | ||
| return [.byteOrder32Little, CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedFirst.rawValue)] | ||
| case .abgr: | ||
| return [.byteOrder32Big, CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue)] | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
The Core Graphics logic seems fine to me. For our use case, you can think of the CGBitmapInfo as a way to interpret the color information stored in the buffer. Since we are using RGB, here’s how the different values of CGImageAlphaInfo will behave for .byteOrder32Little:
none: RGB
first: ARGB
last: RGBA
alphaOnly: A
noneSkipFirst: _RGB // _ means to ignore whatever value is in there
noneSkipLast: RGB_
For .byteOrder32Big, the result will be the same values as before, but reversed.
So, to summarize, if you want to create a CGBitmapInfo for a particular pixel format (other than only A), you can follow these steps:
- Are the RGB components in that order (R first and B last) in the pixel format? If yes, use
.byteOrder32Little, otherwise use.byteOrder32Big. - If the RGB components are not in that same order in the pixel format, reverse the pixel format.
- Is the alpha at the start of the pixel format obtained in step 2? If yes, use
.premultipliedFirst, otherwise use.premultipliedLast.
swift/Colorspace.swift
Outdated
| /// Specifiies the methods of combining the 8-bit color channels into 32-bit color. | ||
| public enum Colorspace { | ||
| /// The channels are joined in the order: alpha, blue, green, red. Colors are alpha-premultiplied. (a << 24 | b << 16 | g << 8 | r) | ||
| case abgr | ||
| /// The channels are joined in the order: alpha, red, green, blue. Colors are alpha-premultiplied. (a << 24 | r << 16 | g << 8 | b) | ||
| case argb | ||
| } |
There was a problem hiding this comment.
These are not really color spaces but pixel formats. Also, you want to use the term component instead of channel. A channel refers to the set of all the values, for example, saying that an image has an alpha channel means that all the pixels have transparency information associated to them.
So, when you say:
The channels are joined in the order: alpha, red, green, blue
That really means that somehow an image is stored in a way that all the alpha values come first, then all the red, and so on.
There was a problem hiding this comment.
That's a fair call. I've gotten swept up in the ThorVG naming without actually questioning whether it is a good name!
I'll update the naming to move to PixelFormat.
ray-canva
left a comment
There was a problem hiding this comment.
LGTM, thank Rey and Andy
This PR introduces the
PixelFormatenum type, which internally wraps theTvg_Colorspacetype.Introduce
PixelFormatAPIThese changes introduce the following additions to the public API.
This allows consumers to use different pixel formats if they wish, determining the order or the color components for each pixel value.