Bringing Wide Color to PhotoEditor SDK


3 min read
Bringing Wide Color to PhotoEditor SDK

Inspired by Mike Krieger’s great post on supporting wide color images in Instagram, I decided to challenge myself by introducing the same wide color support into our PhotoEditor SDK within a day.

Mike did an excellent job describing everything that is needed to support wide color images so I am not going to reiterate his explanation. Instead, here I’m going to share two findings I made in the process.

Image Export

Mike suggests that in order to preserve the color space while converting an UIImage into a JPEG, the UIImageJPEGRepresentation(_:_:) method has to be replaced with the new UIGraphicsImageRenderer.jpegData(withCompressionQuality:actions:) method. Checking the color profile of the generated JPEG by UIImageJPEGRepresentation(_:) I noticed that it seemed unnecessary and resulting in more work than needed. To verify this, I started Hopper and took a look at the internals of the new method. Here’s what it basically does:

  1. It passes actions to UIGraphicsRenderer.runDrawingActions(_:completionActions:).
  2. It obtains the resulting image using UIGraphicsImageRendererContext.currentImage.
  3. It passes that image to UIImageJPEGRepresentation(_:_:).

As you can see, it makes use of UIImageJPEGRepresentation(_:_:) internally too, so there really is no need to go the extra mile. If you are actually doing any drawing within the actions block using wide color UIColors it absolutely makes sense to use Mike’s apoproach. But if you only want to convert an UIImage into a JPEG image, using the old method is far easier.

Core Image

We are using CIImage, CIFilter and CIContext for most of our processing. In our live preview we use a CIContext to directly render a CIImage into a GLKView and I’ve run into exactly the same problem as Mike has — namely not being able to get that view to be color space-aware. Using the offscreen buffer workaround that he suggested was going to be too much work though and also I didn’t want to sacrifice any performance.

After doing some research and digging further into the GLKit and Core Image assembly I found a nice solution. It looks like in order to render wide color images into OpenGL, OpenGLES 3 and a GLKViewDrawableColorFormat (that is currently not defined as an enum case) has to be used. The following code works like a charm for me:

let api: EAGLRenderingAPI
let colorFormat: GLKViewDrawableColorFormat
 
if #available(iOS 10.0, *) {
  if UIScreen.main.traitCollection.displayGamut == .P3 {
    api = .openGLES3
    colorFormat = GLKViewDrawableColorFormat(rawValue: 10)!
  } else {
    api = .openGLES2
    colorFormat = .RGBA8888
  }
} else {
  api = .openGLES2
  colorFormat = .RGBA8888
}

let context = EAGLContext(api: api)
let previewView = GLKView(frame: CGRect.zero, context: context!)
previewView.delegate = self
previewView.drawableColorFormat = colorFormat

This code creates a GLKView that is backed by a CAEAGLLayer with the following drawableProperties set:

[
 "EAGLDrawablePropertyRetained": 0,
 "EAGLDrawablePropertyColorFormat": EAGLColorFormatRGBA_XR10_64BPP
]

Unfortunately EAGLColorFormatRGBA_XR10_64BPP also doesn’t seem to be documented at this point, but my guess is that setting this color format on Instagram’s EAGLView would also solve their problem of making the view color space-aware.

Thanks to Core Image we can just switch to OpenGLES 3 without having to do any other modifications to our code. However, I am currently not sure if this is considered a private API use. I would greatly appreciate if anyone could shed some light on this.

I have not yet been able to test this on any device other than the iPhone 7 Plus, but I will update this post if I run into any problems with future tests.

Final Thoughts

Getting an app to support wide color images is definitely worth it and not as difficult as I had imagined. Unfortunately, it currently looks like iOS is still missing (public) support for wide color in some parts of its frameworks. I hope that Apple addresses these problems soon. Nevertheless we’re optimistic that we’ll be able to include wide color image support in the next release of our SDK. Feel free to check out our demo app in the App Store which we will update as soon as possible.

Related Articles

How To Add Text to Video in Swift
4 min read
How to Trim and Crop Video in Swift
6 min read
How to Create Image Filters for iOS
7 min read
How to Draw on an Image With JavaScript
7 min read
How To Resize an Image in React
6 min read

We’d love to use cookies

Help us improve our Website
We use cookies to provide you with a user-friendly website and for marketing. By clicking accept you agree to the terms of our privacy policy, and help us improve our web presence.

We’d love to use cookies

Help us improve our Website
We use cookies to provide you with a user-friendly website and for marketing. By clicking accept you agree to the terms of our privacy policy, and help us improve our web presence.
Essential Cookies
..are necessary for you to browse our website and use its features, such as accessing secure areas of the site. These cookies are first-party session cookies, and do not provide any information that would make you identifiable.
Functional Cookies
..allow our website to remember choices you have made in the past, like what language you prefer, or what your user name is so you can log in more easily. Generally speaking, they enable us to provide you with a more convenient experience of our website.
Performance Cookies
..collect information about how you use our website, like which pages you visited and which links you clicked on. These cookies are all aggregated and, therefore, anonymized. None of this information can be used to identify you. It is their sole purpose to improve website functions. This includes cookies from third-party analytics services which are for our exclusive use.
Marketing Cookies
..track your online activity and make you identifiable to us. They help us deliver more relevant content to you and therefore improve your user journey. These cookies can share that information with other organizations or advertisers.

GO TOP